Documentation
¶
Index ¶
Constants ¶
const ( StatePending = "Pending" StateFailed = "Failed" StateSuccess = "Success" StateInProgress = "InProgress" )
The available states of a job's DownloadState/CallbackState.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Aggregation ¶
type Aggregation struct {
ID string `json:"aggr_id"`
// Maximum numbers of concurrent download requests
Limit int `json:"aggr_limit"`
// Proxy url for the client to use, optional
Proxy string `json:"aggr_proxy"`
}
Aggregation is the concept through which the rate limit rules are defined and enforced.
func NewAggregation ¶
func NewAggregation(id string, limit int, proxy string) (*Aggregation, error)
NewAggregation creates an aggregation with the provided ID and limit. If any of the prerequisites fail, an error is returned.
func (*Aggregation) UnmarshalJSON ¶
func (a *Aggregation) UnmarshalJSON(b []byte) error
UnmarshalJSON populates the aggregation with the values in the provided JSON.
type Callback ¶ added in v0.1.0
type Callback struct {
// Success refers to whether a job download was successful or not
Success bool `json:"success"`
// Error contains errors that occured during a job download
Error string `json:"error"`
// Extra are opaque/pass through data
Extra string `json:"extra"`
// ResourceURL is the url of the requested for download resource
ResourceURL string `json:"resource_url"`
// DownlaodURL the url where the downloaded resource resides
DownloadURL string `json:"download_url"`
// JobID is the unique id of a Job
JobID string `json:"job_id"`
// ResponseCode is the http response for the downloaded resource e.g 200, 404
ResponseCode int `json:"response_code"`
// Delivered signifies where the callback has been delivered or not
Delivered bool `json:"delivered"`
// DeliveryError contains the error occured while delivering a callback
DeliveryError string `json:"delivery_error"`
}
Callback holds info to be posted back to the provided callback destination.
type Job ¶
type Job struct {
// Auto-generated
ID string `json:"-"`
// The URL pointing to the resource to be downloaded
URL string `json:"url"`
// AggrID is the ID of the aggregation the job belongs to.
//
// TODO: should this be a pointer to an Aggregation? If so, then
// NewJob should be a function on Aggregation
AggrID string `json:"aggr_id"`
DownloadState State `json:"-"`
// How many times the download request was attempted
DownloadCount int `json:"-"`
// Auxiliary ad-hoc information. Typically used for communicating
// download errors back to the user.
DownloadMeta string `json:"-"`
CallbackState State `json:"-"`
CallbackType string `json:"callback_type"`
CallbackDst string `json:"callback_dst"`
// TODO: Remove CallbackURL, in favor of CallbackType and CallbackDst, after
// all users of Downloader have upgraded their request scheme.
CallbackURL string `json:"callback_url"`
// Auxiliary ad-hoc information used for debugging.
CallbackMeta string `json:"-"`
// How many times the callback request was attempted
CallbackCount int `json:"-"`
// Arbitrary info provided by the user that are posted
// back during the callback
Extra string `json:"extra"`
// Response code of the download request
ResponseCode int `json:"response_code"`
// Mime type pattern provided by the client
MimeType string `json:"mime_type"`
// Http client timeout for download in seconds
DownloadTimeout int `json:"download_timeout"`
// The HTTP request headers provided by the user will be used for
// downloading files.
// This attribute is optional.
RequestHeaders map[string]string `json:"request_headers,omitempty"`
}
Job represents a user request for downloading a resource.
It is the core entity of the downloader and holds all info and state of the download.
TODO: should this be valid with an empty aggregation id?
func (*Job) CallbackInfo ¶ added in v0.1.0
CallbackInfo validates the state of a job and returns a callback info along with an error if appropriate. The expected argument downloadURL is the base path of a downloaded resource in the downloader.
func (*Job) UnmarshalJSON ¶
UnmarshalJSON is used to populate a job from the values in the provided JSON message.