Documentation
¶
Overview ¶
Package folders watches configured paths for archives to extract.
Index ¶
- Constants
- Variables
- func NormalizeExcludePaths(basePath string, excludes []string) []string
- func NormalizeWaitExtensions(exts []string) []string
- func PathContains(watch, item string) bool
- func ValidateList(list []*FolderConfig, parseMax func(string) (uint64, bool, error)) error
- func WaitFileInTop(path string, exts []string) string
- type Event
- type Folder
- type FolderConfig
- type Folders
- func (f *Folders) Add(folder string) error
- func (f *Folders) Close()
- func (f *Folders) FSNotifyPaths() []string
- func (f *Folders) PollerSummaries() []string
- func (f *Folders) ProcessEvent(event *Event, now time.Time)
- func (f *Folders) Remove(folder string)
- func (f *Folders) StartPollers()
- func (f *Folders) WatchFSNotify()
- type Logs
- type WatchConfig
Constants ¶
const ( MinimumPollInterval = 5 * time.Millisecond DefaultDeleteAfter = 10 * time.Minute KindPolling = "polling" KindFSNotify = "fsnotify" )
Variables ¶
var ErrNilConfig = errors.New("nil config entry")
ErrNilConfig is returned when a folder list contains a nil entry.
var ErrNoPath = errors.New("path is required")
ErrNoPath is returned when a folder instance has no watch path.
Functions ¶
func NormalizeExcludePaths ¶
NormalizeExcludePaths expands and cleans exclude paths relative to a watch folder.
func NormalizeWaitExtensions ¶
NormalizeWaitExtensions trims empty values and forces a leading period.
func PathContains ¶
PathContains reports whether item is watch or a child of watch. Root paths (`/` and Windows volume roots) keep their trailing separator. Windows folds case without probing the filesystem (ReFS and some network mounts can be case-sensitive).
func ValidateList ¶
ValidateList applies folder defaults and parses max_bytes.
func WaitFileInTop ¶
WaitFileInTop returns the first non-directory name in path that ends with a wait extension. It does not walk subfolders.
Types ¶
type Event ¶
type Event struct {
Config *FolderConfig
Name string
File string
Op string
}
Event is a filesystem event for a watched folder.
type Folder ¶
type Folder struct {
// Updated is last write or status change. Start delay, retry delay,
// delete after, and the wait-extension ReadDir skip all use this.
Updated time.Time
// WaitFile is a top-level name matching wait_extensions (.part, etc).
// Empty means extraction is not blocked on an incomplete download.
WaitFile string
// Status is the extract lifecycle for this item.
Status extract.Status
// Config is the watch-root settings this item belongs to.
Config *FolderConfig
// Files are extracted output paths (xtractr NewFiles). Used with delete_files.
Files []string
// Retries is how many times a failed extract has been started again.
Retries uint
// Archives are the input archives xtractr found. Used with delete_original.
Archives xtractr.ArchiveList
// PreFiles is the snapshot of each archive dest before extraction
// (MoveBack only). Dest folders come from FindCompressedFiles so nested
// archive dirs are included. Kept across retries so failed cleanups are
// not recaptured as download content. Nil means remnant handling is skipped.
PreFiles map[string]os.FileInfo
// NoRetry is set when remnant_action=off leaves a blocker.
NoRetry bool
}
Folder is a tracked archive or directory inside a watch path.
type FolderConfig ¶
type FolderConfig struct {
DeleteOrig bool `json:"delete_original" toml:"delete_original" xml:"delete_original" yaml:"delete_original"`
DeleteFiles bool `json:"delete_files" toml:"delete_files" xml:"delete_files" yaml:"delete_files"`
DisableLog bool `json:"disable_log" toml:"disable_log" xml:"disable_log" yaml:"disable_log"`
MoveBack bool `json:"move_back" toml:"move_back" xml:"move_back" yaml:"move_back"`
DeleteAfter *cnfg.Duration `json:"delete_after" toml:"delete_after" xml:"delete_after" yaml:"delete_after"`
ExtractPath string `json:"extract_path" toml:"extract_path" xml:"extract_path" yaml:"extract_path"`
ExtractISOs bool `json:"extract_isos" toml:"extract_isos" xml:"extract_isos" yaml:"extract_isos"`
PreserveExec bool `json:"preserve_exec" toml:"preserve_exec" xml:"preserve_exec" yaml:"preserve_exec"`
DisableRecursion bool `json:"disableRecursion" toml:"disable_recursion" xml:"disable_recursion" yaml:"disableRecursion"`
MaxNested int `json:"maxNested" toml:"max_nested" xml:"max_nested" yaml:"maxNested"`
ExtrasMaxDepth int `json:"extrasMaxDepth" toml:"extras_max_depth" xml:"extras_max_depth" yaml:"extrasMaxDepth"`
AllowSymlinks bool `json:"allowSymlinks" toml:"allow_symlinks" xml:"allow_symlinks" yaml:"allowSymlinks"`
MaxBytes string `json:"maxBytes" toml:"max_bytes" xml:"max_bytes" yaml:"maxBytes"`
MaxFiles int `json:"maxFiles" toml:"max_files" xml:"max_files" yaml:"maxFiles"`
MaxRatio float64 `json:"maxRatio" toml:"max_ratio" xml:"max_ratio" yaml:"maxRatio"`
// ResolvedMaxBytes is 0 when unset: folder watcher is uncapped.
ResolvedMaxBytes uint64 `json:"-" toml:"-" xml:"-" yaml:"-"`
ExcludePaths []string `json:"exclude_paths" toml:"exclude_paths" xml:"exclude_path" yaml:"exclude_paths"`
Interval cnfg.Duration `json:"interval" toml:"interval" xml:"interval" yaml:"interval"`
WaitExtensions []string `json:"wait_extensions" toml:"wait_extensions" xml:"wait_extension" yaml:"wait_extensions"`
SkipEmpty bool `json:"skip_empty" toml:"skip_empty" xml:"skip_empty" yaml:"skip_empty"`
Path string `json:"path" toml:"path" xml:"path" yaml:"path"`
}
FolderConfig defines the input data for a watched folder.
func Check ¶
func Check(list []*FolderConfig, log Logs) ([]*FolderConfig, []string)
Check stats all configured folders and returns only "good" ones.
func CloneList ¶
func CloneList(src []*FolderConfig) []*FolderConfig
CloneList copies folder configs without sharing DeleteAfter, ExcludePaths, or WaitExtensions.
func (*FolderConfig) IsExcludedPath ¶
func (c *FolderConfig) IsExcludedPath(path string) bool
IsExcludedPath returns true if path is the exclude or a child of it.
func (*FolderConfig) UsesPoller ¶
func (c *FolderConfig) UsesPoller() bool
UsesPoller is true when this folder has its own radovskyb poller.
type Folders ¶
type Folders struct {
Logs
Config []*FolderConfig
Folders map[string]*Folder
Events chan *Event
Updates chan *xtractr.Response
FSNotify *fsnotify.Watcher
IgnoreSuffix string
// contains filtered or unexported fields
}
Folders holds all known (created) folders in all watch paths.
func (*Folders) Close ¶
func (f *Folders) Close()
Close stops pollers and fsnotify. Safe for tests; the daemon does not call this.
func (*Folders) FSNotifyPaths ¶
FSNotifyPaths are watch roots that use filesystem events, not a poller.
func (*Folders) PollerSummaries ¶
PollerSummaries lists each poller as "path @ interval" for startup logs.
func (*Folders) ProcessEvent ¶
ProcessEvent processes the event that was received.
func (*Folders) StartPollers ¶
func (f *Folders) StartPollers()
StartPollers starts one radovskyb watcher per polled folder.
func (*Folders) WatchFSNotify ¶
func (f *Folders) WatchFSNotify()
WatchFSNotify reads file system events from a channel and processes them. This runs in its own go routine, and eventually sends the event back into the main routine.
type Logs ¶
type Logs interface {
Printf(msg string, v ...any)
Errorf(msg string, v ...any)
Debugf(msg string, v ...any)
}
Logs interface for folders.
type WatchConfig ¶
type WatchConfig struct {
Buffer uint `json:"buffer" toml:"buffer" xml:"buffer" yaml:"buffer"`
}
WatchConfig is the undocumented folders event-buffer setting.
func (WatchConfig) NewWatcher ¶
func (c WatchConfig) NewWatcher( folderConfig []*FolderConfig, logger Logs, updateBuf int, ignoreSuffix string, ) (*Folders, error)
NewWatcher returns a new folder watcher. Call Close() when you are done with it (tests). The daemon leaves it open.