cache

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {

	// Validity is the duration for which the cache is valid.
	Validity time.Duration
	// Jitter is the jitter to apply when considering a cached entry valid or not.
	Jitter time.Duration
	// contains filtered or unexported fields
}

Config holds the cache configuration.

func NewConfig

func NewConfig() Config

NewConfig is the constructor for Config.

func (*Config) IsSet

func (c *Config) IsSet() bool

IsSet tell whether a cache configuration is present.

func (*Config) UnmarshalYAML

func (c *Config) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML puts the unmarshalled yaml data into the internal cache parser struct. This prevents access to the string data of jitter and validity.

type SQLite3Storage

type SQLite3Storage struct {
	// SQLite filename.
	Filename string
	// Duration till which a link is skipped.
	Validity time.Duration
	// Clear cache at start if true.
	ClearCache bool
	// Jitter is used to add jitter when checking cache validity. 0 by default.
	Jitter time.Duration
	// contains filtered or unexported fields
}

Sqlite3Storage implements a SQLite3 caching backend for Colly.

func (*SQLite3Storage) CacheURL

func (s *SQLite3Storage) CacheURL(URL string) error

CacheURL inserts new URL into cache database.

func (*SQLite3Storage) Clear

func (s *SQLite3Storage) Clear() error

Clear removes all entries from cache.

func (*SQLite3Storage) Close

func (s *SQLite3Storage) Close() error

Close cache database.

func (*SQLite3Storage) DeleteURL

func (s *SQLite3Storage) DeleteURL(URL string) error

DeleteURL deletes a URL from cache database.

func (*SQLite3Storage) Init

func (s *SQLite3Storage) Init(validity time.Duration, jitter time.Duration) error

Init initializes cache database.

func (*SQLite3Storage) IsCached

func (s *SQLite3Storage) IsCached(URL string) (bool, error)

IsCached checks if URL has already been visited.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址