Repository navigation
Add option to sync from beginning of log file #83
Description
From the Markdown:
"remote_syslog automatically detects and activates new log files that match its file specifiers. For example, *.log may be provided as a file specifier, and remote_syslog will detect a some.log file created after it was started. Globs are re-checked every 10 seconds.
Note: messages may be written to files in the 0-10 seconds between when the file is created and when the periodic glob check detects it. This data is not acted on."
It would be useful to tell the system to seek to the beginning of the new log file instead of the end. In best case, no log lines are dropped. In the worse case there are several duplicates which can either be uniqued or run through an idempotent back-end.
IMO the downside of possible is dupes is worth not dropping any data in the common case of log rotation.
Activity
bradvido commentedon Nov 23, 2015
+1
It would be ideal if the state was persisted as well, meaning only start from the beginning once. In the case that the
remote_syslog2 serviceis restarted after it has tailed some files from the beginning, I wouldn't want it to restart at the beginning of all the files all over again.For reference, the
ruby-filewatchfile-tailing library has a concept of asincedbwhich tracks the position tailed in the file and also handles rotation gracefully: https://github.com/jordansissel/ruby-filewatchjbardin commentedon Jan 13, 2016
IMHO, this should be the default. I don't think the expected behavior would be to lose the start of every new log file in directories that are being watched.