[JustKiddingCode]

Allows to buffer flows in memory and (additionally) on disk in zstd compressed files.

Add a small segment (filegate) for testing purposes to pause the pipeline when a file exists.

[georg-e]

Hi, thank you for the PR. I took a first look on it and have some comments and remarks to be adressed. Maybe you have already identified some of my points and maybe we need some more detailed testing and validation of this feature.

• General structure and categorization:

  • i think the filegate segments suits better to the already existing testing directory/category instead of a new category like dev.
  • the new diskbuffer segment will be better categorized within the controlflow category. i think its rather an offloading and manipulation of the control-flow in the current flowpipeline instance
  • if possible, it would be nice to add some test cases to the segment. To start maybe a simple "passthrough" test for knowing, that the segment works and passes through flows. For an example please find pass_test.go

• Diskbuffer segment

  • diskbuffer.go line 89 is probably reading the wrong configuration variable
  • it looks like data is still written to disk after filesize is reached. (after debug log in line 298 was triggered)