Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

4 Commits

Folders and files

Repository files navigation

Monitorink

Request a URL processes the results by a pipeline:

The Producer Pipeline

  1. Produce a scheduled http request to a configured URL
  2. Process the response by calculating the response time
  3. Push the processed data to a provided consumer ( in memeory queue , Apache Kafka etc.)

The Consumer Pipeline

  1. Consumes pushed messages from the Producer pipeline
  2. Persists them into any configured store ( in memory store, Prostgres, etc.)

Packages & Directories

Files

  • docker-compose.yml: Runs zoekeeper, kafka and postgres locally.
  • env.sh: environment variables needed to run the system.
  • Makefile: some targets to setup and run the application

Testing

$ make test

Running

  1. Requirements: docker, docker-compose, >=go1.14, golangci-lint
  2. Update env.sh contents with the appropriate settings
  3. $ make run-infra
  4. $ make migrate
  5. $ make run-producer
  6. $ make run-consumer

Todos

  • Extend tests
  • Add regex matcher stage to pipeline
  • Improve offset handling in consumer pipelin
  • Improve error handling
  • Containerize produce and consumer
  • Add cloud provider support (Aiven, Confluent etc.) for queue and store
  • Support multiple url registrations
  • Create kubernetes deployment manifests

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages

⌂ 换网址