Sarama config parser from Viper. It means it supports out of the box all the formats provided by Viper (env, yaml, toml, etc.) and most settings in Sarama.
Sometimes is difficult to parse all the config settings from Sarama from env variables or configuration files. Following some guidelines, this package will help you to get up & running Sarama configuration from your favourite config system.
-
The keys should be the same than the Sarama config file. Just transform they fields into a nested structure with lowercase names separated by a dot per level. Example:
Versionwould be justversionandAdmin -> Retry -> Maxwould beadmin.retry.max. -
Strings, ints, boolean just works! Durations can be specified with parsed durations like
1msor3s(stdtime.ParseDurationwill be used under the hood). -
Arrays can be specified joining the values with commas. Per example
1,2,5would be a valid int array. -
In general, functions (i.e. retry backoff) and interfaces (i.e. proxy dialer),will not be parsed. You can setup any of them with the Sarama config got from the package.
-
There are a few magic exceptions from the rule above.
-
versionwill be parsed as string (not the decomposed [4]int) following the Sarama rules (sarama.ParseKafkaVersionfunction will be used). -
net.sasl.scramclientgeneratorfuncwill be parsed as string (with the possible values SHA256 and SHA512) implementing a generic XDG Scram client using the hash generator you specified. -
tls.ca,tls.clientpemandtls.clientkeyare special keys used to provide certificates and the client key to configure TLS. These certificates should be encoded in base64 encoding. Don't forget to enable TLS using the regular convention:net.tls.enable = true. Check the example to see how to configure TLS.
-
-
Choose your config system and write (or fill) the configuration settings, following the previous rules.
-
Setup the viper configuration, and make sure is loaded with
viper.AutomaticEnv()(env) oviper.ReadInConfig()(files). For example, for aconf.tomlwould be:
viper.AddConfigPath(".")
viper.SetConfigName("conf")
viper.SetConfigType("toml")
if err := viper.ReadInConfig(); err != nil {
panic(err)
}- Call the package with the viper you have just configured and check the error.
cfg, err := saramaconfig.NewFromViper(viper.GetViper())
if err != nil {
panic(err)
}- Just use
cfgas a regular Sarama config to run producers, consumers, etc.
There are some examples to demonstrate how to work with the package. The tests also can describe some of the behaviour of the package, check them!
- Go doc.
- Now, if you provide a wrong key, that would not be parsed but it won't throw any error. It would be nice to report an error and maybe to advise the right key.
- Implement more magic non parseable keys, like functions or interfaces. For example,
Net -> Proxy -> Dialerwhich is aproxy.Dialer, and could be set up implementing a struct with some proxy details provided in the config.