ilyakaznacheev / cleanenv Goto Github PK
View Code? Open in Web Editor NEW✨Clean and minimalistic environment configuration reader for Golang
License: MIT License
✨Clean and minimalistic environment configuration reader for Golang
License: MIT License
Write a Godoc example test to show how to use all of the supported library tags.
I want to use embed config files. But ReadConfig()
only use file path.
If I can use io.Reader
for parseFile()
like below, it would be nice. What do you think?
Or, If there is any way to use embed files, please let me know.
func ReadConfigFromReader(r io.Reader, cfg interface{}) {
err := parseFile(r, cfg)
if err != nil {
return err
}
return readEnvVars(cfg, false)
}
func ReadConfig(path string, cfg interface{}) {
f, err := os.OpenFile(path, os.O_RDONLY|os.O_SYNC, 0)
if err != nil {
return err
}
defer f.Close()
err := parseFile(f, cfg)
if err != nil {
return err
}
return readEnvVars(cfg, false)
}
This is how I'm simply reading the config file:
func Read(path string) {
readError := cleanenv.ReadConfig(path, &App)
if readError != nil {
log.Panic("Failed to read yaml file", readError)
}
}
This is working on a my local machine (Mac) and in Docker container running locally.
But when I run the app on EC2 (built as RPM package and ran as a service), I get this error:
Process: 20421 ExecStart=/usr/bin/store-service -f /etc/store-service/config.yml (code=exited, status=2)
The very same code runs well when I parse the config using a library like jessevdk/go-flags
.
Any idea?
Hello!
Nice library.
Is there avaiable to mark config parameter as required and get error if it is not set?
It's obvious now that we don't want to support very old versions and at the same time, we want to use new stuff released with newer versions of Go.
I really want to make it easier for everyone to use the library even with older projects, so I'll make it formal and describe the number of supported versions and add them to the test version matrix.
I realized that a SetValue()
defined on a custom struct type is never called. Insted, the struct is flattened. This prevents me from parsing e.g., a yaml string from an ENV variable:
type Config struct {
Complex ComplexYamlConfig `env:"COMPLEX_CONFIG"`
}
type ComplexYamlConfig struct {
Left string `yaml:"myProp"`
}
func (c *ComplexYamlConfig) SetValue(s string) error {
// will never be called by cleanenv
return yaml.Unmarshal([]byte(s), c)
}
Is this something that can be added? Or is there a different way of achieving the same thing?
I'm trying to fill a required boolean field in the JSON file, and when the value is false
, the following error raises:
field "boolVarName" is required but the value is not provided
.
But when I change the value to true
, everything goes well.
I think the issue is in the condition of line 316 of cleanenv.go
, or the condition of line 329 where you check meta.isFieldValueZero()
.
Thank you for this solution. I implemented it and was running in minutes.
Question: I am running on Linux and for this web service project I have found that my app only reads the config.yml
if I launch the service from the same directory as the web service (where the config file resides as well). This works fine:
cd \usr\bin\myapp
myapp &
This does not work:
\usr\bin\myapp\myapp &
I am using the recommended code:
err := cleanenv.ReadConfig("config.yml", &cfg)
I'd like to be able to start the service in the background and have it read the config from the execution directory. (Or is there a better method? Should I pass the config file location as a cmd line parameter?) Sorry if this is basic, I am new to Linux development.
Thanks again for your help.
I was going to use viper and then stumbled on this... Want to understand the differences to make a choice.
For example, I have the following Config root
var RootConfig Root
type Root struct {
Http map[string]HttpConfig `yaml:"http" env:"HTTP"`
}
type HttpConfig struct {
Host string `yaml:"host"`
TimeoutMs int `yaml:"timeout_ms"`
}
func Load(configPath string) {
cleanenv.ReadConfig(configPath, &RootConfig)
}
I know I can provide the following to populate http
key via the following yaml
structure:
http:
payment:
host: "http://payment:4000"
timeout_ms: 2000
But what about via environment variable? What would HTTP
env var look like?
I would like to use yaml.Node
in my config files but have found they don't work with cleanenv. As an experiment, I changed the import in cleanenv.go
to use yaml v3 instead and things with yaml.Node
started working. I did no testing otherwise though.
So, I'm curious, any plans to move to yaml v3?
Wehnever env-default tag is defined, it's value overwrites value read from yml file (which is not intended behaviour, according to https://github.com/ilyakaznacheev/cleanenv#read-configuration )
100% reproducibility, observed while running go app in vagrant box (ubuntu/bionic).
configs/config.yml
# server config
server:
host: "localhost"
port: 3000
# database config
db:
type: "mongodb"
mongoConfig:
# full endpoint: address + port
URL: "mongodb://127.0.0.1:27017"
ProjectsDBName: "Projects"
SchemasDBName: "Schemas"
config.go
package common
package common
import (
"github.com/ilyakaznacheev/cleanenv"
"fmt"
)
const YmlConfFile string = "./configs/config.yml"
type AppConfig struct {
ServerConfig struct {
PORT string `yaml:"port" env:"MYAPP_PORT" env-default:"3000" env-upd`
HOST string `yaml:"host" env:"MYAPP_HOST" env-default:"localhost" env-upd`
} `yaml:"server"`
DBConfig struct {
DBType string `yaml:"type" env:"MYAPP_DB_SRV" env-default:"mongodb"`
MongoConfig struct {
URL string `yaml:"URL" env:"MYAPP_DB_URL" env-default:"mongodb://localhost:27017" env-upd`
ProjectsDB string `yaml:"ProjectsDBName" env:"MYAPP_PROJECTS_DB" env-default:"TProjects" env-upd`
SchemasDB string `yaml:"SchemasDBName" env:"MYAPP_SCHEMAS_DB" env-default:"TSchemas" env-upd`
} `yaml:"mongoConfig"`
} `yaml:"db"`
}
func NewAppConfig(confFilePath string) *AppConfig {
var AppC AppConfig
err := cleanenv.ReadConfig(confFilePath, &AppC)
if err != nil { panic(err) }
fmt.Println("AppC.DBConfig.MongoConfig.URL %v",AppC.DBConfig.MongoConfig.URL)
return &AppC
}
main.go
package main
import "myappp/common"
func main() {
appConfig := common.NewAppConfig(common.YmlConfFile)
}
it prints:
AppC.DBConfig.MongoConfig.URL %v mongodb://localhost:27017
(as in env-default)
when I remove respective env-default tag with value, e.g.
it prints
AppC.DBConfig.MongoConfig.URL %v mongodb://127.0.0.1:27017
(as taken from file)
Aside from report above, let me mention that's a great effort to conveniently handle different configuration sources. Thank you!
I've noticed that when reading a config file with a field that is not available in env, its custom function won't run.
The reason seems to be that for these kind of fields, parseValue
won't be called and therefore the corresponding custom function.
Custom functions runs only if the field was provided as an env argument or with a zero'ed value:
I am attempting to use cleanenv with a nested struct and the env-default
tag is not being applied on the child struct.
Is this something cleanenv could support?
Here's a test case replicating the issue:
func TestNestedStructsParseFileEnv(t *testing.T) {
type child struct {
Number int64 `yaml:"number"`
DefaultNumber int64 `yaml:"default_number" env-defult:"-1"`
}
type parent struct {
Children map[string]child `yaml:"children"`
}
children := make(map[string]child)
children["first"] = child{1, -1}
wantConfig := parent{children}
tests := []struct {
name string
file string
ext string
want *parent
wantErr bool
}{
{
name: "yaml",
file: `
children:
first:
number: 1`,
ext: "yaml",
want: &wantConfig,
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tmpFile, err := ioutil.TempFile(os.TempDir(), fmt.Sprintf("*.%s", tt.ext))
if err != nil {
t.Fatal("cannot create temporary file:", err)
}
defer os.Remove(tmpFile.Name())
text := []byte(tt.file)
if _, err = tmpFile.Write(text); err != nil {
t.Fatal("failed to write to temporary file:", err)
}
var prnt parent
if err = ReadConfig(tmpFile.Name(), &prnt); (err != nil) != tt.wantErr {
t.Errorf("wrong error behavior %v, wantErr %v", err, tt.wantErr)
}
if err == nil && !reflect.DeepEqual(&prnt, tt.want) {
t.Errorf("wrong data %v, want %v", &prnt, tt.want)
}
})
}
}
Hello,
Thanks for writing this great library.
reading from multiple sources would be a great future
Below given is the configuation file
type Config struct { Order struct { Date time.Time ``yaml:"date" env:"ORDER_DATE" env-layout:"02/01/2006 03:04:05 PM"`` } ``yaml:"order"`` }
Below is the yml file
order: date: 17/01/2022 10:00:00 AM subtime: 5m
I am getting the following error:
config file parsing error: parsing time "Mon Jan 17 10:00:00 2022" as "2006-01-02T15:04:05Z07:00": cannot parse "Mon Jan 17 10:00:00 2022" as "2006"
I also cloned your repo, edited the test eleanenv_test.go to include the following to the TestParseFile
function
Time time.Time ``yaml:"time" json:"time" toml:"time" env-layout:"02/01/2006 03:04:05 PM"``
On running the test I got the following errors:
go test github.com/ilyakaznacheev/cleanenv --- FAIL: TestParseFile (0.00s) --- FAIL: TestParseFile/yaml (0.00s) cleanenv_test.go:624: wrong error behavior config file parsing error: parsing time "17/01/2022 10:00:00 AM" as "2006-01-02T15:04:05Z07:00": cannot parse "1/2022 10:00:00 AM" as "2006", wantErr false --- FAIL: TestParseFile/json (0.00s) cleanenv_test.go:624: wrong error behavior config file parsing error: parsing time "\"17/01/2022 10:00:00 AM\"" as "\"2006-01-02T15:04:05Z07:00\"": cannot parse "1/2022 10:00:00 AM\"" as "2006", wantErr false --- FAIL: TestParseFile/toml (0.00s) cleanenv_test.go:624: wrong error behavior config file parsing error: parsing time "17/01/2022 10:00:00 AM" as "2006-01-02T15:04:05Z07:00": cannot parse "1/2022 10:00:00 AM" as "2006", wantErr false FAIL FAIL github.com/ilyakaznacheev/cleanenv 0.004s FAIL
Add parsing from extensible data notation files
For now, I see that Actions works a bit faster than Travis CI. and it would be easier to set up all in one place.
It's too much for such a simple library. While there are no CI/CDs or environments bound to each branch, the develop
branch just gives unnecessary complexity.
yaml file
isdebug: false
rule:
cachesize: 10485760000
golang struct
type AppConfig struct {
IsDebug bool `env:"RE_IS_DEBUG" env-required:"true"`
Rule struct {
CacheSize int `env:RULE_CACHE_SIZE" env-required:"true"`
}
i got an error field \"IsDebug\" is required but the value is not provided"
, but if set isdebug: true
- all ok, config parse without any errors.
Hi,
If file doesn't exists or corrupted - nothing got at all. TOML/ENV can be empty (null) to be ok all others - fail.
Expected to get params from env and/or default env
package main
import (
"github.com/ilyakaznacheev/cleanenv"
"github.com/spf13/pflag"
"github.com/davecgh/go-spew/spew"
)
type Config struct {
BindAddr string `toml:"bindaddr" env:"YAACC_BINDADDR" env-default:":5013"`
DBPort int `toml:"dbport" env:"YAACC_DBPORT" env-default:"5432"`
DBHost string `toml:"dbhost" env:"YAACC_DBHOST" env-default:"localhost"`
DBName string `toml:"dbname" env:"YAACC_DBNAME" env-default:"postgres"`
DBUser string `toml:"dbuser" env:"YAACC_DBUSER" env-default:"cdr"`
DBPassword string `toml:"dbpassword" env:"YAACC_DBPASSWORD"`
Logfile string `toml:"logfile"`
}
func main() {
cfgfile := pflag.StringP("config", "c", "./yaacc.toml", "File to read config params from")
logfile := pflag.StringP("logfile", "l", "", "File to write exec time messages.")
pflag.Lookup("logfile").NoOptDefVal = "./yaacc.log"
pflag.Parse()
jww.SetPrefix("yaacc")
jww.SetLogThreshold(jww.LevelTrace)
jww.SetStdoutThreshold(jww.LevelInfo)
err := cleanenv.ReadConfig(*cfgfile, &cfg)
if err != nil {
jww.WARN.Printf("Error loading config (%s): %q", *cfgfile, err)
}
spew.Dump(cfg)
}
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
goroutine 1 [running]:
reflect.valueInterface({0x95f8a0?, 0xc00055c248?, 0xabea8f?}, 0xa?)
E:/software/dev_core/go1.18.1/src/reflect/value.go:1441 +0xd8
reflect.Value.Interface(...)
E:/software/dev_core/go1.18.1/src/reflect/value.go:1430
github.com/ilyakaznacheev/cleanenv.readStructMetadata({0x9c7900?, 0xc00055c1c0?})
E:/text/code-project/GolangProject/pkg/mod/github.com/ilyakaznacheev/[email protected]/cleanenv.go:274 +0x771
github.com/ilyakaznacheev/cleanenv.readEnvVars({0x9c7900, 0xc00055c1c0}, 0x0)
E:/text/code-project/GolangProject/pkg/mod/github.com/ilyakaznacheev/[email protected]/cleanenv.go:333 +0x45
github.com/ilyakaznacheev/cleanenv.ReadConfig({0xace493?, 0xc00041f9a8?}, {0x9c7900, 0xc00055c1c0})
E:/text/code-project/GolangProject/pkg/mod/github.com/ilyakaznacheev/[email protected]/cleanenv.go:95 +0x48
It doesn't skip unexported field
Originally posted by @838239178 in #68 (comment)
package main
import (
"github.com/ilyakaznacheev/cleanenv"
)
type Children struct {
name string `yaml:"name" env:"name"`
}
type Config struct {
private Children `yaml:"private" env-prefix:"private"`
Public string `yaml:"public" env:"public"`
}
func main() {
var conf Config
if err := cleanenv.ReadConfig("config.yml", &conf); err != nil {
panic(err)
}
}
Documentation should give a better understanding of how to use the library and what features it has.
Hi,
i'm currently migrating from a combination of godotenv, uber-config and envconfig to your library because AFAIK your library is doing a good job integrating this loading mechanism into a single library without hastle.
But as i have already a decent config struct based on envconfig i see thats its pretty hard to add for every property an env tag (AFAIK this is needed).
So i would like to request some kind of default naming schema like envconfig has already. Further i'm missing the usage of prefixes but i've seen that there is already an issue addressing this.
For so the default naming here is short example:
type Config struct {
Server ServerConfig
}
type ServerConfig struct {
Address string
Port int `env-default:"1328"`
FriendlyName string
}
Currently i would to add everywhere an env
tag with the name.. which is mostly the same as the property.
So it would be cool if this config would be sufficient to have the env config working like this:
SERVER_ADDRESS=cleanenv.is.great
SERVER_PORT=1337
SERVER_FRIENDLYNAME="Yeah cleanenv is great"
What do you think? Could this be an future feature?
Thanks
Need to create a project logo and add it to the project header
Adding "env-unset" tag would allow unsettling sensitive data after successfully read it out from the environment.
Need to check if it is possible to parse time.Time
and add the possibility to set parsing layouts.
Hello, I have used the lib for building config from env and a file. If I created field as pointer then I thought that it is as optional and it is not bound (nil value). When I set value for pointer field in a config file my logic is gone well, but if I set via env variable then my program throw exception ("unsupported type .")
type Config struct {
optField *MyField `env:"OPT_VALUE"`
}
I think that 'cleanenv' have to implement same logic with env that a config file and this way not provide an unexpected error.
I'm having hard time overrriding a value from enviroment variable. Here is what I want. I want to read config from a enviroment file. Then read enviroment varialbes and overwrite them even If they exist in the config file. The following code does not overwrite value from enviroment varialbe as it supposed to:
type ConfigDatabase struct {
Port string `env:"PORT" env-default:"5432"`
Host string `env:"HOST" env-default:"localhost"`
Name string `env:"NAME" env-default:"postgres"`
User string `env:"USER" env-default:"user"`
Password string `env:"PASSWORD"`
}
var cfg ConfigDatabase
err := cleanenv.ReadConfig("config.env", &cfg)
if err != nil {
log.Println(err)
}
err1 := cleanenv.ReadEnv(&cfg)
if err1 != nil {
log.Println(err1)
}
fmt.Println(cfg)
Even witih cleanenv.UpdateEnv
It still does not work. To my understanding, Each Read* function should overwrite the previous values.
Also the env-update struct tag is not working either.
Password string `env:"PASSWORD" env-upd`
I get an warning in VSCode
struct field tag
env:"PASSWORD" env-upd not compatible with reflect.StructTag.Get: bad syntax for struct tag pair
How can we use map of string to array of strings in ENV?
Suppose we have :
MapStringArrayString map[string][]string
env:"TEST_MAPSTRINGARRAYSTRING"``
Is it even possible with current implementation?
According to #36
yml
tags seem not to work for some users, but yaml
work at the same moment. Why?
For the following example, if I provide a non-valid PORT=asd
. Then in the error message, I would see "strconv.ParseInt: parsing "asd": invalid syntax". I think it would be great to have a custom error that would collect and provide some details on the type of error that happened to be able to write it to the log and give corresponding feedback to the runner of the application.
type Config struct {
Web struct {
Port int `env:"PORT" env-default:"8000"`
}
}
if err := cleanenv.ReadEnv(&config); err != nil {
fmt.Printf("Error: %v", err)
}
The message can have the following format: PORT must be a valid number
or DB_HOST is a required variable
.
I'm not a Golang developer but have written some code in Go, so if you give your opinion on this and some hints I might be able to implement it and send a PR.
Found that even if my struct has the env-required
or env-default
set for the fields, if the environment variable was set but left empty, no error is rised, default value is not set either if used. Eg:
Lets say this var was set like this:
export MY_VAR=""
If my struct were:
type MyVars struct {
MyVar string `env:"MY_VAR" env-required:"true"`
}
No error is raised at all, and i get and empty struct. If otherwise my struct had a default:
type MyVars struct {
MyVar string `env:"MY_VAR" env-default:"something"`
}
Id get an empty struct again rather than one with the default value.
Is this an expected behavior or could it be improved on?
Some code snippets have wrong formatting
Trying to read a value that contains .
and it doesn't matter what I set the env-separator
to. It gets clipped anyway.
Am I using it wrong or this a bug?
Add groups with prefixes and group descriptions
The group should be defined as a structure with a set of tags, which defines the structure as a group, specifies an environment variable prefix (optional) and a group description, that will be shown in help (optional).
Hi,
it would be cool if the library would support expansion of environment variables using os.ExpandEnv
A use-case for this feature would be if you need to construct a configuration variable based on multiple different ones.
e.g. building an URL wrong multiple environment variables
I've tried to figure out where this must be added but couldn't find the write code part.
Thanks
PS: i'm thinking about something like this
https://mtyurt.net/post/go-using-environment-variables-in-configuration-files.html
Need to add a helper to include a flag
package help into the main help to build a one-line help setup.
Example
Need to add more examples for different cases to example
directory.
Github repo secrets are only shard among contributors, and based on recent changes by GH, workflow does not run automatically for unknown contributors as well.
In this case its better to disable the workflow for notification on pull request, either by deleting pull_request.yml or commenting out the content of file.
Connects to #73
Hello, I used v1.2.5 and alway gone all right. But problems with time fields have appeared in 1.2.6.
I have this error pattern:
line N: cannot unmarshal !!int `M` into time.Duration
.
I have config same as:
type Config {
...
Timeout time.Duration `yaml:"timeout"`
...
}
Example:
psql:
host: ${DB_HOST}
port: ${DB_PORT}
user: ${DB_USER}
password: ${DB_PASSWORD}
database: default
where DB_HOST,DB_PORT,... are the environment variables
Need to run CI for PRs into the master branch
Hi,
i really like your simple approach for env configuration and also the possibility to easily print the env variables.
But i'm not that fully happy with the multiline format for the env variables and currently its pretty hard to adjust it.
So maybe it would be useful to have a possibility to pass an own format for the description lines like
"$1 [$2] - $3 (default $4)"
which would be create an output like this
TEST_ENV [string] - This is a test env variable (default "test")
Another maybe more easier approach to implement would be to add an method which returns the parsed metadata so it could be used to print using an own format.
Thanks
It would be nice if we have the possibility to override a base config.
Like:
.env.override
.config.yml.override
We are trying to beat 100% test coverage :)
Hi,
I'm experiencing a strange behavior and I'd like to understand if it's the expected behavior or a bug (in that case I can create a PR and fix it).
I do have a bool field in my config and the env-default value is true. If I define the field in my config file to false I would expect the value to be set to false, but I get true.
I looked in the code and the isZero function is considering false as zero (which makes sense in general but probably not in this case).
Could you confirm what would be the expected behavior?
Thanks,
Marco
I am new to AWS and Multi Docker Container.
Not able load config.yml and I don't know how to configure Dockerrun.aws.json to make sure the config.yml file is accessible in the docker container.
var err = cleanenv.ReadConfig("config.yml", &cfg)
if err != nil {
log.Fatal(err)
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.