Standard library · System and tools

env

Environment variables: reading them with the right type, .env files, the run profile, and masking sensitive values.

import env

Settings such as a server port, a database address and API keys belong in environment variables rather than in the code, the "twelve-factor" way of configuring a service. The env package reads them with the right type and a default, masks secrets for logs, loads .env files, picks files per environment and fills ${NAME} templates. For the bare value of one variable, os.Env is enough; to parse or write .env files as data without touching the environment, use encoding.envfile; for command line options, flag.

The "profile" says whether the program is running in development, test or production. It is read from SALAM_ENV; when that is unset or empty, APP_ENV and then ENV are tried, and the default is development. Profile returns the value lower-cased but otherwise as written. IsProduction, IsDevelopment and IsTest also accept prod, dev and testing, but LoadProfile uses the name as written, so SALAM_ENV=prod loads .env.prod, not .env.production.

What people get wrong:

  • Trusting the typed getters to validate. GetInt gives the default for abc and 12px just as for a missing variable, and GetBool knows only lower case true, 1, yes, on and false, 0, no, off, so DEBUG=True gives the default.
  • Expecting a .env file to win. Load and LoadProfile never replace a variable that is already set, which is right in production; Overload and LoadDotEnv do replace.
  • Logging secrets. Pass them through Mask, and keep .env.local files out of version control.

Reading the whole configuration at start-up

Read every setting once, in one function, into a struct. The rest of the program then takes the struct and never calls env itself, which keeps the list of variables in one place and makes the code easy to test. Print the configuration at start-up, but pass secrets through Mask first.

env-guide-config.salam
import env
import str

struct Config:
    pub host: str = ""
    pub port: i64 = 0
    pub workers: i64 = 0
    pub debug: bool = false
    pub db_url: str = ""
    pub api_key: str = ""
end

func load_config(): Config:
    ret Config {
        host = env.GetOr("APP_HOST", "127.0.0.1"),
        port = env.GetInt("APP_PORT", 8080),
        workers = env.GetInt("APP_WORKERS", 4),
        debug = env.GetBool("APP_DEBUG", false),
        db_url = env.GetOr("APP_DB_URL", "sqlite://app.db"),
        api_key = env.Get("APP_API_KEY")
    }
end

func main:
    env.Set("APP_PORT", "9000")
    env.Set("APP_DEBUG", "yes")
    env.Set("APP_API_KEY", "sk-test-1234567890")
    env.Unset("APP_HOST")
    env.Unset("APP_WORKERS")
    env.Unset("APP_DB_URL")
    c := load_config()
    println "listening on", c.host + ":" + str.FromInt(c.port as int)
    println "workers:", c.workers, "debug:", c.debug
    println "database:", c.db_url
    println "api key:", env.Mask(c.api_key)
    if c.api_key.Len() == 0: println "warning: APP_API_KEY is not set" end
end
Outputlistening on 127.0.0.1:9000 workers: 4 debug: true database: sqlite://app.db api key: sk****90

What a .env file can contain

A .env file has one NAME=value per line. Lines starting with # are comments, a leading export is allowed so the same file works with a shell, and values may be in double or single quotes. Inside double quotes and unquoted values, ${NAME} is replaced; single quotes keep the text exactly as written.

env-guide-dotenv.salam
import env
import io

func main:
    io.WriteFile(".env", "# local settings\nexport APP_NAME=\"Salam Shop\"\nAPP_PORT=8080\nAPP_URL=http://localhost:${APP_PORT}\nAPP_GREETING='hello ${APP_NAME}'\nEMPTY=\n")
    env.Set("APP_PORT", "3000")
    println "read:", env.Load(".env")
    println env.Get("APP_NAME")
    println env.Get("APP_PORT")
    println env.Get("APP_URL")
    println env.Get("APP_GREETING")
    println env.Has("EMPTY"), "[" + env.Get("EMPTY") + "]", env.GetOr("EMPTY", "fallback")
end
Outputread: 4 Salam Shop 3000 http://localhost:8080 hello ${APP_NAME} true [] fallback

Three things to notice. Load never replaces a variable that is already set, so APP_PORT stays 3000, and the number it returns counts only the variables it actually set. A ${APP_PORT} inside the file uses the file's own value. And an empty value is set but empty: Has says true while GetOr gives the default.

One set of files per environment

LoadProfile reads up to four files from a folder, later ones winning over earlier ones: .env, .env.local, .env.<profile> and .env.<profile>.local. Commit the first and third, and keep the .local files, which hold each developer's own values and secrets, out of version control. A variable that is already set in the real environment still wins over every file.

env-guide-profiles.salam
import env
import io

func main:
    io.WriteFile(".env", "DB_HOST=localhost\nLOG_LEVEL=debug\nCACHE=off\n")
    io.WriteFile(".env.local", "DB_HOST=127.0.0.1\n")
    io.WriteFile(".env.production", "DB_HOST=db.internal\nLOG_LEVEL=warn\n")
    io.WriteFile(".env.production.local", "LOG_LEVEL=error\n")
    env.Set("SALAM_ENV", "production")
    env.Set("CACHE", "on")
    println "profile:", env.Profile(), env.IsProduction()
    println "loaded:", env.LoadProfile(".")
    println "DB_HOST =", env.Get("DB_HOST")
    println "LOG_LEVEL =", env.Get("LOG_LEVEL")
    println "CACHE =", env.Get("CACHE")
end
Outputprofile: production true loaded: 2 DB_HOST = db.internal LOG_LEVEL = error CACHE = on

Here DB_HOST comes from .env.production, LOG_LEVEL from .env.production.local, and CACHE stays on because it was set before loading. Missing files are skipped without an error.

Filling a template with variables

Expand replaces ${NAME} in any text, and ${NAME:-default} uses the default when the variable is unset or empty. It suits connection strings and paths that are partly configured. An unknown name without a default becomes an empty string, so check the important variables with Has first.

env-guide-expand.salam
import env

func main:
    env.Set("DB_USER", "shop")
    env.Set("DB_NAME", "orders")
    env.Unset("DB_HOST")
    env.Unset("DB_PORT")
    template := "postgres://${DB_USER}@${DB_HOST:-localhost}:${DB_PORT:-5432}/${DB_NAME}"
    println env.Expand(template)
    env.Set("DB_HOST", "db.example.com")
    println env.Expand(template)
    println env.Expand("missing: [${NOT_SET_ANYWHERE}]")
end
Outputpostgres://shop@localhost:5432/orders postgres://shop@db.example.com:5432/orders missing: []

Pitfalls

  • GetInt and GetFloat return the default when the variable is unset or empty and also when it is not a number: abc and 12px both give the default. When a wrong setting must stop the program, read the text with Get and parse it with conv.ParseInt inside try.
  • GetBool treats anything it does not recognise as false, whatever the default. It compares case-sensitively and accepts only true, 1, yes and on; TRUE and Yes are false.
  • Load keeps existing values, Overload replaces them, and LoadDotEnv replaces them too. Use Load when the real environment should win, which is what you want in production.
  • Set changes the environment of this program and of the processes it starts afterwards, never the shell that started it.

Functions

Get

func Get(name: str): str
The value of a variable, or an empty string when it is not set; an empty and an unset variable look the same here.
env-Get.salam
import env

func main:
    env.Set("PORT", "8080")
    env.Set("DEBUG", "true")
    env.Set("RATIO", "0.75")
    println env.Get("PORT"), env.Has("PORT"), env.Has("NOPE")
    println env.GetInt("PORT", 80), env.GetInt("NOPE", 80)
    println env.GetBool("DEBUG", false), env.GetFloat("RATIO", 1.0)
    println env.GetOr("HOST", "localhost")
    env.Unset("PORT")
    println env.Has("PORT")
    env.Set("API_KEY", "example-not-a-real-key")
    println env.Mask("example-not-a-real-key"), env.GetMasked("API_KEY")
    env.Set("NAME", "Salam")
    println env.Expand("${NAME} on port ${PORT:-3000}")
end
Output8080 true false 8080 80 true 0.75 localhost false ex****ey ex****ey Salam on port 3000

GetOr

func GetOr(name: str, def: str): str
The value of a variable, or the default when it is not set or is empty.

The example for this item is shown with the ones above: env-Get.salam

Has

func Has(name: str): bool
True when the variable is set, even to an empty value.

The example for this item is shown with the ones above: env-Get.salam

GetInt

func GetInt(name: str, def: i64): i64
The value as a whole number, or the default when it is not set, is empty or is not a whole number: abc and 12px give the default.

The example for this item is shown with the ones above: env-Get.salam

GetFloat

func GetFloat(name: str, def: f64): f64
The value as a decimal number, or the default when it is not set, is empty or is not a number.

The example for this item is shown with the ones above: env-Get.salam

GetBool

func GetBool(name: str, def: bool): bool
The value as a boolean: exactly true, 1, yes or on (lower case, spaces around allowed) count as true and false, 0, no or off as false. Anything else, and an unset or empty variable, gives the default.

The example for this item is shown with the ones above: env-Get.salam

Set

func Set(name: str, value: str): bool
Sets a variable for this program and for the processes it starts afterwards; true on success.

The example for this item is shown with the ones above: env-Get.salam

Unset

func Unset(name: str): bool
Removes a variable.

The example for this item is shown with the ones above: env-Get.salam

Mask

func Mask(value: str): str
Masks a sensitive value for a log: a value longer than eight characters keeps its first and last two characters around ****, a shorter one becomes all stars, and an empty one stays empty.

The example for this item is shown with the ones above: env-Get.salam

GetMasked

func GetMasked(name: str): str
Returns a variable's value masked.

The example for this item is shown with the ones above: env-Get.salam

Profile

func Profile(): str
The name of the run profile in lower case, from SALAM_ENV, else APP_ENV, else ENV, else development.
env-Profile.salam
import env

func main:
    env.Set("SALAM_ENV", "production")
    println env.Profile(), env.IsProduction(), env.IsDevelopment(), env.IsTest(), env.IsStaging()
    env.Set("SALAM_ENV", "test")
    println env.Profile(), env.IsTest()
end
Outputproduction true false false false test true

IsProduction

func IsProduction(): bool
True in the production or prod profile.

The example for this item is shown with the ones above: env-Profile.salam

IsDevelopment

func IsDevelopment(): bool
True in the development or dev profile, which is the default.

The example for this item is shown with the ones above: env-Profile.salam

IsTest

func IsTest(): bool
True in the test or testing profile.

The example for this item is shown with the ones above: env-Profile.salam

IsStaging

func IsStaging(): bool
True in the staging profile.

The example for this item is shown with the ones above: env-Profile.salam

Load

func Load(path: str): int
Loads a KEY=VALUE file without changing variables that are already set, expanding ${NAME} references. It returns how many variables it set; a missing file sets none.
env-Load.salam
import env
import io

func main:
    io.WriteFile(".env", "DB_HOST=localhost\nDB_PORT=5432\n# a comment\n")
    println env.Load(".env"), env.Get("DB_HOST")
    env.Set("DB_PORT", "1")
    env.Load(".env")
    println env.Get("DB_PORT")
    env.Overload(".env")
    println env.Get("DB_PORT")
    println env.LoadText("A=1\nB=2"), env.Get("B")
    println env.LoadDotEnv(".env")
    io.WriteFile(".env.test", "MODE=testing\n")
    env.Set("SALAM_ENV", "test")
    println env.LoadProfile("."), env.Get("MODE")
end
Output2 localhost 1 5432 2 2 2 1 testing

Overload

func Overload(path: str): int
Like Load, but it replaces existing variables with the file's values too.

The example for this item is shown with the ones above: env-Load.salam

LoadText

func LoadText(text: str): int
Does the same as Load from text instead of a file.

The example for this item is shown with the ones above: env-Load.salam

LoadDotEnv

func LoadDotEnv(path: str): int
Loads the given file and, like Overload, replaces variables that are already set.

The example for this item is shown with the ones above: env-Load.salam

LoadProfile

func LoadProfile(dir: str): int
Loads .env, .env.local, .env.<profile> and .env.<profile>.local from the given directory, later files winning, without replacing variables already set. Missing files are skipped.

The example for this item is shown with the ones above: env-Load.salam

Expand

func Expand(text: str): str
Replaces ${name} and ${name:-default} in text with the values of the variables; an unknown name without a default becomes empty.

The example for this item is shown with the ones above: env-Get.salam