Managing Elixir configs
by Tomasz Kowal
Have you ever tried to deploy your Elixir/Phoenix application and the deployment failed due to a missing environment variable?
Or worse! The deployment succeeded, but the app crashed on the staging environment?
Or even worse! The environment variables were different in the staging environment and production, so you had to roll back?
I am going to talk about structuring your configs in such a way that minimizes differences between environments.
How do Elixir configs work?
I will not describe everything in detail, but I’ll highlight points essential to understanding the problem and proposed solution.
MIX_ENV and default environments
MIX_ENV
describes the environment in which you apply configs. It is very flexible. You can define your environments according to your needs. However, the default generator provides you with three: dev
, test
and prod
. The names look self-explanatory, but they can be misleading.
E.g. is your staging environment development or production? Should you create a new one like stag
?
The consensus is that dev
is your local development, test
is for running mix test
locally and in CI, and prod
is everything that is deployed. That is the first important point; you will rarely need more than those three environments.
Compile-time vs runtime
Elixir evaluates compile-time configs during compile-time. That means:
- You can’t use your dependencies here because they are not compiled yet
-
You can
import_config
here based onMIX_ENV
. -
You can
import_config
based onMIX_ENV
, so the default is to haveconfig.exs
,dev.exs
,test.exs
, andprod.exs
.
Elixir evaluates runtime configs before booting the app (either with mix
or as a release). That means:
- You can use your dependencies
-
There is no
MIX_ENV
because we might be in a release -
There is
config_env()
with the same set of values, but you can’timport_config
. -
Any config depending on the mix environment needs to live under
if config_env() == ...
.
Order of evaluation
-
First,
config.env
during the compilation -
Then
[MIX_ENV].exs
still during compilation -
Then
runtime.exs
when the application starts
Elixir “deep-merges” configurations. If a key occurs later during evaluation, it overrides the previous value. Unless both values are keyword lists, then Elixir merges them.
Assumptions
The need to control compilation with environment variables is infrequent. The only “compile-time environment variable” should be MIX_ENV
. It would be best to read all other variables only in runtime.exs
.
I’ll assume you have a deploy mechanism that prevents the deployment in case of an error during startup. If a variable is required to run, it is best to fail early. Use System.fetch_env!/1
instead System.get_env/1
. Use System.get_env/2
only with meaningful default values that will work in production.
The problem with the default config setup
Our problem with configs generated by default is that they have big if config_env() == :prod
in the runtime.exs
. It is easy to configure your local and test environment, push changes, see the CI green and try to deploy without noticing it won’t work when deployed.
I want to use the same environment variables in development and production to automate checking if they are all set on remote servers.
Part1 of the solution - direnv
Direnv is an incredible tool that handles setting environment variables for you. You define the .envrc
file with your exports, it asks you to confirm you like them (it is a security feature), and voilà, you are good to go.
With this, you can remove the nasty if config_env() == :prod
. However, our tests would like to overwrite a handful of those values, e.g. database name or port.
Part2 of the solution - ConfigHelper
def fetch_env!(var) do
value =
with :test <- config_env(),
{:ok, test_val} <- System.fetch_env("TEST_" <> key) do
test_val
else
_ -> System.fetch_env!(key)
end
end
This helper will behave like System.fetch_env!
except in :test
environment, it will first try to find TEST_ENV
, and if it is not there, it will fall back to ENV
.
We discovered that there is usually only a handful of “test overrides”, so it is OK to keep them in the .envrc
alongside regular variables.
Benefits
With all necessary variables in .envrc
and no conditional logic in runtime.exs
, it is easy to compare local and deployed environments (dev
and prod
). We find it more maintainable.
Drawbacks
If someone unfamiliar with the convention enters the project without a proper introduction, they might be confused about all those TEST_
variables that are seemingly never read.
Should Phoenix adopt that technique?
I don’t think so. Needs vary, and the default configuration is not surprising. However, with a web app that only sticks to the basics, you can consider unifying environment variables across local and deployed environments with that trick.
Credits
I’d like to thank my colleague Henrik Sjööh who proposed that solution during our discussions at Bluecode
Did you like it? Follow me on Mastodon or Twitter (for as long as it lasts)