Skip to content

Allow env.local to use delayed eval vars immediately, if needed #644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tlvu wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Allow env.local to use delayed eval vars immediately, if needed #644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b995b6e
allow env.local to use delayed eval vars immediately, if needed
tlvu Feb 6, 2026
18b6764
CHANGES: update for eval_delayed_var()
tlvu Feb 6, 2026
4f93f0c
CHANGES: simplify eval_delayed_var example (mishaschwartz feedback)
tlvu Feb 11, 2026
deb6c6e
README: mention eval_delayed_var (fmigneault feedback)
tlvu Feb 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,41 @@
[Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest)
------------------------------------------------------------------------------------------------------------------

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)
## Changes
- Allow `env.local` to use delayed eval var immediately

Before, `env.local` can not immediately use any of the delayed eval var
because their real values are only available after `env.local` is fully read.

So something like this in `env.local` do not work because `JUPYTERHUB_USER_DATA_DIR`
is a delayed eval var:
```sh
export JUPYTERHUB_README_FR="$JUPYTERHUB_USER_DATA_DIR/jupyter-readme/LISMOI.ipynb"

# Error, file not found even if the file is actually on disk
# because JUPYTERHUB_README_FR resolve to "${BIRDHOUSE_DATA_PERSIST_ROOT}/jupyterhub_user_data/jupyter-readme/LISMOI.ipynb"
# because JUPYTERHUB_USER_DATA_DIR is a delayed eval var.
if [ -f "$JUPYTERHUB_README_FR" ]; then
(...)
```

`env.local` could simply append `JUPYTERHUB_README_FR` to `DELAYED_EVAL` list
but since we need to use its value immediately in `env.local`, we need to
eval it immediately.

With the new `eval_delayed_var`, we can do the following, because
`JUPYTERHUB_USER_DATA_DIR` is a delayed eval var, any vars that depend on it
should also be delayed eval'ed.

```sh
export JUPYTERHUB_README_FR="$JUPYTERHUB_USER_DATA_DIR/jupyter-readme/LISMOI.ipynb"
Copy link
Collaborator

@mishaschwartz mishaschwartz Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this in single quotes otherwise this won't wait until the eval_delayed_var is called in order to evaluate JUPYTERHUB_USER_DATA_DIR

Copy link
Collaborator Author

@tlvu tlvu Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is real code that we actually use in production and it works as-is.

In env.local it is not necessary to use single-quote because env.local is always last to be read. In all other default.env, you are right, we need single quote because env.local can still override those variables.

Copy link
Member

@fmigneault fmigneault Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you fully override JUPYTERHUB_USER_DATA_DIR=/my/custom/path? In that case, indeed, quotes do not really matter because the result is set as the final value you want anyway.

However, if you used something like

export JUPYTERHUB_USER_DATA_DIR='${BIRDHOUSE_DATA_PERSIST_ROOT}/alternate-jupyter'

You could let a test/staging server still set BIRDHOUSE_DATA_PERSIST_ROOT accordingly to different locations, after the fact, with another env file loaded as:

export BIRDHOUSE_EXTRA_CONF_DIRS="
   # ... usual/shared stuff ...
   /my/deploy-types/staging   # load the `default.env` overrides of this specific env
"

And you wouldn't have to maintain JUPYTERHUB_USER_DATA_DIR in each sub-env variant.
It is somewhat of a niche use case since you can override anything you want with explicit definitions in the final env.local if you prefer, but still a nice feature.

eval_delayed_var JUPYTERHUB_README_FR

# Now this works because JUPYTERHUB_README_FR resolve properly to "/data/jupyterhub_user_data/jupyter-readme/LISMOI.ipynb"
if [ -f "$JUPYTERHUB_README_FR" ]; then
(...)
```


[2.21.2](https://github.com/bird-house/birdhouse-deploy/tree/2.21.2) (2026-02-05)
------------------------------------------------------------------------------------------------------------------
Expand Down
8 changes: 8 additions & 0 deletions birdhouse/README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,14 @@ See `env.local.example <env.local.example>`_ (:download:`download </birdhouse/en
Most variables that can be set in the local environment file (``env.local`` by default) can also be specified as environment variables when running ``bin/birdhouse``
commands. Environment variables will take precedence over those specified in the ``env.local`` file.

In `env.local`, if you wish to immediately use any of the delayed eval variable value, you can use `eval_delayed_var DELAYED_VAR_NAME`
to compute their value. If not, their real values are only available after `env.local` is read. If that delayed eval variable depends
on another variable that you override in `env.local`, then you should define the override before calling `eval_delayed_var`. Ex:
`JUPYTERHUB_USER_DATA_DIR` is a delayed eval var because it depends on the value of `BIRDHOUSE_DATA_PERSIST_ROOT` which can be
overridden in `env.local`. So if you need the value of `JUPYTERHUB_USER_DATA_DIR` immediately in `env.local`, you need to
`eval_delayed_var JUPYTERHUB_USER_DATA_DIR` and if you need to override `BIRDHOUSE_DATA_PERSIST_ROOT`, then you need to override it
before calling `eval_delayed_var JUPYTERHUB_USER_DATA_DIR`.
Comment on lines +115 to +118
Copy link
Member

@fmigneault fmigneault Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the example be as follows?
Seems quicker to interpret the feature and makes sure the order is explicit.

# in your env.local
export BIRDHOUSE_DATA_PERSIST_ROOT=/my/data  # define dependency needed by delayed var
echo "${JUPYTERHUB_USER_DATA_DIR}"           # delayed '${BIRDHOUSE_DATA_PERSIST_ROOT}/jupyterhub_user_data'
eval_delayed_var JUPYTERHUB_USER_DATA_DIR    # evaluate the desired variable
echo "${JUPYTERHUB_USER_DATA_DIR}"           # evaluated '/my/data/jupyterhub_user_data'


If the file `env.local` is somewhere else, symlink it here, next to `docker-compose.yml <docker-compose.yml>`_ (:download:`download </birdhouse/docker-compose.yml>`) because many scripts assume this location.
If autodeploy scheduler job is enabled, the folder containing the `env.local` file needs to be added to `BIRDHOUSE_AUTODEPLOY_EXTRA_REPOS`.

Expand Down
14 changes: 11 additions & 3 deletions birdhouse/read-configs.include.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,16 @@ check_required_vars() {
}


# env.local can call this function if it wishes to use a delayed eval var.
# Otherwise all delayed eval vars are only available after env.local is fully read.
eval_delayed_var() {
DELAYED_VAR_NAME="$1"
v="`eval "echo \\"\\$${DELAYED_VAR_NAME}\\""`" # should keep new lines and leading empty spaces
value=`eval "echo \"${v}\""`
eval 'export ${DELAYED_VAR_NAME}="${value}"'
}


# All scripts sourcing default.env and env.local and needing to use any vars
# in DELAYED_EVAL list need to call this function to actually resolve the
# value of each var in DELAYED_EVAL list.
Expand All @@ -382,9 +392,7 @@ process_delayed_eval() {
# only eval each variable once (in case it was added to the list multiple times)
continue
fi
v="`eval "echo \\"\\$${i}\\""`" # should keep new lines and leading empty spaces
value=`eval "echo \"${v}\""`
eval 'export ${i}="${value}"'
eval_delayed_var ${i}
log DEBUG "delayed eval '$(env | grep -e "^${i}=")'"
ALREADY_EVALED="
${ALREADY_EVALED}
Expand Down