opam `env_hook` is too agressive, and makes the default install configuration of opam behave invasively in the shell environment

[shonfeder]

Context

The env_hook works like this:

opam/src/state/shellscripts/env_hook.sh

Lines 1 to 8 in b5254e3

	_opam_env_hook() {
	local previous_exit_status=$?;
	eval $(opam env --shell=bash --readonly 2> /dev/null <&- );
	return $previous_exit_status;
	};
	if ! [[ "$PROMPT_COMMAND" =~ _opam_env_hook ]]; then
	PROMPT_COMMAND="_opam_env_hook;$PROMPT_COMMAND";
	fi

Prepending the _opam_env_hook to the PROMPT_COMMAND, causes it to be invoked before every new command prompt. opam env always prefixes its $HOME/.opam/default/bin directory to the user's PATH. As a result, once users have the env_hook tied into their shell's rc file, it becomes impossible for users to prefix this PATH with any other directories. This happens "behind the backs" of unsuspecting users, since the only way to figure out what is happening is to dig thru a few layers of scripts.

Problem

I say that the current behavior is "invasive" because (IMO) it infringes on the user's expectation that they can control their path, by aggressively dominating the prefix.

Here's an example of pathological behavior:

In my current switch, I have a binary $OPAM_SWITCH_PREFIX/bin/omd. I want to define my own binary, so I do

$ mkdir mybin
$ cat > mybin/omd <<EOF
> #!/usr/bin/sh
> echo "Orchestral Manoeuvres in the Dark"
> EOF
$ chmod +x mybin/omd 
$ ./mybin/omd 
Orchestral Manoeuvres in the Dark

now, within my current shell session, I want to be able to have this binary in my path, so I run

$ export PATH=$HOME/mybin:$PATH

I should now expect I can invoke my new omd binary, and it will take precedence. But instead

$ omd --help | head
omd [options] [inputfile1 .. inputfileN] [options]
  -o                  file.html Specify the output file (default is stdout).
  --auto-identifiers  Should identifiers be automatically assigned to headings.
  --version           Display the version of the currently installed omd.
  --                  Consider all remaining arguments as input file names.
  -help               Display this list of options
  --help              Display this list of options

Because the env_hook has insisted on putting the opam binary dir first in my path:

$ echo $PATH
/home/me/.opam/default/bin:/home/me/mybin:<...>

This is quite unfortunate behavior, IMO, and it has led me to scrub all remnants of the current opam hooks from my rc files.

Proposal

In general, I think we should try to find a way where we can maintain the convenience of automatic switch management without clobbering a users path.

AFAIU, this is the essential property we want the automatic switch management to guarantees, w/r/t binaries:

1. When a user first activates a switch (explicitly via or implicitly by changing into a directory that has a local switch), the binaries installed in the switch should shadow any synonymous binaries that may be in the users environment.

The current implementation ensures this property, but it does so thru this much stronger property, which I think we don't want:

2. Whenever a user runs a command, the binaries installed in the active switch shadow any synonymous binaries that may be in the users environment.

The most similar tool I am familiar with is direnv. It does not have this problem, so I have taken to using direnv to manage local switches by adding the following files to projects that use them:

$ cat .envrc 
eval $(opam env)

IIUC, direnv manages to avoid the problem of (2) by computing the diff between the current environment and environment entailed by it's .envrc file, and only updating the environment if the settings entailed by the .envrc file are not already in the environment.

In the context of opam switches, I think this would mean opam env would only prefix a a switche binary path if the current binary path is not already in the path. If it is already in the path, it should return the env unchanged.

I think this should ensure (1), because the switches directory will be prefixed a new switch is activated, but it will not be prefixed on command prompts that don't don't introduce a new switch.

Related

• Installing developer preview didn't set up PATH correctly on zsh dune#10963
• Call the env hook only on PWD change in fish? #5193