DOPPLER_ENV=1 crashes poetry

[drawnwren]

Environment

• Doppler Env version: v3.66.5
• Python version: 3.12.1
• Operating System: Arch

Description

python-doppler-env breaks poetry.

Steps to Reproduce

• Using a poetry setup described here
• export DOPPLER_ENV =1
• poetry install

[doppler-env]: DOPPLER_ENV environment variable set. Fetching secrets using Doppler CLI
Loading configuration file /home/wing/.config/pypoetry/config.toml

  StopIteration

  1

  at ~/.conda/envs/xoul_API/lib/python3.12/json/decoder.py:353 in raw_decode
      349│         have extraneous data at the end.
      350│
      351│         """
      352│         try:
    → 353│             obj, end = self.scan_once(s, idx)
      354│         except StopIteration as err:
      355│             raise JSONDecodeError("Expecting value", s, err.value) from None
      356│         return obj, end
      357│

The following error occurred when trying to handle this error:


  Stack trace:

  14  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/cleo/application.py:327 in run
       325│
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  13  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│

  12  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│

  11  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/cleo/application.py:473 in _run_command
       471│
       472│         if error is not None:
     → 473│             raise error
       474│
       475│         return terminate_event.exit_code

  10  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/cleo/application.py:454 in _run_command
       452│
       453│         try:
     → 454│             self._event_dispatcher.dispatch(command_event, COMMAND)
       455│
       456│             if command_event.command_should_run():

   9  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/cleo/events/event_dispatcher.py:26 in dispatch
        24│
        25│         if listeners:
     →  26│             self._do_dispatch(listeners, event_name, event)
        27│
        28│         return event

   8  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/cleo/events/event_dispatcher.py:85 in _do_dispatch
        83│                 break
        84│
     →  85│             listener(event, event_name, self)
        86│
        87│     def _sort_listeners(self, event_name: str) -> None:

   7  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/poetry/console/application.py:299 in configure_env
       297│
       298│         env_manager = EnvManager(poetry, io=io)
     → 299│         env = env_manager.create_venv()
       300│
       301│         if env.is_venv() and io.is_verbose():

   6  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/poetry/utils/env/env_manager.py:444 in create_venv
       442│             # Already inside a virtualenv.
       443│             current_python = Version.parse(
     → 444│                 ".".join(str(c) for c in env.version_info[:3])
       445│             )
       446│             if not self._poetry.package.python_constraint.allows(current_python):

   5  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/poetry/utils/env/base_env.py:77 in version_info
        75│     @property
        76│     def version_info(self) -> tuple[int, int, int, str, int]:
     →  77│         version_info: tuple[int, int, int, str, int] = self.marker_env["version_info"]
        78│         return version_info
        79│

   4  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/poetry/utils/env/base_env.py:95 in marker_env
        93│     def marker_env(self) -> dict[str, Any]:
        94│         if self._marker_env is None:
     →  95│             self._marker_env = self.get_marker_env()
        96│
        97│         return self._marker_env

   3  ~/.conda/envs/xoul_API/lib/python3.12/site-packages/poetry/utils/env/virtual_env.py:72 in get_marker_env
        70│         output = self.run_python_script(GET_ENVIRONMENT_INFO)
        71│
     →  72│         env: dict[str, Any] = json.loads(output)
        73│         return env
        74│

   2  ~/.conda/envs/xoul_API/lib/python3.12/json/__init__.py:346 in loads
       344│             parse_int is None and parse_float is None and
       345│             parse_constant is None and object_pairs_hook is None and not kw):
     → 346│         return _default_decoder.decode(s)
       347│     if cls is None:
       348│         cls = JSONDecoder

   1  ~/.conda/envs/xoul_API/lib/python3.12/json/decoder.py:337 in decode
       335│
       336│         """
     → 337│         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
       338│         end = _w(s, end).end()
       339│         if end != len(s):

  JSONDecodeError

  Expecting value: line 1 column 2 (char 1)

  at ~/.conda/envs/xoul_API/lib/python3.12/json/decoder.py:355 in raw_decode
      351│         """
      352│         try:
      353│             obj, end = self.scan_once(s, idx)
      354│         except StopIteration as err:
    → 355│             raise JSONDecodeError("Expecting value", s, err.value) from None
      356│         return obj, end
      357│

Expected Behavior

When DOPPLER_ENV is unset, poetry installs and manages packages normally. Expected behavior is to not break the package manager.

[watsonian]

@drawnwren Hey! What's your use case for setting DOPPLER_ENV while performing operations with poetry? Typically, the expected use case for this is having it set specifically for your application's environment (e.g., DOPPLER_ENV=1 python app.py). It's not really expected that you would set this inside the bash profile of an interactive shell, for example.

[jermatic]

spent an entire day debugging this in my docker container .. the issue is poetry runs scripts and captures stdout (for like to get info on the current interpreter thats executing) .. if all doppler-env logging is turned off then that resolves the issue, DOPPLER_ENV_LOGGING=0 is not sufficent, it must not be set AT ALL

the JSONDecode error is where poetry takes the recent stdout where it expects the script it called to print json and tries to parse it, but with the added print from doppler-env its obv no longer a valid json string
https://github.com/python-poetry/poetry/blob/master/src/poetry/utils/env/virtual_env.py

my use case is i wanted to set it in my container, so that when i logged into my server , i could use my scripts and cli tool n have env vars available without any extra steps

[watsonian]

my use case is i wanted to set it in my container, so that when i logged into my server , i could use my scripts and cli tool n have env vars available without any extra steps

@jermatic Would using something like direnv work for that use-case? Or alternately, just exporting the variables via the Doppler CLI in your bash profile? Either of those options should allow you to get your environment variables into the shell environment. Or is it that you have DOPPLER_ENV set so that then your scripts will auto-load the environment based on what they're configured to use via doppler setup?

Regardless, it might make sense to update the behavior around DOPPLER_ENV_LOGGING. The logging is disabled by default, but how it's setup causes it to enable if DOPPLER_ENV_LOGGING is set at all (as you mentioned):

python-doppler-env/src/doppler_env/__init__.py

Line 15 in 8bccc28

LOGGING_ENABLED = True if os.environ.get('DOPPLER_ENV_LOGGING') is not None else False

Did you set that to 0 during your troubleshooting? Or had you just set that to 0 as part of your initial setup thinking it needed to be disabled?