Granular "feeling safe" control (#5045)

Instead of one big all-or-nothing switch, finer control over various
dangerous action is needed. Users can still allow everything, with
`--feeling-safe`, or allow individual dangerous functionalities.

Author: happz

agents/AGENTS.md

@@ -69,7 +69,7 @@ make build-deps
 pre-commit run --all-files
 
 # Run the core test plan - runs on developer's workstation
-tmt --feeling-safe -vv run -a provision -h local plan -n '^/plans/features/core$'
+tmt --allow-unsafe-behavior=provision/local -vv run -a provision -h local plan -n '^/plans/features/core$'
 ```
 
 ### Building and Packaging

docs/guide.rst

@@ -62,9 +62,9 @@ directly on your localhost:
 
 .. code-block:: shell
 
-    tmt --feeling-safe run --all provision --how local
+    tmt --allow-unsafe-behavior=provision/local run --all provision --how local
 
-Note that the extra ``--feeling-safe`` option is needed for the
+Note that the extra ``--allow-unsafe-behavior`` option is needed for the
 :ref:`/plugins/provision/local` provision plugin as it can be
 dangerous to execute unknown code directly on your system. If
 you're afraid that the test could break your machine or just want

docs/guide/execute-tests.inc.rst

@@ -377,7 +377,7 @@ directly on your ``local`` host:
 
 .. code-block:: shell
 
-    tmt --feeling-safe run --all provision --how=local
+    tmt --allow-unsafe-behavior=provision/local run --all provision --how=local
 
 In order to reboot a provisioned guest use the ``reboot`` command.
 By default a soft reboot is performed which should prevent data

docs/overview.rst

@@ -397,11 +397,27 @@ TMT_PLUGINS
     separated with the ``:`` character can be provided as well.
 
 TMT_FEELING_SAFE
-    Set this variable to ``1`` to enable potentially dangerous
-    operations such as executing tests directly on the test runner
-    using the ``local`` provision method. Use with caution, only
-    when you can fully trust the ``tmt`` metadata or if you know
-    what you are doing.
+    Set this variable to enable potentially unsafe behavior such
+    as executing tests directly on the test runner using the
+    ``local`` provision method. See ``tmt --help`` for the full
+    documentation, and interaction between ``--feeling-safe`` and
+    ``--allow-unsafe-behavior`` which allows more granular control,
+    and the corresponding ``TMT_ALLOW_UNSAFE_BEHAVIOR`` environment
+    variable.
+
+    Use with caution, only when you can fully trust the ``tmt`` metadata
+    or if you know what you are doing.
+
+TMT_ALLOW_UNSAFE_BEHAVIOR
+    Set this variable to enable potentially unsafe behavior such
+    as executing tests directly on the test runner using the
+    ``local`` provision method. See ``tmt --help`` for the full
+    documentation, and interaction between ``--allow-unsafe-behavior``
+    and ``--feeling-safe`` and the corresponding ``TMT_FEELING_SAFE``
+    environment variable.
+
+    Use with caution, only when you can fully trust the ``tmt`` metadata
+    or if you know what you are doing.
 
 TMT_EXPOSABLE_RUNNER_DEVICES
     Guests may require access to devices of the runner, and only devices

docs/releases/pending/granular-feeling-safe.fmf

@@ -0,0 +1,12 @@
+description: |
+  The :tmt:story:`/stories/features/feeling-safe` is no longer working
+  in the all-or-nothing mode, and new ``--allow-unsafe-behavior`` option
+  now allows enabling unsafe behavior in a granular manner.
+
+  Users of :tmt:story:`/stories/features/feeling-safe` need to account
+  for the following changes:
+
+  * ``--feeling-safe`` works as it used to.
+  * ``--allow-unsafe-behavior`` option now offers the granularity,
+    compared to ``--feeling-safe`` being the all-or-nothing switch.
+  * step-level ``--feeling-safe`` option is no longer supported.

stories/features/feeling-safe.fmf

@@ -22,10 +22,18 @@ description: |
     correct and safe. It would be beneficial to allow informed use
     of these scripts.
 
-    ``tmt`` command gains a new option, ``--feeling-safe``. When
-    specified, tmt will allow and obey possibly unsafe metadata keys
-    and command line options. When not specified, keys and options
-    deemed unsafe would lead to an error.
+    ``tmt`` command has two options, ``--feeling-safe`` and
+    ``--allow-unsafe-behavior``. When specified, tmt will allow and obey
+    possibly unsafe metadata keys and command line options. When not
+    specified, keys and options deemed unsafe would lead to an error.
+
+    It is possible to allow either all unsafe behavior, with
+    ``--feeling-safe`` or ``--allow-unsafe-behavior=all``, or allow just
+    a subset. This may be very handy when the otherwise useful behavior
+    could be constrained by other means user controls. For example, CI
+    maintainer would like to allow custom reboot scripts the
+    :py:ref:`/plugins/provision/connect` plugin supports while still
+    preventing use of the :py:ref:`/plugins/provision/local` plugin.
 
     .. versionadded:: 1.30
 

tests/core/feeling-safe/test.sh

@@ -8,6 +8,7 @@ rlJournalStart
         rlRun "rundir=$(mktemp -d)" 0 "Creating tmp directory"
         rlRun "options='run -i $rundir --scratch provision -h local plan -n /plans/features/core'"
         rlRun "unset TMT_FEELING_SAFE"
+        rlRun "unset TMT_ALLOW_UNSAFE_BEHAVIOR"
     rlPhaseEnd
 
     rlPhaseStartTest "Feeling Paranoid"
@@ -16,18 +17,93 @@ rlJournalStart
         rlAssertNotGrep "User is feeling safe" $rlRun_LOG
 
         # Environment variable
-        rlRun -s "TMT_FEELING_SAFE=0 tmt $options" 2
+        rlRun -s "TMT_FEELING_SAFE= tmt $options" 2
+        rlAssertNotGrep "User is feeling safe" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR= tmt $options" 2
         rlAssertNotGrep "User is feeling safe" $rlRun_LOG
     rlPhaseEnd
 
     rlPhaseStartTest "Feeling Safe"
         # Command line option
-        rlRun -s "tmt --feeling-safe $options"
-        rlAssertGrep "User is feeling safe" $rlRun_LOG
+        rlRun -s "tmt --allow-unsafe-behavior=all $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=1 $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local $options"
+        rlAssertGrep "User is feeling safe: 'local' provisioning plugin allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --allow-unsafe-behavior=all $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --allow-unsafe-behavior=1 $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --feeling-safe $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=none $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=0 $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --allow-unsafe-behavior=all --allow-unsafe-behavior=none $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --allow-unsafe-behavior=all --allow-unsafe-behavior=0 $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --feeling-safe --allow-unsafe-behavior=none $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local --feeling-safe --allow-unsafe-behavior=0 $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
 
         # Environment variable
-        rlRun -s "TMT_FEELING_SAFE=1 tmt $options"
-        rlAssertGrep "User is feeling safe" $rlRun_LOG
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR=all tmt $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR=1 tmt $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR=provision/local tmt $options"
+        rlAssertGrep "User is feeling safe: 'local' provisioning plugin allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local all' tmt $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local 1' tmt $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local' TMT_FEELING_SAFE=1 tmt $options"
+        rlAssertGrep "User is feeling safe: all unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR=none tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR=0 tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local all none' tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local all 0' tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local none' TMT_FEELING_SAFE=1 tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local 0' TMT_FEELING_SAFE=1 tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local all' TMT_FEELING_SAFE=0 tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
+
+        rlRun -s "TMT_ALLOW_UNSAFE_BEHAVIOR='provision/local 1' TMT_FEELING_SAFE=0 tmt $options" 2
+        rlAssertGrep "User is not feeling safe: no unsafe behavior allowed" $rlRun_LOG
     rlPhaseEnd
 
     rlPhaseStartCleanup

tests/plan/select/test.sh

@@ -45,12 +45,12 @@ rlJournalStart
     done
 
     rlPhaseStartTest "tmt plan ls --condition requires --feeling-safe"
-        rlRun -s "TMT_FEELING_SAFE=0 tmt plan ls --condition 'True'" 2
-        rlAssertGrep "requires the '--feeling-safe' option" $rlRun_LOG
+        rlRun -s "unset TMT_FEELING_SAFE; tmt plan ls --condition 'True'" 2
+        rlAssertGrep "'--condition' command-line option is allowed only with the '--allow-unsafe-behavior=cli.condition' or '--feeling-safe' option." $rlRun_LOG
     rlPhaseEnd
 
-    rlPhaseStartTest "tmt plan ls --condition with --feeling-safe"
-        rlRun -s "tmt --feeling-safe plan ls --condition 'True'"
+    rlPhaseStartTest "tmt plan ls --condition with --allow-unsafe-behavior=cli.condition"
+        rlRun -s "tmt --allow-unsafe-behavior=cli.condition plan ls --condition 'True'"
         rlAssertGrep "/plans/features/core" $rlRun_LOG
         rlAssertGrep "/plans/features/basic" $rlRun_LOG
     rlPhaseEnd

tests/policy/test-test.sh

@@ -83,7 +83,7 @@ rlJournalStart
             local option="$1"
             local envvar="$2"
 
-            rlRun -s "$envvar tmt --feeling-safe -vv run --id $run --scratch $option discover provision -h local execute report -h display -vvv plan --default test --name /basic"
+            rlRun -s "$envvar tmt --allow-unsafe-behavior=provision/local -vv run --id $run --scratch $option discover provision -h local execute report -h display -vvv plan --default test --name /basic"
 
             rlAssertGrep "content: Spiked test." $rlRun_LOG
             rlAssertEquals \
@@ -131,10 +131,10 @@ rlJournalStart
                 policy_name_envvar=""
             fi
 
-            rlRun -s "tmt --feeling-safe -vv run --id $run --scratch $policy_root_option $policy_file_option $policy_name_option discover provision -h local execute report -h display -vvv plan --default test --name /basic" "$expected_code"
+            rlRun -s "tmt --allow-unsafe-behavior=provision/local -vv run --id $run --scratch $policy_root_option $policy_file_option $policy_name_option discover provision -h local execute report -h display -vvv plan --default test --name /basic" "$expected_code"
             rlAssertGrep "$expected_error" $rlRun_LOG
 
-            rlRun -s "$policy_root_envvar $policy_file_envvar $policy_name_envvar tmt --feeling-safe -vv run --id $run --scratch discover provision -h local execute report -h display -vvv plan --default test --name /basic" "$expected_code"
+            rlRun -s "$policy_root_envvar $policy_file_envvar $policy_name_envvar tmt --allow-unsafe-behavior=provision/local -vv run --id $run --scratch discover provision -h local execute report -h display -vvv plan --default test --name /basic" "$expected_code"
             rlAssertGrep "$expected_error" $rlRun_LOG
         }
 

tests/provision/local/test.sh

@@ -28,7 +28,7 @@ rlJournalStart
 
     rlPhaseStartTest "Test script installation"
         # Run tmt with local provision
-        rlRun -s "tmt --feeling-safe run --id $run -vvv"
+        rlRun -s "tmt --allow-unsafe-behavior=provision/local run --id $run -vvv"
         rlAssertNotGrep "No such file or directory" $rlRun_LOG
         rlAssertNotGrep "command not found" $rlRun_LOG
         rlAssertGrep "pass /test/beakerlib" $rlRun_LOG
@@ -43,7 +43,7 @@ rlJournalStart
         done
 
         # Check that warning is raised when TMT_SCRIPTS_DIR is used
-        rlRun -s "TMT_SCRIPTS_DIR=$tmp tmt --feeling-safe run --scratch --id $run -vvv"
+        rlRun -s "TMT_SCRIPTS_DIR=$tmp tmt --allow-unsafe-behavior=provision/local run --scratch --id $run -vvv"
         rlAssertGrep "The 'TMT_SCRIPTS_DIR' variable is not supported" $rlRun_LOG
     rlPhaseEnd