fix(docker): Add default crontab download command (#1321)

* docs(docker): Recommended image DEFAULT_WORKSPACE

* fix(docker): Add default crontab download command

Clarify the `/config/ytdl-sub-configs/cron` script with explanatory comments and a
default `--dry-run` command. Also change the wrapper script to echo commands for easier
debugging. To update an existing script, move the old script aside, restart the
container to regenerate it, and edit the new script.

Also clarify the Automating page in the Getting Started guide docs.

* fix(docker): Unintentional unattended dry runs

[PR feedback](#1321 (comment))
prompted me to reconsider having a default command at all. We should assume,
unfortunately, that many new users will just skim the docs enough to enable the image's
cron integration but not actually incrementally test their configuration. In those
cases, they'd end up sending dry-run non-download requests for all their subscriptions
every 6 hours for no good reason. There's just no way to provide a default command that
isn't also providing a footgun.

* docs(docker): More open cron schedule generator

From [PR
feedback](#1321 (comment)), this
seems like less of an ad than the previous and the source for the page is itself open source.

* docs(docker): Document image environment footgun

From [PR
feedback](#1321 (comment)).

* docs(automate): Avoid external link 404 responses

* docs(automate): False simultaneous run warning

* docs(automate): Clarify Docker daemon restarts

* docs(automate): Revert run start non-recommended

* docs(automate): Remove footgun manual run command

Addressing this underlying issue requires more thought and should be a separate PR.

* docs(automate): Restore env var footgun in example

Author: rpatterson

README.md

@@ -68,7 +68,7 @@ __preset__:
 
   # Pass any arg directly to yt-dlp's Python API
   ytdl_options:
-    cookiefile: "/config/cookie.txt" 
+    cookiefile: "/config/ytdl-sub-configs/cookie.txt"
 
 ###################################################################
 # TV Show Presets. Can replace Plex with Plex/Jellyfin/Emby/Kodi

docker/root/custom-cont-init.d/defaults

@@ -47,6 +47,8 @@ if [ "$CRON_SCHEDULE" != "" ] ; then
 
     # create cron script wrapper
     echo '#!/bin/bash' > "$CRON_WRAPPER_SCRIPT"
+    # Echo commands for easier user debugging:
+    echo "set -x" >> "$CRON_WRAPPER_SCRIPT"
     echo "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" >> "$CRON_WRAPPER_SCRIPT"
     echo "cd \"$DEFAULT_WORKSPACE\"" >> "$CRON_WRAPPER_SCRIPT"
     echo ". \"$CRON_SCRIPT\" >> \"$LOGS_TO_STDOUT\" 2>&1" >> "$CRON_WRAPPER_SCRIPT"

docker/root/defaults/cron

@@ -1,4 +1,15 @@
-echo "Beginning cron job..."
-
 # Place your ytdl-sub command(s) here.
-# This script is executed in the same relative path as this file.
+#
+# This script is executed in the same directory as this file which also contains the
+# default `./config.yaml` and `./subscriptions.yaml`, so you don't need to use the
+# `--config` CLI option or pass a `SUBPATH` to the `$ ytdl-sub sub` sub-command.
+#
+# Test your configuration and subscriptions carefully before automating downloads to
+# prevent triggering throttles or bans:
+#
+# https://ytdl-sub.readthedocs.io/en/latest/guides/getting_started/downloading.html
+#
+# Once you've tested your configuration and you're ready to download entries unattended,
+# remove the next line and un-comment the following line:
+echo "WARNING: Read /config/ytdl-sub-configs/cron and modify to automate downloads."
+# ytdl-sub sub

docker/root/defaults/subscriptions.yaml

@@ -20,7 +20,7 @@ __preset__:
 
   # Pass any arg directly to yt-dlp's Python API
   # ytdl_options:
-  #   cookiefile: "/config/cookie.txt"
+  #   cookiefile: "/config/ytdl-sub-configs/cookie.txt"
 
 ###################################################################
 # Subscriptions nested under this will use the

docs/source/config_reference/subscription_yaml.rst

@@ -31,7 +31,7 @@ supply a cookies file path.
 
     # Directly set plugin options:
     ytdl_options:
-      cookiefile: "/config/cookie.txt"
+      cookiefile: "/config/ytdl-sub-configs/cookie.txt"
 
 Layout
 ------

docs/source/guides/getting_started/automating_downloads.rst

@@ -1,69 +1,132 @@
-Automating Downloads
-====================
+Automating
+==========
 
-:ref:`Guide for Docker and Unraid Containers <guides/getting_started/automating_downloads:docker and unraid>`
+Automate downloading your subscriptions by running the :ref:`'sub' sub-command
+<usage:subscriptions options>` periodically. There are various tools that can run
+commands on a schedule you may use any of them that work with your installation
+method. Most users use `cron`_ in `Docker containers <docker and unraid_>`_.
 
-:ref:`Guide for Linux <guides/getting_started/automating_downloads:linux>`
-
-:ref:`Guide for Windows <guides/getting_started/automating_downloads:windows>`
-
-.. _cron scheduling syntax: https://crontab.guru/#0_*/6_*_*_*
-
-
-.. _docker-unraid-setup:
 
 Docker and Unraid
 -----------------
 
-Cron is preconfigured in every ytdl-sub docker container. Enable by adding the following
-ENV variables to your docker setup.
+:doc:`The 'ytdl-sub' Docker container images <../install/docker>` provide optional cron
+support. Enable cron support by setting `a cron schedule`_ in the ``CRON_SCHEDULE``
+environment variable:
 
 .. code-block:: yaml
+  :caption: ./compose.yaml
+  :emphasize-lines: 4
 
   services:
     ytdl-sub:
       environment:
-        - CRON_SCHEDULE="0 */6 * * *"
-        - CRON_RUN_ON_START=false
+        CRON_SCHEDULE: "0 */6 * * *"
+	# WARNING: See "Getting Started" -> "Automating" docs regarding throttles/bans:
+	# CRON_RUN_ON_START: false
+
+Then recreate the container to apply the change and start it to generate the default
+``/config/ytdl-sub-configs/cron`` script. Read the comments in that script and edit as
+appropriate.
+
+The container cron wrapper script will write output from the cron job to
+``/config/ytdl-sub-configs/.cron.log``. The default image ``ENTRYPOINT`` will ``$ tail
+...`` that file so you can monitor the cron job in the container's output and thus also
+in the Docker logs.
 
+You may also set the ``CRON_RUN_ON_START`` environment variable to ``true`` to have the
+image run your cron script whenever the container starts in addition to the cron
+schedule.
 
-- ``CRON_SCHEDULE`` follows the standard `cron scheduling syntax`_. The above value will
-  run the script once every 6 hours.
-- ``CRON_RUN_ON_START`` toggles whether to run your cron script on container start in
-  addition to the cron schedule.
+.. warning::
 
-The cron script will reside in the main directory with the file name ``cron``.  Cron
-logs should show when viewing the Docker logs.
+   Using ``CRON_RUN_ON_START`` may cause your cron script to run too often and may
+   trigger throttles and bans. When enabled, your cron script will run *whenever* the
+   container starts including when the host reboots, when ``# dockerd`` restarts such as
+   when upgrading Docker itself, when a new image is pulled, when something applies
+   Compose changes, etc.. This may result in running ``ytdl-sub`` right before or after
+   the next cron scheduled run.
 
 
 .. _linux-setup:
 
-Linux 
------
-Must configure crontab manually, like so:
+Linux, Mac OS X, BSD, or other UNIX's
+-------------------------------------
+
+For installations on systems already running ``# crond``, you can also use cron to run
+``ytdl-sub`` periodically. Write a script to run ``ytdl-sub`` in the cron job. Be sure
+the script changes to the same directory as your configuration and uses the full path to
+``ytdl-sub``:
 
 .. code-block:: shell
+   :caption: ~/.local/bin/ytdl-sub-cron
+   :emphasize-lines: 2,3
+
+   #!/bin/bash
+   cd "~/.config/ytdl-sub/"
+   ~/.local/bin/ytdl-sub --dry-run sub -o '--ytdl_options.max_downloads 3' |&
+       tee -a "~/.local/state/ytdl-sub/.cron.log"
+
+Then tell ``# crond`` when to run the script:
 
-  crontab -e
-  0     */6     *       *       *       /config/run_cron
+.. code-block:: console
+
+   echo "0 */6 * * * ${HOME}/.local/bin/ytdl-sub-cron" | crontab "-"
+
+Remove the ``--dry-run`` and ``-o ...`` CLI options from your cron script when you've
+tested your configuration and you're ready to download entries unattended.
 
 
 .. _windows-setup:
 
 Windows
 -------
 
-To be tested (please contact code owner or join the discord server if you can test this
-out for us)
+For most Windows users, the best way to run commands periodically is `the Task
+Scheduler`_:
+
+.. attention::
 
-.. code-block:: powershell
+   These instructions are untested. Use at your own risk. If you use them, whether they
+   work or not, please let us know how it went in `a support post in Discord`_ or `a new
+   GitHub issue`_.
 
-  ytdl-sub.exe --config \path\to\config\config.yaml sub \path\to\config\subscriptions.yaml
+#. Open the Task Scheduler app.
+
+#. Click ``Create Basic Task`` at the top of the right sidebar.
+
+#. Set all the fields as appropriate until you get to the ``Action``...
+
+#. For the ``Action``, select ``Start a program``...
+
+#. Click ``Browse...`` to the installed ``ytdl-sub.exe`` executable...
+
+#. Add CLI arguments to ``Add arguments (optional):``, for example ``--dry-run sub -o
+   '--ytdl_options.max_downloads 3'``...
+
+#. Set ``Start in (optional):`` to the directory containing your configuration.
+
+#. Finish the rest of the ``Create Basic Task`` wizard.
 
 
 Next Steps
 ----------
 
-Once you have a significant quantity of subscriptions or have use cases not served using
-:doc:`YAML keys and the special characters <./subscriptions>`, it's time to start
-:doc:`defining your own custom presets <./first_config>`.
+At this point, ``ytdl-sub`` should run periodically and keep your subscriptions current
+in your media library without your intervention. As your :doc:`subscriptions file
+<./subscriptions>` grows or you discover new use cases, it becomes worth while to
+simplify things by :doc:`defining your own custom presets <./first_config>`.
+
+
+
+.. _`cron`:
+   https://en.wikipedia.org/wiki/Cron
+.. _`a cron schedule`:
+   https://crontab.cronhub.io/
+
+.. _`the Task Scheduler`:
+   https://learn.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page
+.. _`a support post in Discord`:
+   https://discord.com/channels/994270357957648404/1084886228266127460
+.. _`a new GitHub issue`:
+   https://github.com/jmbannon/ytdl-sub/issues/new

docs/source/guides/install/docker.rst

@@ -38,6 +38,13 @@ For example::
 
   $ docker compose run --rm --user="${PUID}:${PGID}" --entrypoint="ytdl-sub" ytdl-sub sub
 
+.. note::
+
+   In `the recommended GUI image <gui image_>`_, the ``DEFAULT_WORKSPACE`` directory is
+   ``/config/ytdl-sub-configs/`` which is used throughout the documentation and
+   examples. In the headless images, that directory is just ``/config/``, so substitute
+   that path if using a headless image.
+
 
 Install with Docker Compose
 ---------------------------

examples/advanced/tv_show_config.yaml

@@ -69,7 +69,7 @@ presets:
     # ytdl_options lets you pass any arg into yt-dlp's Python API
     ytdl_options:
       # Set the cookie file
-      # cookiefile: "/config/youtube_cookies.txt"
+      # cookiefile: "/config/ytdl-sub-configs/youtube_cookies.txt"
 
       # For YouTube, get English metadata if multiple languages are present
       extractor_args: