MRG: Merge pull request #739 from octue/rewrite-docs

Improve CLI  and rewrite documentation

Author: cortadocodes

.pre-commit-config.yaml

@@ -9,7 +9,6 @@ repos:
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer
-      - id: check-yaml
       - id: check-added-large-files
 
   - repo: https://github.com/astral-sh/ruff-pre-commit

README.md

@@ -11,7 +11,7 @@
 The python SDK for running [Octue](https://octue.com) data services, digital twins, and applications - get faster data
 groundwork so you have more time for the science!
 
-Read the docs [here.](https://octue-python-sdk.readthedocs.io/en/latest/)
+Read the docs [here.](https://twined.octue.com/)
 
 Uses our [twined](https://twined.readthedocs.io/en/latest/) library for data validation.
 
@@ -41,7 +41,6 @@ Usage: octue [OPTIONS] COMMAND [ARGS]...
   Read more in the docs: https://octue-python-sdk.readthedocs.io/en/latest/
 
 Options:
-  --logger-uri TEXT               Stream logs to a websocket at the given URI.
   --log-level [debug|info|warning|error]
                                   Log level used for the analysis.  [default:
                                   info]

docs/authentication.md

@@ -1,37 +1,35 @@
-# Authentication
+You need authentication while using Twined to:
 
-You need authentication while using `octue` to:
-
-- Access data from Google Cloud Storage
-- Use, run, or deploy Twined services
+- Use or run services
+- Access input and output data from analyses run on services
 
 Authentication is provided by a GCP service account.
 
 ## Creating a service account
 
 By setting up your Twined service network with the
-[Twined Terraform modules](/deploying_services), a set of maintainer service accounts have already been
-created with the required permissions.
+[Twined Terraform modules](core_concepts/deploying_services.md), a set of maintainer service accounts have already been
+created with the required permissions. These will have names starting with `maintainer-`.
 
 ## Using a service account
 
 ### Locally
 
 1.  Access your service accounts
     [here](https://console.cloud.google.com/iam-admin/serviceaccounts),
-    making sure the correct project is selected
+    making sure the correct project is selected, or ask your Twined service network administrator
 2.  Click on the relevant service account, go to the "Keys" tab, and
     create (download) a JSON key for it - it will be called
     `<project-name>-XXXXX.json`.
 
     !!! danger
 
-        It's best not to store this in your project to prevent accidentally
+        It's best not to store this in your repository to prevent accidentally
         committing it or building it into a docker image layer. Instead, keep it
         somewhere else on your local system with any other service account keys
         you already have.
 
-        If you must keep within your project, it's good practice to name the
+        If you must keep within your repository, it's good practice to name the
         file `gcp-credentials.json` and make sure that `gcp-cred*` is in your
         `.gitignore` and `.dockerignore` files.
 
@@ -46,8 +44,7 @@ created with the required permissions.
 
 ### On GCP infrastructure / deployed services
 
-- Credentials are automatically provided when running code or services
-  on GCP infrastructure, including the Kubernetes cluster
-- `octue` uses these when when running on these platforms, so there's
-  no need to upload a service account key or include one in service
-  docker images
+- Credentials are automatically provided when running code or services on GCP infrastructure, including the Kubernetes
+  cluster
+- Twined uses these when running on these platforms, so there's no need to upload a service account key or include one
+  in service docker images

docs/available_filters.md

@@ -21,7 +21,7 @@ to a child for processing/analysis. Questions can be:
     or custom logic in your own webserver.
 
 Questions are always asked to a _revision_ of a service. You can ask a
-service a question if you have its [SRUID](/services/#service-names), project ID, and the necessary permissions.
+service a question if you have its [SRUID](services.md/#service-names), project ID, and the necessary permissions.
 
 ## Asking a question
 
@@ -49,7 +49,7 @@ answer["output_manifest"]["my_dataset"].files
 
     If you're using an environment other than the `main` environment, then before asking any questions to your Twined
     services, set the `TWINED_SERVICES_TOPIC_NAME` environment variable to the name of the Twined services Pub/Sub topic
-    (this is set when [deploying a service network](/deploying_services/#deploying-services-advanced-developers-guide).
+    (this is set when [deploying a service network](deploying_services.md).
     It will be in the form `<environment-name>.octue.twined.services`
 
 !!! note
@@ -64,7 +64,7 @@ You can also set the following options when you call `Child.ask`:
 
 - `children` - If the child has children of its own (i.e. grandchildren of the parent), this optional argument can be
   used to override the child's "default" children. This allows you to specify particular versions of grandchildren to
-  use (see [this subsection below](#overriding-a-childs-children).
+  use (see [this subsection below](#overriding-a-childs-children)).
 - `subscribe_to_logs` - if true, the child will forward its logs to you
 - `allow_local_files` - if true, local files/datasets are allowed in any input manifest you supply
 - `handle_monitor_message` - if provided a function, it will be called on any monitor messages from the child
@@ -242,10 +242,10 @@ at once instead of one after another.
 ## Asking a question within a service
 
 If you have
-[created your own Twined service](/creating_services) and want to ask children questions, you can do this more
+[created your own Twined service](creating_services.md) and want to ask children questions, you can do this more
 easily than above. Children are accessible from the `analysis` object by
 the keys you give them in the
-[service configuration](/creating_services/#octueyaml) file. For example, you can ask an `elevation` service a
+[service configuration](creating_services.md/#octueyaml) file. For example, you can ask an `elevation` service a
 question like this:
 
 ```python
@@ -314,7 +314,7 @@ you want it to use (dynamic children) to the
 children will instead go to the dynamic children. Note that:
 
 - You must provide the children in the same format as they're provided
-  in the [service configuration](/creating_services/#octueyaml)
+  in the [service configuration](../core_concepts/creating_services.md/#octueyaml)
 - If you override one static child, you must override others, too
 - The dynamic children must have the same keys as the static children
   (so the child knows which service to ask which questions)

docs/asking_questions.md renamed to docs/core_concepts/asking_questions.md

@@ -1,4 +1,4 @@
-# app.py file {#creating_apps}
+# The `app.py` file
 
 The `app.py` file is, as you might expect, the entrypoint to your app.
 It can contain any valid python including imports and use of any number

docs/creating_apps.md renamed to docs/core_concepts/creating_apps.md

@@ -12,19 +12,16 @@ run locally on any machine or be deployed to the cloud. Currently:
 
 ## Anatomy of a Twined service
 
-A Twined service is defined by the following files (located in the
-repository root by default).
+A Twined service is defined by the following files (located in the repository root by default).
 
 ### app.py
 
-This is the entrypoint into your code - read more [here](/creating_apps).
+This is the entrypoint into your code - read more [here](creating_apps.md).
 
 ### twine.json
 
-This file defines the schema for the service's configuration, input,
-and output data. Read more
-[here](https://twined.readthedocs.io/en/latest/) and see an example
-[here](https://twined.readthedocs.io/en/latest/quick_start_create_your_first_twine.html).
+This file defines the schema for the service's configuration, input, and output data. Read more [here](twines/anatomy.md) and
+see an example [here](twines/twine_file_quickstart.md).
 
 ### Dependencies file
 
@@ -45,26 +42,26 @@ are supported.
 
     ``` yaml
     services:
-     - namespace: my-organisation 
+     - namespace: my-organisation
        name: my-app
     ```
     It may also need the following key-value pairs:
-   
+
     - `app_source_path: <path>` - if your `app.py` file is not in the repository root
-   
+
     All paths should be relative to the repository root. Other valid
     entries can be found in the `ServiceConfiguration` constructor.
-   
+
     !!! warning
-   
+
         Currently, only one service can be defined per repository, but it must
         still appear as a list item of the "services" key. At some point, it
         will be possible to define multiple services in one repository.
-   
+
     If a service's app needs any configuration, asks questions to any
     other Twined services, or produces output datafiles/datasets, you will
     need to provide some or all of the following values for that service:
-   
+
     - `configuration_values`
     - `configuration_manifest`
     - `children`
@@ -95,36 +92,35 @@ are supported.
     - [An OpenFAST service](https://github.com/octue/openfast-service/blob/main/Dockerfile)
 
     If you do provide one, you must provide its path relative to your
-    repository to the [build-twined-services] GitHub Actions [workflow](https://github.com/octue/workflows/blob/main/.github/workflows/build-twined-service.yml).
+    repository to the `build-twined-services` GitHub Actions [workflow](https://github.com/octue/workflows/blob/main/.github/workflows/build-twined-service.yml).
 
     As always, if you need help with this, feel free to drop us a message or raise an issue!
 
-
 ### Where to specify the namespace, name, and revision tag
 
-See [here](/services/#service-names) for service naming requirements.
+See [here](services.md/#service-names) for service naming requirements.
 
 **Namespace**
 
 - Required: yes
 - Set in:
-    - `octue.yaml`
-    - `OCTUE_SERVICE_NAMESPACE` environment variable (takes priority)
+  - `octue.yaml`
+  - `OCTUE_SERVICE_NAMESPACE` environment variable (takes priority)
 
 **Name**
 
 - Required: yes
 - Set in:
-    - `octue.yaml`
-    - `OCTUE_SERVICE_NAME` environment variable (takes priority)
+  - `octue.yaml`
+  - `OCTUE_SERVICE_NAME` environment variable (takes priority)
 
 **Revision tag**
 
 - Required: no
 - Default: a random "coolname" (e.g. `hungry-hippo`)
 - Set in:
-    - `OCTUE_SERVICE_REVISION_TAG` environment variable
-    - If using `octue twined service start` command, the `--revision-tag` option (takes priority)
+  - `OCTUE_SERVICE_REVISION_TAG` environment variable
+  - If using `octue twined service start` command, the `--revision-tag` option (takes priority)
 
 ## Template apps
 
@@ -153,4 +149,4 @@ Automated deployment with Octue means:
   other services are sent to it, meaning there are minimal costs to
   having it deployed but not in use.
 
-If you'd like help deploying services, contact us. To do it yourself, see [here](/deploying_services).
+If you'd like help deploying services, [contact us](../support.md). To do it yourself, see [here](deploying_services.md).