Skip to content

Commit 898faa3

Browse files
committed
DOC: Add getting started guide for creating services
skipci
1 parent ebdef0f commit 898faa3

File tree

1 file changed

+142
-0
lines changed

1 file changed

+142
-0
lines changed

docs/creating_services/getting_started.md

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
Turn your analysis code into production-ready services - no infrastructure or DevOps skills needed - and share with
22
your team and beyond.
33

4+
This guide walks you through creating an example Twined service deployed in the cloud. By the end, you'll have a real
5+
service and be able to ask it questions from your computer and receive answers from it over the internet.
6+
47
## Prerequisites
58

69
Before you begin, ensure you:
@@ -11,5 +14,144 @@ Before you begin, ensure you:
1114
- Have the following tools installed:
1215
- Python >= 3.10
1316
- The `octue` python library / CLI (see [installation instructions](../installation.md))
17+
- Have access to an existing Twined services network - see [authentication instructions](/creating_services/authentication)
18+
- Have the ability to create a GitHub repository under the same GitHub account or organisation as used for the
19+
[`github_account` variable](https://github.com/octue/terraform-octue-twined-core?tab=readme-ov-file#input-reference)
20+
used for the Twined services network
1421

1522
<!-- prettier-ignore-end -->
23+
24+
## Create and clone a GitHub repository
25+
26+
Create a GitHub repository for the service. It must be owned by the GitHub account used as the `github_account` variable
27+
for the Twined services network. This variable is an input to the
28+
[Twined core Terraform module](https://github.com/octue/terraform-octue-twined-core?tab=readme-ov-file#input-reference)
29+
and should be stated in a `variables.tf` or another `.tf` file in the Terraform configuration used to deploy the services
30+
network. Ask the person who created or manages your infrastructure if you're not sure.
31+
32+
Clone this repository to your computer
33+
34+
```shell
35+
git clone <my-repository>
36+
```
37+
38+
## Install the python dependencies
39+
40+
Create a `pyproject.toml` file to define the service as a python package and list its dependencies:
41+
42+
```toml
43+
[tool.poetry]
44+
name = "example-service"
45+
version = "0.1.0"
46+
description = "An example Twined data service."
47+
authors = ["Your name <your email>"]
48+
packages = [{include = "example_service"}]
49+
50+
[tool.poetry.dependencies]
51+
python = "^3.11"
52+
octue = "0.69.0"
53+
54+
[build-system]
55+
requires = ["poetry-core"]
56+
build-backend = "poetry.core.masonry.api"
57+
```
58+
59+
!!! tip
60+
61+
We use Poetry in this example but you can use Pip or another package manager with either a `setup.py` or
62+
`pyproject.toml` file.
63+
64+
## Write the service python code
65+
66+
We'll make the example service do the most basic thing possible - return dummy output values and a dummy output dataset.
67+
68+
Create a directory called `example_service` with an empty `__init__.py` file and an `app.py` file inside. In `app.py`,
69+
add the following code:
70+
71+
```python
72+
import logging
73+
import os
74+
import tempfile
75+
import time
76+
77+
from octue.resources import Datafile, Dataset
78+
79+
logger = logging.getLogger(__name__)
80+
81+
82+
def run(analysis):
83+
logger.info("Started example analysis.")
84+
time.sleep(2)
85+
analysis.output_values = [1, 2, 3, 4, 5]
86+
87+
with tempfile.TemporaryDirectory() as temporary_directory:
88+
with Datafile(os.path.join(temporary_directory, "output.dat"), mode="w") as (datafile, f):
89+
f.write("This is some example service output.")
90+
91+
analysis.output_manifest.datasets["example_dataset"] = Dataset(path=temporary_directory, files={datafile})
92+
93+
logger.info("Finished example analysis.")
94+
```
95+
96+
## Add the twine file
97+
98+
The Twine file is a JSON file stating what kind of data is expected as inputs and outputs of the service. It shows users
99+
what can be sent to the service and what to expect as output. Inputs and outputs that violate the schemas won't be
100+
processed and will cause an error.
101+
102+
Create a file at the top level of the repository called `twine.json`:
103+
104+
```json
105+
{
106+
"input_values_schema": {
107+
"type": "object",
108+
"required": ["some_input"],
109+
"properties": {
110+
"some_input": {
111+
"type": "integer"
112+
}
113+
}
114+
},
115+
"output_values_schema": {
116+
"title": "Output values",
117+
"description": "Some dummy output data.",
118+
"type": "array",
119+
"items": {
120+
"type": "number"
121+
}
122+
},
123+
"output_manifest": {
124+
"datasets": {
125+
"example_dataset": {}
126+
}
127+
}
128+
}
129+
```
130+
131+
## Add the service configuration file
132+
133+
The service configuration file names the service and tells it things like where to store output data. Create an
134+
`octue.yaml` file, replacing `<handle>` with the value of `github_account` mentioned above. Check the other values with
135+
whoever manages your Twined service network (they can find them in the outputs of the Terraform modules used to deploy
136+
the service network).
137+
138+
```yaml
139+
services:
140+
- namespace: <handle>
141+
name: example-service
142+
app_source_path: example_service
143+
144+
# Get these from whoever manages your Twined service network.
145+
event_store_table_id: octue_twined.service-events
146+
diagnostics_cloud_path: gs://<GCP project name>-octue-twined/example-service/diagnostics
147+
output_location: "gs://<GCP project name>-octue-twined/example-service/outputs"
148+
service_registries:
149+
- name: <handle>'s services
150+
endpoint: https://europe-west9-octue-twined-services.cloudfunctions.net/<environment>-octue-twined-service-registry
151+
```
152+
153+
## Add the GitHub Actions reusable workflow
154+
155+
## Send the service its first question
156+
157+
## Next steps

0 commit comments

Comments
 (0)