-
Notifications
You must be signed in to change notification settings - Fork 15
Expand file tree
/
Copy pathstart_here.py
More file actions
51 lines (37 loc) · 1.67 KB
/
Copy pathstart_here.py
File metadata and controls
51 lines (37 loc) · 1.67 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
"""
This is a very simple script to show the basic instrumentation capabilities of Flowcept, using its most straightforward
way of capturing workflow provenance from functions: using @decorators. It is meant to be executed in offline model.
Flowcept will flush its internal buffer to a simple JSONL file at the end of the run.
This very simple scenario does not need any database, streaming service, message queue or any other external service.
It should run fine after installing Flowcept via `pip install flowcept`.
For more complex features, such as online provenance analysis, HPC requirements, federated/highly distributed execution,
data observability from existing adapters, PyTorch models, telemetry capture optimization, query requirements, or
any other provided feature or custom requirements, see the rest of examples/ directory and Flowcept docs.
Note:
- Adding output_names is not required, but they will make the generated provenance look nicer (and more semantic).
"""
from flowcept import Flowcept, flowcept_task
from flowcept.instrumentation.flowcept_decorator import flowcept
@flowcept_task(output_names="o1")
def sum_one(i1):
return i1+1
@flowcept_task(output_names="o2")
def mult_two(o1):
return o1*2
@flowcept
def main():
"""
This contains the workflow code.
"""
n = 3
o1 = sum_one(n)
o2 = mult_two(o1)
print("Final output", o2)
if __name__ == "__main__":
main()
# Reporting and verifications:
raw_records = Flowcept.read_buffer_file()
assert len(raw_records) == 2
workflow_card_path = "WORKFLOW_CARD.md"
Flowcept.generate_report(output_path=workflow_card_path)
print(f"{workflow_card_path} generated!")