[WIP] Enclaved reactors

[erlingrj]

This PR introduces enclaves into reactor-uc. To do so we must extend the platform API with a notion of threads, because each enclave executes in its own execution context. Currently the threading API is only implemented in POSIX and enclaves can thus still be considered experimental.

Enclaves behave as federates and are coordinated decentrally. This means that we could get STP violations and that we do not support ZDC cycles between enclaves

[github-actions]

Memory usage after merging this PR will be:

Memory Report

action_empty_test_c

	from	to	increase (%)
text	69205	70742	2.22
data	752	768	2.13
bss	12416	12640	1.80
total	82373	84150	2.16

action_microstep_test_c

	from	to	increase (%)
text	70066	71603	2.19
data	760	776	2.11
bss	12480	12704	1.79
total	83306	85083	2.13

action_overwrite_test_c

	from	to	increase (%)
text	69911	71445	2.19
data	752	768	2.13
bss	12480	12704	1.79
total	83143	84917	2.13

action_test_c

	from	to	increase (%)
text	69807	71341	2.20
data	760	776	2.11
bss	12480	12704	1.79
total	83047	84821	2.14

deadline_test_c

	from	to	increase (%)
text	64694	66231	2.38
data	768	784	2.08
bss	11776	12000	1.90
total	77238	79015	2.30

delayed_conn_test_c

	from	to	increase (%)
text	70181	71786	2.29
data	752	768	2.13
bss	13248	13472	1.69
total	84181	86026	2.19

event_payload_pool_test_c

	from	to	increase (%)
text	24353	24927	2.36
data	720	736	2.22
bss	480	512	6.67
total	25553	26175	2.43

event_queue_test_c

	from	to	increase (%)
text	29092	29711	2.13
data	728	744	2.20
bss	480	512	6.67
total	30300	30967	2.20

nanopb_test_c

	from	to	increase (%)
text	44114	44114	0.00
data	1864	1864	0.00
bss	320	320	0.00
total	46298	46298	0.00

physical_clock_test_c

	from	to	increase (%)
text	48646	49822	2.42
data	832	864	3.85
bss	640	864	35.00
total	50118	51550	2.86

port_test_c

	from	to	increase (%)
text	70084	71689	2.29
data	752	768	2.13
bss	13056	13280	1.72
total	83892	85737	2.20

reaction_queue_test_c

	from	to	increase (%)
text	28292	28984	2.45
data	728	744	2.20
bss	480	512	6.67
total	29500	30240	2.51

request_shutdown_test_c

	from	to	increase (%)
text	70044	71583	2.20
data	752	768	2.13
bss	12480	12704	1.79
total	83276	85055	2.14

startup_test_c

	from	to	increase (%)
text	64291	65819	2.38
data	760	776	2.11
bss	11776	12000	1.90
total	76827	78595	2.30

tcp_channel_test_c

	from	to	increase (%)
text	91057	92067	1.11
data	2224	2224	0.00
bss	14720	14944	1.52
total	108001	109235	1.14

timer_test_c

	from	to	increase (%)
text	64295	65829	2.39
data	752	768	2.13
bss	11776	12000	1.90
total	76823	78597	2.31

[github-actions]

Benchmark results after merging this PR:

Benchmark results

Performance:

PingPongUc:
Best Time: 234.832 msec
Worst Time: 244.053 msec
Median Time: 235.520 msec

PingPongC:
Best Time: 167.611 msec
Worst Time: 174.255 msec
Median Time: 167.229 msec

ReactionLatencyUc:
Best latency: 44036 nsec
Median latency: 60889 nsec
Worst latency: 101832 nsec

ReactionLatencyC:
Best latency: 49760 nsec
Median latency: 61005 nsec
Worst latency: 96543 nsec

Memory usage:

PingPongUc:
text data bss dec hex filename
40822 776 8488 50086 c3a6 bin/PingPongUc

PingPongC:
text data bss dec hex filename
45826 880 360 47066 b7da bin/PingPongC

ReactionLatencyUc:
text data bss dec hex filename
26803 752 2568 30123 75ab bin/ReactionLatencyUc

ReactionLatencyC:
text data bss dec hex filename
41536 848 360 42744 a6f8 bin/ReactionLatencyC

[github-actions]

Coverage after merging enclaves into main will be

53.87%

Coverage Report

File	Stmts	Branches	Funcs	Lines	Uncovered Lines
src
action.c	70%	65.38%	62.50%	71.88%	113–114, 127–128, 131–134, 137–138, 140–143, 145, 149, 152–155, 25, 40–42, 46, 48, 55–56, 63–65, 65, 65–68
builtin_triggers.c	90.91%	70%	100%	96.77%	14, 18, 40, 43
clock_synchronization.c	0%	0%	0%	0%	101–104, 106–107, 107, 107–110, 115–117, 117, 117–119, 121–123, 123, 123–125, 125, 125, 125, 125–127, 130–131, 133, 136, 138–139, 14, 140, 142, 144, 144, 144–146, 148–149, 15, 150, 150, 150–152, 155, 157, 159, 161–162, 162, 162–164, 164, 164, 167, 167, 167–168, 17, 171–176, 176, 176–177, 18, 181, 183–189, 189, 189, 19, 190, 193, 196–199, 20, 200–201, 201, 201–202, 204, 206, 209, 21, 211–215, 215, 215–216, 216, 216–225, 225, 225–226, 228, 231–233, 235–237, 239, 239, 239, 24, 24, 24, 240, 242, 245–246, 248–249, 25, 250–255, 255, 255–257, 26, 262–265, 265, 265–267, 269, 27, 27, 27, 270, 274–276, 278–279, 279, 279, 28, 280, 280, 280–288, 288, 288–289, 291, 295, 298–299, 302–309, 31, 310–311, 311, 311–312, 315, 318–321, 324, 324, 324, 324, 324, 324, 324, 324, 324–339, 34, 340–345, 348–349, 35, 351, 355–359, 36, 360–363, 365, 368, 368, 368–369, 373–377, 380, 380, 380–381, 384, 386, 41, 43, 45, 47, 50, 50, 50–52, 52, 52–53, 56, 59, 63, 65–67, 67, 67, 70, 70, 70–71, 74–80, 80, 80–81, 84, 90–91, 94, 96, 96, 96–98
connection.c	77.12%	51.11%	100%	86.87%	10, 10, 104, 11, 110, 123–124, 137–138, 14, 14, 144, 146–147, 149, 16–17, 21–22, 22, 22–23, 25, 27–28, 34, 48, 48, 48–49, 55, 60–62, 97
enclaved.c	0%	0%	0%	0%	101, 104, 108–109, 113, 115–118, 122–126, 128, 132–134, 136, 138, 14, 141–145, 15–17, 19, 19, 19–20, 22, 22, 22, 22, 22–23, 23, 23–25, 28, 28, 28–30, 33–34, 42–45, 45, 45, 47, 47, 47–50, 52, 54–55, 55, 55–56, 61, 63–64, 66, 68, 68, 68, 70–72, 72, 72, 75, 77, 79, 89–90, 90, 90–91, 91, 91–92, 94–96, 96, 96–98, 98, 98–99
event.c	100%	100%	100%	100%
federated.c	6.57%	3.26%	10%	8.02%	10, 100–101, 103, 103, 103–104, 106, 106, 106, 106, 106–107, 107, 107–109, 11, 112, 112, 112–114, 117–118, 121–123, 123, 123–124, 124, 124–126, 128, 13, 131, 133, 135–139, 14, 140, 144–146, 148, 148, 148, 15, 150–153, 155, 157, 157, 157–158, 16, 160–161, 164–165, 172–174, 174, 174–175, 177, 179, 179, 179, 18, 18, 18, 180–182, 182, 182, 182, 182, 182, 182–189, 189, 189, 19, 19, 19, 190, 192–199, 201–204, 208, 21, 21, 21, 211, 211, 211–213, 215, 217, 219, 22, 220–221, 221, 221, 221, 221–228, 23, 23, 23, 230–233, 233, 233–234, 236, 239, 24, 240–242, 244, 25, 250–252, 265–266, 266, 266–267, 267, 267–268, 268, 268–269, 269, 269–270, 270, 270–271, 271, 271–272, 272, 272, 274, 274, 274–275, 275, 275–276, 276, 276–277, 277, 277–278, 278, 278, 280, 29–30, 34–39, 41, 43, 43, 43–44, 44, 44–45, 45, 45–46, 46, 46, 48, 50–53, 55, 55, 55–58, 58, 58–59, 61, 63–64, 64, 64–65, 69, 72–73, 73, 73–74, 76–78, 80, 84, 86, 88–90, 93, 95–99
logging.c	88.52%	83.33%	100%	89.36%	25, 38–40, 47, 60–61
network_channel.c	69.23%	62.50%	100%	70.59%	40, 40, 40, 45–48, 57
physical_clock.c	89.61%	83.33%	71.43%	93.10%	26, 41, 57–58, 81, 83
port.c	75%	42.86%	100%	91.67%	10, 10, 10, 15, 15, 15–16, 22, 26, 31, 31–33, 33, 33–34, 45, 45, 45–46
queues.c	85.33%	77.27%	100%	87.76%	100–106, 132, 134–135, 138–141, 144–145, 163, 168, 171, 179, 34, 34, 38–44, 69–70,

[erlingrj]

I realized there is a couple of things missing:

• Support communication between a deeply nested enclave and the "main" environment higher up. Currently only enclave -> enclave support
• Figure out keep_alive, has_async_events for federated, enclaved and normal. We might need to rename these.
• Throw error if we try to register a reaction triggered directly by an enclave port.
• Throw error if we put a trigger or a reaction in enclaved or federated reactor.
• Figure out how to properly do shutdown by starvation. This can be done when all inputs coming from other enclaves have last_known_tag of FOREVER