DOCS-3699, DOCS-2540: Add module lifecycle and dependencies guidance #4156

JessamyT · 2025-03-27T00:46:45Z

Discovery service will be further fleshed out separately

Staging:
https://deploy-preview-4156--viam-docs.netlify.app/operate/get-started/other-hardware/#how-and-where-do-modules-run and https://deploy-preview-4156--viam-docs.netlify.app/operate/get-started/other-hardware/dependencies/

netlify · 2025-03-27T00:47:03Z

✅ Deploy Preview for viam-docs ready!

Name	Link
🔨 Latest commit	`ae38347`
🔍 Latest deploy log	https://app.netlify.com/sites/viam-docs/deploys/67e707bed486dd000812eaf7
😎 Deploy Preview	https://deploy-preview-4156--viam-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 26 (🔴 down 19 from production) Accessibility: 100 (no change from production) Best Practices: 100 (no change from production) SEO: 100 (no change from production) PWA: 60 (no change from production) View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

michaellee1019 · 2025-03-27T19:31:42Z

docs/operate/get-started/other-hardware/dependencies.md

@@ -191,7 +191,7 @@ If you need to maintain the state of your resource, see [(Optional) Create and e
 {{< tabs >}}
 {{% tab name="Python" %}}

-If you prefer to use explicit dependencies (for example, for an optional dependency), the steps are the same as for implicit dependencies above, except that you do not need to return the dependency from the `validate_config` method and can instead return an empty list:
+If you prefer to use explicit dependencies (for example, for an optional dependency), the steps are the same as for implicit dependencies, except that you do not need to return the dependency from the `validate_config` method and can instead return an empty list:


I think that saying "for example, for an optional dependency" is wrong here. A resource will still wait for all resources declared in depends_on to be available before starting.

We have two mechanisms for declaring dependencies which are essentially the same result. Implicit is more common than explicit

What I mean by "optional" here is that if the user doesn't configure a "camera_name" at all, either in depends_on or in attributes, that the modular resource should still work. But I'm realizing the example I have here doesn't indicate that--I should make this validate_config explicit dependency example show "camera_name" as an optional attribute, and keep it as a required attribute for the implicit dependency example.

Is it true that people should use implicit dependencies for required things and explicit dependencies for optional things?

I guess, is it currently possible for people to have an optional implicit dependency, or not until the upcoming work happens?

No way to have an optional explicit dependency. Yeah you could make an implicit dependency optional. Its a bit annoying right now:

See in validate that its not defined and don't return it as a dependency.

In your implementation you check to make sure the dependency is not null and gracefully skip using it.

To illuminate other reviewers: Based on offline sync, we were using "optional" in different ways--If a user does put something in depends_on, the resource won't currently start if the dependency doesn't start. What I was meaning with "optional" is it's optional for the user to even configure it at all. But sounds like you can do that with either implicit or explicit deps--it's all about how you implement validate.

Based on offline convo I will remove explicit dependencies since they just confuse users and aren't actually necessary except in cases of backwards compatibility.

JessamyT · 2025-03-27T20:19:06Z

docs/operate/get-started/other-hardware/dependencies.md

+{{< /tabs >}}
+
+{{% hiddencontent %}}
+There is not currently an SDK method to access configuration attributes of dependencies in Python or Go, but in Python it is possible to use `get_robot_part` to return information including the whole configuration of a machine part, and then access the configuration attributes of the dependency from there.


Maybe this isn't right--looking into this more

JessamyT · 2025-03-28T01:56:46Z

@michaellee1019 I changed it to optional vs required but LMK if you think that is still confusing or overkill and it'd be better to just have one heading with "use dependencies" in general, with the required deps examples, and omit the optional examples.

docs/operate/get-started/other-hardware/_index.md

cheukt

did a quick pass, I'll be out next week so no need for my approval - if we want an opinion from netcode we can add @benjirewis

docs/operate/get-started/other-hardware/_index.md

cheukt · 2025-03-28T15:22:36Z

docs/operate/get-started/other-hardware/dependencies.md

+{{< /tabs >}}
+
+{{% hiddencontent %}}
+There is not currently an SDK method to directly access configuration attributes of dependencies in Python or Go, but in Python it is possible to use `get_robot_part` to return information including the whole configuration of a machine part, and then access the configuration attributes of the dependency from there.


what does this mean? I think we would want to discourage users from looking at configs for other resources

We got a chatbot question about this specifically (from Sean Yu) and the bot answer was incorrect so was hoping to fix it. This hidden content short code means it’s not visible on the page, only to the chatbot if someone specifically asks. But if we don’t want people to do this at all should I just change this to say “you can’t, period”?

ok, I guess in that case this is fine - we should definitely discourage it tho

benjirewis

This is awesome! You guys do a great job of adding stuff around this somewhat opaque logic. Some comments and I can help further while Cheuk is on PTO.

benjirewis · 2025-03-28T18:45:41Z

docs/operate/get-started/other-hardware/_index.md

+1. `viam-server` starts, and if it is connected to the internet, it checks for configuration updates.
+1. `viam-server` starts any configured modules.
+1. When a module initializes, it registers its model or models and associated [APIs](/dev/reference/apis/) with `viam-server`, making the models available for use.
+1. For each modular resource configured on the machine, `viam-server` uses the resource's `validate` function and the `depends_on` field in the resource configuration to determine the dependencies of the resource.


Just a note that an upcoming NetCode change may further complicate this by adding a new type of "optional" dependency https://viam.atlassian.net/browse/RSDK-10135.

Yeah...I figured since the work hasn't started on implementing it it makes sense to try and get this merged with what is true now. Any idea on timeline for this change though?

docs/operate/get-started/other-hardware/_index.md

benjirewis · 2025-03-28T18:55:14Z

docs/operate/get-started/other-hardware/_index.md

+1. If at any point the user changes the configuration of the machine, `viam-server` reconfigures the affected resources within 15 seconds.
+1. If a resource crashes, `viam-server` attempts to rebuild it.
+1. When `viam-server` shuts down, it attempts to stop all running modules.
+   If a module fails to shut down within 90 seconds, `viam-server` will forcefully kill it after 15 seconds.


I believe it's actually:

SIGTERM to module process

wait 10s

SIGTERM to module process group

wait 20s

SIGKILL to module process group

So the timings you have there are inaccurate.

The SIGTERMs are on a different timer, they are only triggered when we actually get to the clean module shutdown phase.
SIGKILLs are triggered after 90s from when the restart/shutdown is started, and the SIGTERMS can happen at any time between the shutdown is started and 90s

Ah I forgot about those SIGKILLs thanks @cheukt. So, yeah, slightly complicated "killing" logic (because we can SIGKILL a module due to it taking 90s since viam-server started shutdown or it taking 30s since viam-server tried to start shutting down a single module, right?)

ya, you're right about that - I forgot about the SIGKILL after 30s from the viam-server trying to shutdown a module.

So, this?

Suggested change

If a module fails to shut down within 90 seconds, `viam-server` will forcefully kill it after 15 seconds.

If `viam-server` attempts to shut down an individual module (for example due to a user disabling a module) and the module does not shut down within 30 seconds, `viam-server` kills the module.

1. When `viam-server` shuts down, it first shuts down all modules and kills them if they fail to shut down within 90 seconds.

benjirewis · 2025-03-28T18:58:57Z

docs/operate/get-started/other-hardware/dependencies.md

+     if cfg.CameraName == "" {
+       return nil, resource.NewConfigValidationFieldRequiredError(path, "camera_name")
+     }
+     if reflect.TypeOf(cfg.CameraName).Kind() != reflect.String {


I don't think you need this reflection check. cfg.CameraName will definitely be a string at this point.

Does it get checked by viam-server automatically because CameraName string json:"camera_name" specifies type?

Basically yep! It's not technically viam-server that does the conversion from proto to a golang string, but the Golang module library code does it and will error if the thing specified in JSON is not actually a string/cfg.CameraName will just "".

Ah okay ty!! Wait it'll error or it'll assign it an empty string or both?

docs/operate/get-started/other-hardware/dependencies.md

viambot added the safe to build This pull request is marked safe to build from a trusted zone label Mar 27, 2025

JessamyT force-pushed the 3699modules branch from b0062e1 to 1a5e569 Compare March 27, 2025 00:47

JessamyT requested review from michaellee1019 and cheukt March 27, 2025 00:48

michaellee1019 reviewed Mar 27, 2025

View reviewed changes

JessamyT commented Mar 27, 2025

View reviewed changes

JessamyT added 9 commits March 27, 2025 18:54

DOCS-3699: Add module dependencies guidance

e495137

Add more content including Python examples and hidden content

167ded9

Add implicit Go and add explicit

60b6f1d

Fix viam-server, move lifecycle to expander, etc

af654bc

lint

41c21ec

Fix nits/linter

ff1f74c

Change explicit validate to make attr optional

5899106

lint

06fd06c

Deprecate explicit deps and change wording to optional or required

bb8a644

JessamyT force-pushed the 3699modules branch from 5b0f92e to bb8a644 Compare March 28, 2025 01:54

JessamyT requested a review from michaellee1019 March 28, 2025 01:54

JessamyT commented Mar 28, 2025

View reviewed changes

docs/operate/get-started/other-hardware/_index.md Outdated Show resolved Hide resolved

cheukt reviewed Mar 28, 2025

View reviewed changes

benjirewis reviewed Mar 28, 2025

View reviewed changes

JessamyT and others added 2 commits March 28, 2025 13:18

Apply suggestions from code review

7863bb2

Remove unnecessary go camera_name type check

ae38347

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOCS-3699, DOCS-2540: Add module lifecycle and dependencies guidance #4156

DOCS-3699, DOCS-2540: Add module lifecycle and dependencies guidance #4156

JessamyT commented Mar 27, 2025 •

edited

Loading

netlify bot commented Mar 27, 2025 •

edited

Loading

michaellee1019 Mar 27, 2025

michaellee1019 Mar 27, 2025

JessamyT Mar 27, 2025

JessamyT Mar 27, 2025

michaellee1019 Mar 27, 2025

JessamyT Mar 27, 2025

JessamyT Mar 27, 2025

JessamyT commented Mar 28, 2025

cheukt left a comment

cheukt Mar 28, 2025

JessamyT Mar 28, 2025

cheukt Mar 28, 2025

benjirewis left a comment

benjirewis Mar 28, 2025

JessamyT Mar 28, 2025

benjirewis Mar 28, 2025

cheukt Mar 28, 2025

benjirewis Mar 28, 2025

cheukt Mar 28, 2025

JessamyT Mar 28, 2025 •

edited

Loading

benjirewis Mar 28, 2025

JessamyT Mar 28, 2025

benjirewis Mar 28, 2025

JessamyT Mar 28, 2025

	If a module fails to shut down within 90 seconds, `viam-server` will forcefully kill it after 15 seconds.
	If `viam-server` attempts to shut down an individual module (for example due to a user disabling a module) and the module does not shut down within 30 seconds, `viam-server` kills the module.
	1. When `viam-server` shuts down, it first shuts down all modules and kills them if they fail to shut down within 90 seconds.

DOCS-3699, DOCS-2540: Add module lifecycle and dependencies guidance #4156

Are you sure you want to change the base?

DOCS-3699, DOCS-2540: Add module lifecycle and dependencies guidance #4156

Conversation

JessamyT commented Mar 27, 2025 • edited Loading

netlify bot commented Mar 27, 2025 • edited Loading

✅ Deploy Preview for viam-docs ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JessamyT commented Mar 28, 2025

cheukt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjirewis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JessamyT Mar 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JessamyT commented Mar 27, 2025 •

edited

Loading

netlify bot commented Mar 27, 2025 •

edited

Loading

JessamyT Mar 28, 2025 •

edited

Loading