A way to know that all OnStart hooks have completed

[Southclaws]

This may be a bit of a hack due to my usage but it felt useful enough to bring up (maybe a way to do this exists already)

I've been looking for a way to execute some code after every OnStart hook has finished, or at least after a specific hook I know has run to completion. I thought that hooks were executed in order of dependency, so if A depends on B, then B's lifecycle hooks will run first then A's - but I'm not sure about this assumption, it doesn't appear to be the case possibly due to parallelism?

Describe the solution you'd like

I'm not entirely sure, an additional hook doesn't feel like the right solution but perhaps a channel on App similar to Done()

Describe alternatives you've considered

I could create my own signal channel on the hooks I'm interested in, which would work for my use-case as it's only a couple.

Is this a breaking change?
Not as far as I can see, it would merely be exposing something I assume fx is already aware of - but maybe it isn't if hooks are paralellised. Though I assumed it would be aware given the error handling and discussion in #1059

Additional context

This came about due to how I'm using fx in integration tests, and the way our app initialises external connections (database, message queue) in start hooks. The database is connected to inside a Provide() but the schema is migrated inside a start hook. This means tests must always run inside a start hook too, otherwise the database is empty. (example here) which works fine, as I assume the hooks run in order of dependency. Though I ran into an issue while writing tests involving my message queue, which operates in a similar way: connection is set up in a Provide() and then queues are prepared during start hooks, but even though the pubsub provider is a very shallow dependency and appears high up in the dependency tree of tests, its start hook runs after the test's start hook so the tests fail due to there being no queues available. I think this is due to the pubsub start hook running after the test itself and I couldn't really figure out why. (Perhaps my issue lies in how I've set it up and this issue is just a wild goose chase in the wrong direction)

Either way, I do feel like a "Run this code after all start hooks have completed" would be a useful addition, if not already available. Thanks!

[abhinav]

Hello!

I thought that hooks were executed in order of dependency, so if A depends on B, then B's lifecycle hooks will run first then A's - but I'm not sure about this assumption, it doesn't appear to be the case possibly due to parallelism?

That assumption is correct. There's no in hook execution. The hooks may spawn goroutines, but the hooks themselves run serially. General advice here is that the blocking part of the OnStart do any short-lived setup (open a connection, start a listener, etc.), and then spawn a goroutine to do any long-lived background work (sending requests on the connection, serving HTTP requests, etc.).

Hooks are executed in dependency order:
OnStart hooks added from fx.Provide(A) will run before OnStart hooks added in fx.Provide(B) if B depends on A. OnStop hooks run in the reverse order.
Hook ordering is undefined if A and B do not have a dependency relation.

execute some code after every OnStart hook has finished

I haven't thought this through fully, but does a hook like that introduce a "What about level N+1?" question?
If there's, say, an "OnStarted" hook, and a bunch of fx.Provides contribute to that, will there eventually be need for running code after all of those hooks have also finished running?

One way you to go about this is without an Fx-level feature: don't use Run and call Start and Stop manually:

app := fx.New(...)

if err := app.Start(ctx); err != nil {
   ...
}

codeThatShouldRunAfterAllHooks()

<-app.Wait()

if err := app.Stop(ctx); err != nil {
  ...
}

If the codeThatShouldRunAfterAllHooks needs access to anything from the Fx container, you can leverage fx.Populate to request those values in fx.New.

Separately, regarding this:

the pubsub provider is a very shallow dependency and appears high up in the dependency tree of tests, its start hook runs after the test's start hook so the tests fail due to there being no queues available. I think this is due to the pubsub start hook running after the test itself and I couldn't really figure out why. (Perhaps my issue lies in how I've set it up and this issue is just a wild goose chase in the wrong direction)

I haven't looked at the code, so just guessing here:
Are you perhaps scheduling the OnStart for queue setup in an fx.Invoke separate from the fx.Provide that sets up the pubsub client used by the test? If so, there might not be a dependency relation between the two.

[m13t]

I've just stumbled across this as it seems a little similar to something I've run in to which I believe is probably me misunderstanding the ordering of hooks.

I have an application and there are a bunch of data source abstraction interfaces that get provided to FX. These dependencies are created in order as expected but each of the data source abstractions calls an initialisation method in a start hook. The abstractions themselves are not usable until the initialisation process has finished, as it sets some pointer fields on the underlying type. The whole initialise method is synchronous and returns an error on failure, so naturally FX will crash out if there's an error.

The problem I am having is that my single Invoke fx option assembles all the dependencies and then runs the main application logic. However, I was under the impression that Invoke options were called last, including after all start hooks had run to completion. I am guessing this is not the case?

I totally understand that constructors need to be run before hooks otherwise no dependency resolution could occur. I could of course add some logic in to various dependencies to indicate they were 'initialised' or some kind of WaitForInitialised blocking method, but that feels a bit messy. The other option would be move all the initialisation logic into the constructor but then I'd have to provide a context to FX as a value. The start hooks play nice with the initialisation as it provides the start context for you which doesn't then live forever as a weird global context.

I'm keen to understand how these kinds of situations are tackled at Uber as despite it clearly being used heavily there, there's no real good examples of a well defined FX application that is public. Most examples are fairly trivial and very much along the 'hello world' theme rather than more concrete realistic examples.