How to best design higher-order components

[askvortsov1]

As part of a recent school project on database-driven web applications, I've decided to explore the OCaml web programming ecosystem, with Dream for the backend, Bonsai for the frontend, and a graphql layer in between. I'm pretty familiar with React/Mithril, so I'm comfortable with vdom-based frontends, but I'm having a bit of trouble grasping some Bonsai concepts.

To cut back on code duplication, I want to build a higher-order GraphQL Query Loader component, which would execute a graphql query, and when it gets a result, render a "child" component.

I've figured out that the inner component should probably be constructed with Bonsai.of_module1 or Bonsai.state_machine1, yielding the type:

val component: Q.t Value.t -> Vdom.Node.t Computation.t

That brings me to the query loader component. Ideally, I'd like something along the lines of:

module Loader (Q: Query) = struct
    val component: (Q.t Value.t -> Vdom.Node.t Computation.t) -> Vdom.Node.t Computation.t
end

But this is where I've reached a bit of a dead end. Regardless of what I've tried so far, I keep ending up with:

module Loader (Q: Query) = struct
    val component: (Q.t Value.t -> Vdom.Node.t Computation.t) -> Vdom.Node.t Computation.t Computation.t
end

My current implementation (below) uses the Bonsai.of_module1 pattern, but I've also tried doing this with Bonsai.state_machine, and haven't been able to find a combination of combinators that avoids the nested Computation.t Computation.t. One attempt that came close was having the inner component be of the form:

val component: (Q.t -> Vdom.Node.t) Computation.t

Which avoided the nested Computation.t, but lost the ability to have the Q.t input factored into the computation, so the inner component didn't redraw when the data came in.

Are there any cleaner design patterns I can use for higher-order components like this?

---

Current Query Loader Code:

open! Core
open! Bonsai_web
open Bonsai.Let_syntax

(* A collection of GraphQL modules generated by graphql-ppx *)
module G = Nittany_market_frontend_graphql

module Loader (Q : G.Queries.Query) = struct
  module T = struct
    module Input = struct
      type t = Q.t option Value.t -> Vdom.Node.t Computation.t
    end

    module Model = struct
      type t = { loaded : bool; res : Q.t option }

      (* The query result doesn't matter for diffing purposes so I excluded it *)
      let sexp_of_t t = Sexplib0.Sexp.Atom (Bool.to_string t.loaded)

      let t_of_sexp v =
        { loaded = Bool.of_string (Sexplib0.Sexp.to_string v); res = None }

      let equal a b = Bool.equal a.loaded b.loaded
    end

    module Action = struct
      module SexpableQ = G.Queries.SexpableQuery (Q)

      (* Although funnily enough, then it turned out it does need to be sexpable
         so that the action can be encoded, so I had to make a functor for that anyway :) *)
      type t = Loaded of SexpableQ.t option | Unloaded [@@deriving sexp_of]
    end

    module Result = Vdom.Node

    let apply_action ~inject:_ ~schedule_event:_ _input model
        (_action : Action.t) =
      model

    module Client = G.Client.ForQuery (Q)

    let compute ~inject (input : Input.t) (model : Model.t) =
      if model.loaded then
        (* Ideally I'd like to return Vdom.Node.t here,
        but there doesn't seem to be a clear way to do so. *)
        input (Value.return model.res)
      else
        (* A possibly hacky way to run the graphql query. *)
        let _ = 
        (Lwt.ignore_result
         (Lwt.map
            (fun (_resp, body) ->
              Vdom.Effect.Expert.handle_non_dom_event_exn
                (inject (Action.Loaded body)))
            (Client.query ()))) in
            Vdom.Node.text "Loading"

    let name = Source_code_position.to_string [%here]
  end

  module Action = struct end

  let component =
    Bonsai.of_module1 (module T) ~default_model:{ loaded = false; res = None }
end

[pmwhite]

Here's a function I just wrote that should be helpful:

let component
  : type a view.
    (module Bonsai.Model with type t = a)
    -> a Effect.t Value.t
    -> view_while_loading:view Value.t
    -> (a Value.t -> view Computation.t)
    -> view Computation.t
  =
  fun (module Model) effect ~view_while_loading computation ->
  let%sub data, set_data = Bonsai.state_opt (module Model) in
  let%sub on_activate =
    let%arr effect = effect
    and set_data = set_data in
    let%bind.Effect data_fetched_from_server = effect in
    set_data (Some data_fetched_from_server)
  in
  let%sub () = Bonsai.Edge.lifecycle ~on_activate () in
  match%sub data with
  | None -> return view_while_loading
  | Some data -> computation data
;;

Hopefully the type signature makes the purpose of this function clear enough.

There are a lot of parts to this function, and I will surely gloss over an important detail, so feel free to ask for a clarification on any of it. Here are some points of interest:

• This function abstracts over both the type of the data fetched from the server and also the type of the view. In your code, you code be more concrete and replace a with Q.t and view with Vdom.Node.t.
• The model needs to support the loading period, so we use state_opt to make the model be a option instead of a. The initial model for the state_opt computation is None, in which case we display the loading view instead of the function that we passed in.
• state_opt is ultimately implemented in terms of state_machine0, so it's not like you have to memorize a bunch of primitive bonsai functions. You can think of state_opt as more of a utility function that you could have implemented yourself.
• let%sub desugars to calls to sub, which has the type signature:

val sub : 'a Computation.t -> f:('a Value.t -> 'b Computation.t) -> 'b Computation.t

If you're used to monads, sub looks a lot like bind, but it has the important property that you don't actually get access to the data inside, but rather a name that refers to the data.

• In order to transform data inside a 'a Value.t, you need to use let%arr, which maps over a bunch of values and produces a 'a Computation.t.
• Since Bonsai is an incremental programming library, you often end up with a bunch of snippets of code embedded within let%arr or let%map expressions that only get run whenever one of their transitive dependencies is updated (that's the rough intuition at least). It's a bad idea to rely on a particular order or timing of when these functions get run, so we provide the Bonsai.Edge.lifecycle computation which runs the on_activate effect on the first stabilization which the computation is a part of.
• match%sub provides a way to dynamically switch between two computations, so that only one of them is active at a particular time. It can be useful for implementing tabs, for example, in which you don't need any non-visible tabs to keep getting computation, since they won't be displayed.
• Rather than executing side-effects (like sending an RPC) wherever we want, we package them up into a 'a Effect.t which will get executed at the proper time. Since Jane Street uses Async instead of Lwt, we provide function called Effect.of_deferred_fun, but I think it should be possible to define an analogous function for Lwt without needing to modify Bonsai itself (take a look at the source code for Effect.of_deferred_fun if you want a template for how to make the Lwt analogy).

Note that sub, Bonsai.assoc, and match%sub are the three built-in "higher-order" computations, so if you want to build a higher-order computation, you have to define it in terms of one of these three.

I'm happy to elaborate further if needed.

[askvortsov1]

Thank you very much for the example and detailed explanation! After reading over this and the mli again, I think Bonsai may have finally clicked for me. I was successfully able to adapt this to my use case (and define a Effect_lwt.of_deferred_fun util).

I think the only major piece I'm unsure of is Bonsai.Dynamic_scope. Is the goal to be able to share a mutable, computation-compliant variable between components?

you often end up with a bunch of snippets of code embedded within let%arr or let%map expressions that only get run whenever one of their transitive dependencies is updated

Is let%sub also calculated incrementally, or does that only apply to combinators that unwrap 'a Value.t (either directly or all the way from 'a Computation.t)?

[pmwhite]

I think it makes sense to answer your second question first. let%sub runs immediately when the computation is constructed, since all it is doing is minting a name with which you can refer to the right-hand side by. That said, let%sub does facilitate having incrementality, so it feels wrong to answer your question with "no". In other words, let%sub doesn't run incrementally (actually, it only runs once), but the name it puts in scope represents a value that is computed incrementally.

Your question inspired me to write some documentation that is not yet on github, so I'll quote it here. It's not a direct answer, but you may find it useful.

---

When you work with Bonsai, it can be a bit tricky to know when different parts of your code run. Here is a list of several different "*-time"s that you should be aware of.

• Design-time happens before and while you write your code. This is the stage at which you get the most performance wins.
• Compile-time happens before your program even runs. This is when the type-checker runs and also when you can enforce static invariants about your code.
• Run-time refers to the period that your program is running. All Bonsai code runs during runtime.

The above three categories are the typical things you have to think about with any OCaml program. Bonsai further subdivides runtime:

• Construction-time is when the main app function runs and produces a Vdom.Node.t Computation.t; the view is not yet rendered on the screen, and none of the incremental graph has been computed, but the computation has been constructed. Code executed at construction time is only executed once when the program starts, and never after.
• Eval-time is when Bonsai's eval function runs. That function converts a 'a Computation.t into an 'a Incr.t. In other words, it converts the Bonsai "syntax tree" data structure into the Incremental "assembly language" graph. Eval-time code also runs once at the beginning of the app, but it also runs every time the active branch of a match%sub expression changes. Bonsai programmers usually don't have to think about eval-time, since the eval function doesn't execute any user code.
• Stabilize-time is what happens when the Incremental graph gets "stabilized", which is the process of re-running any of the nodes that depend on inputs which changed. Stabilization happens 60 frames per second (aspirationally), so it is the main place where you should think about writing fast code. A lot of the time, however, the point is not really to write fast code, but to prevent your code from running at all by structuring the graph in as incremental a manner as possible. This structuring happens at design-time. You should avoid doing side-effects at Stabilize-time because it's difficult to predict when, if, and in what order incremental nodes will fire.
• Frame-boundary-time is the stuff that happens between frames. This includes the execution of any 'a Effect.ts that have been scheduled. Side-effects should happen at this time, wrapped within an Effect.of_sync_fun or Effect.of_deferred_fun, and they should probably be triggered by a discrete event, such as a button click, or via a Bonsai.Edge.on_change.

In summary, below is some code with print statements representing the various times I just mentioned. You may notice that we aren't printing anything at eval-time; this is because evaluation doesn't execute any user code and is therefore difficult to witness.

open! Core
open! Bonsai_web

let component () =
  let%sub state, set_state = Bonsai.state (module Bool) ~default_model:true in
  print_endline "construction-time";
  let%sub view =
    let%arr state = state in
    print_endline "stabilize-time";
    Vdom.Node.text (Bool.to_string state)
  in
  let%sub on_click_effect =
    let%arr set_state = set_state in
    print_endline "stabilize-time; here we're merely computing the effect, not running it yet";
    let%bind.Effect () = 
      Effect.of_sync_fun print_endline "frame-boundary-time; this is when the effect actually runs"
    in
    set_state false
  in
  let%arr view = view
  and on_click_effect = on_click_effect in
  Vdom.Node.div
    ~attr:(Vdom.Attr.on_click (fun _ -> on_click_effect))
    [ view ]
;;

let (_ : _ Start.Handle.t) =
  let app = component () in
  print_endline "finished construction-time; starting eval-time";
  Start.start Start.Result_spec.just_the_view ~bind_to_element_with_id:"app" app
;;

---

Regarding Dynamic_scope, the goal is to allow you to pass values around without having to thread them through many different functions. For example, often apps need to be parameterized over several different RPCs, and you want to be able to provide real RPC calls in the browser, and mocked versions of the RPCs in tests. I've been working recently on a module that uses Dynamic_scope to provide an abstract connection to the graph; any computation can pull out the connection and invoke an RPC with it, even if the connection wasn't passed as a parameter to the computation.

Dynamic scope is a programming language concept that concerns when you can refer to variables. Most languages default to lexical because it is much easier to think about; lexical scope forces you to explicitly state your inputs, instead of relying implicitly on the environment of the caller. The same trade-off applies with the Dynamic_scope module, so I would be wary of using it a lot.

[askvortsov1]

Sorry for the late reply, ever since finishing my class project I've been swamped with finals and haven't had the chance to revisit Bonsai in depth. Thank you so much for your explanations; this has helped a ton in better understanding Bonsai, its goals, and its design philosophy.

In other words, let%sub doesn't run incrementally (actually, it only runs once), but the name it puts in scope represents a value that is computed incrementally.

Gotcha, that and the design vs compile vs runtime explanation make sense.

Regarding Dynamic_scope, the goal is to allow you to pass values around without having to thread them through many different functions.

I remember covering the concept in my intro to PL class last year, but I think I'm still struggling a bit to understand how it would be used since I haven't seen "real life" examples of it in Bonsai.

I might be totally off here, but is the intent similar to React's Context feature, adding support for shared "semi-global" state among a subgraph of computation, as well as being able to inject context in one place without having it be explicit input for all computations in the subgraph? If so:

1. Since (at least based on the comments I've read) the same instance of 'a Dynamic_scope.t needs to be used in both the set/lookup calls, should that instance be defined globally/statically in a scope available to all components in the tree, not as an instance in each component's definition?
2. Is the name argument in Dynamic_scope.create just for debugging/logging purposes?
3. Is the set argument to derived intended to modify both the derived and original value; that is, is derived intended to be a 2-way alias for some portion of another variable?
4. Is the goal of the set's inside argument to control what's returned by the set call? If so, wouldn't it be better for it to be a 'a t -> 'r Computation.t, so that the availability/usage of the current Dynamic_scope.t is more explicit? I think the wording of "evaluate a function whose resulting Computation.t has access to the value via the [lookup] function" might be leading me astray here, as set doesn't take any functions as arguments. Either way, if this description is accurate, why not separate getting/setting values? This test case, for example, seems like it would lead to anti-patterns.
5. I'm a bit confused about the goal of revert. Is it so that a Dynamic_scope.set' call can be restricted to a "partial" subgraph of the incremental DAG (ie one that doesn't reach the leaves) as opposed to a "full" subgraph (ie one that contains all nodes reachable from the set' call)? And if so, is the intent there to propogate the set to parent, but not downstream, computations? But then, wouldn't that make Dynamic_scope instances effectively global? Why not make that a boolean flag / a default feature of a set_upstream function?
6. If I'm on the right track so far, I think one of the issues that threw me off at first was the naming: I first thought Dynamic_scope was intended to represent an entire scope/namespace that can grow/shrink dynamically, not just a single dynamically scoped value. The wording "you can store values in it" further let me to think that Dynamic_scope.t represents a dynamic dictionary of variables. It might help to clarify this in the mli docblock.

There's also another pattern I'm trying to figure out, but I'll open a new issue for that as to not deviate from the original topic even further.

Thank you again so much for your help with this!

[TyOverby]

Regarding Dynamic_scope, the goal is to allow you to pass values around without having to thread them through many different functions.

I remember covering the concept in my intro to PL class last year, but I think I'm still struggling a bit to understand how it would be used since I haven't seen "real life" examples of it in Bonsai.

Haha, yeah; there aren't a ton of usages of the API in total if I'm honest haha! Our main motivation was to be able to parameterize large swaths of the component tree with things like styling or theming directives. "is dark mode" would be a good example of something that could be put into a dynamic-var and then read from many components without those components needing to have the value passed into them directly. Not the best example because "is-dark-mode" is likely going to be global to the whole component tree, but technically you could set bits and pieces of it as "dark mode" and others as "light mode" with this API.

I might be totally off here, but is the intent similar to React's Context feature, adding support for shared "semi-global" state among a subgraph of computation, as well as being able to inject context in one place without having it be explicit input for all computations in the subgraph?

Yep! Context is react's implementation of dynamically-scoped variables! Emacs-lisp also (infamously) has dynamically scoped variables by default!

1. Since (at least based on the comments I've read) the same instance of 'a Dynamic_scope.t needs to be used in both the set/lookup calls, should that instance be defined globally/statically in a scope available to all components in the tree, not as an instance in each component's definition?

Correct!

2. Is the name argument in Dynamic_scope.create just for debugging/logging purposes?

Yes!

3. Is the set argument to derived intended to modify both the derived and original value; that is, is derived intended to be a 2-way alias for some portion of another variable?

set on a derived-var will "set" the sub-variable in the super-var (and any other derived-vars), but only inside the scope of the computation passed to set.

4. Is the goal of the set's inside argument to control what's returned by the set call? If so, wouldn't it be better for it to be a 'a t -> 'r Computation.t, so that the availability/usage of the current Dynamic_scope.t is more explicit?

The inside computation is the region in which the effect of the set is visible. Outside of that computation, the value is unaffected.

I think the wording of "evaluate a function whose resulting Computation.t has access to the value via the [lookup] function" might be leading me astray here, as set doesn't take any functions as arguments.

Ah, that documentation is out of date, I'll fix that!

Either way, if this description is accurate, why not separate getting/setting values? This test case, for example, seems like it would lead to anti-patterns.

I'm not sure what you mean by this; setting and getting them are separated into the Bonsai.Dynamic_scope.set and Bonsai.Dynamic_scope.lookup functions.

5. I'm a bit confused about the goal of revert. Is it so that a Dynamic_scope.set' call can be restricted to a "partial" subgraph of the incremental DAG (ie one that doesn't reach the leaves) as opposed to a "full" subgraph (ie one that contains all nodes reachable from the set' call)?

Precisely. This is pretty useful when developing higher-order components, where you may want to override a value for one part of your component, but then revert back to the previous value when evaluating the first-class-component that your user passed in.

And if so, is the intent there to propogate the set to parent, but not downstream, computations? But then, wouldn't that make Dynamic_scope instances effectively global? Why not make that a boolean flag / a default feature of a set_upstream function?

I think I cleared this up above, but just to cover everything: the set function only changes the value inside the provided computation, it doesn't propagate the changes back to any parents.

6. If I'm on the right track so far, I think one of the issues that threw me off at first was the naming: I first thought Dynamic_scope was intended to represent an entire scope/namespace that can grow/shrink dynamically, not just a single dynamically scoped value. The wording "you can store values in it" further let me to think that Dynamic_scope.t represents a dynamic dictionary of variables. It might help to clarify this in the mli docblock.

I'm always looking for ways to improve the docs, so thanks for your feedback! In the meantime, you might be well-served by reading the Racket documentation on "parameters", we basically just ripped off that whole API.

Thank you again so much for your help with this!

Not at all! Best of luck with your project!

[askvortsov1]

Ah that explains a lot, thank you! So would it be correct to say that:

• set evaluates a computation with a new value bound to the applicable Dynamic_scope.t
• set' is a shorthand way of getting a Dynamic_scope.t's current value (with lookup and , evaluating a computation with the new value (ie set), but evaluating some "subcomputation"/component of that computation with the current/old value.

Maybe this is just my unfamiliarity with dynamically scoped syntaxes, but to me, "set" implies a void function where subsequent lookup calls return the new value. Of course, this isn't really possible in an embedded DSL, so I was stuck trying to reconcile how it could work.

Out of curiosity, had you considered an API similar to:

val eval_with : 'r Computation.t -> 'a t -> 'a Value.t -> 'r Computation.t

On a slight tangent, it feels like if using Dynamic_scope for dependency injection, it would be preferable to have "required" dynamically scoped variables, without a fallback. It doesn't seem like this would be possible now without potential runtime exceptions, but do you think this would be possible to implement in Bonsai if/when OCaml gets an effect system?

[TyOverby]

Ah that explains a lot, thank you! So would it be correct to say that:

• set evaluates a computation with a new value bound to the applicable Dynamic_scope.t

yep!

• set' is a shorthand way of getting a Dynamic_scope.t's current value (with lookup and , evaluating a computation with the new value (ie set), but evaluating some "subcomputation"/component of that computation with the current/old value.

that’s pretty close to the implementation, but I think of it as being the same as ‘set’ but with an “undo” operation to revert it back to the previous state.

Maybe this is just my unfamiliarity with dynamically scoped syntaxes, but to me, "set" implies a void function where subsequent lookup calls return the new value. Of course, this isn't really possible in an embedded DSL, so I was stuck trying to reconcile how it could work.

My initial implementation did this, but it was hard to reason about because a subcomponent of a subcomponent could ‘set’ and unintentionally override things in the rest of your program. It was basically like having global mutable variables. The current API is very explicit about the computation that can witness the call to “set”.

Out of curiosity, had you considered an API similar to:

val eval_with : 'r Computation.t -> 'a t -> 'a Value.t -> 'r Computation.t

maybe I’m missing something, but isn’t this basically the api for ‘set’ right now?

On a slight tangent, it feels like if using Dynamic_scope for dependency injection, it would be preferable to have "required" dynamically scoped variables, without a fallback. It doesn't seem like this would be possible now without potential runtime exceptions, but do you think this would be possible to implement in Bonsai if/when OCaml gets an effect system?

hmm, I hadn’t thought about it, but it could be possible and would be pretty cool!

[askvortsov1]

maybe I’m missing something, but isn’t this basically the api for ‘set’ right now?

Structurally it's the same, but since the primary "operation" done by set is creating a computation that wraps an inner computation with a dynamically scoped variable bound to a value, a different name / argument order might make it a bit more reader-friendly.

[askvortsov1]

Hi @TyOverby and @pmwhite! I wanted to thank you again for helping me understand Bonsai and answering my questions. While working on my project, I wrote a brief overview of Bonsai since writing explanations helps me work through complex concepts. After my finals ended and I had a bit more free time, I decided to expand it into a tutorial series on OCaml full-stack web development, based around my project. I really enjoyed learning Bonsai, using it in my project, and writing about it. The core idea is incredibly general and flexible, with a relatively lightweight API. I'm looking forward to seeing where it goes next! Thanks again for your help.

[TyOverby]

@askvortsov1 That is so cool! Thanks for sharing!