declarative git-sync manifest

[philipmassouh]

I deployed marimohub ~2wks ago and our research team has been using it and really enjoying it. the remaining gap for us is around credential and provisioning ergonomics for git sync. afaik, every git synced notebook requires a manual create call plus its own one time token copied into ci secrets and its own push step.

id propose defining a marimohub.yml committed to the repo (target proj + branch + notebook path). this way we can have a single repo-scoped sync token authorizing both provisioning and content push. ideally, adding a notebook becomes "add it to the YAML and merge." this deliberately keeps marimohubs push only architecture intact.

I looked into building this on my end with the existing endpoints and found that it's possible but

1. PAT only mintable from browser and unscoped/user-bound
2. POST /projects/{pid}/notebooks/git doesn't accept the idempotency key
3. an external automation won't be able to safely tell which notebooks it owns (and can delete).
4. sync tokens are shown once and stored hash-only, so my reconciler would have to capture the token and write it into the CI secret store itself

would you be open to this?

[0x0f0f0f]

100% agree! Same use case here, but I would avoid defining marimohub.yml as we could just have a Github action in each repo (picking up all or some of the notebooks via a path regex) or a Github App integration that was introduced in #177

[philipmassouh]

100% agree! Same use case here, but I would avoid defining marimohub.yml as we could just have a Github action in each repo (picking up all or some of the notebooks via a path regex) or a Github App integration that was introduced in #177

hello! i wrote this ticket a few days ago and didn't notice yours, would've replied there.

I specifically want a push only design. first, hub-side fetching is deferred in the RFC. second, the push approach keeps the hub free of repo creds and would work with any github alternatives. finally, the push approach makes it far easier for me to gate/check what my users are pushing and allows them to not push notebooks until they are ready.

also, notebooks arent bare files. there are root paths and entry notebooks, local imports, and uv sync. this would be too much for path regex to handle.

either way, your architecture needs the same pieces of listed (repo token, idempotency, ownership tracking, etc).

perhaps your needs could be met with something like:

projects:
  - project: proj-abc123
    discover:
      - glob: "notebooks/**/*.py"
        root: dirname

i.e. the server expands the glob against the pushed archive at apply time; same repo token, ownership tracking, and prune semantics, but zero-touch for your use case. we would also maintain the explicit notebook definitions for my needs

[0x0f0f0f]

You should check out #137

I think the easiest solution is to add a push token as a Github secret, and then use the github actions with the CLI installed (@mscolnick are you going to publish the crate?)

That's how I would (and probably will) do this!

[mscolnick]

Thanks for all the feedback and ideas. We can support these various use cases.

We will support both a push and pull method. Push is needed 1) to avoid GH App creds in marimohub for people who care, and 2) to work with other git providers as a break glass. Pull is probably nicer long-term with GH; I just haven't built out all the features.

@philipmassouh what is wrong with a user-bound PAT? Would it be ok to just put that in the GH Action secrets? Can you define your "repo token"? We can improve the endpoints (or add CLI) with a custom github action to bulk upload. I like your syntax e.g

use: marimo-team/marimohub/actions/upload
with:
    project: proj-abc123
    token: {{ secrets.MARIMOHUB_PAT }}
    discover:
      - glob: "notebooks/**/*.py"
         root: dirname

---

Separately, I plan to add 2 more related features for the GH app only:

1. "Pull-mode" for git sources - server-side clone at session provision (GitHub App only)
2. Manual "Sync now" - server-initiated pull sync for git sources (GitHub App only)

[philipmassouh]

Thanks for all the feedback and ideas. We can support these various use cases.

We will support both a push and pull method. Push is needed 1) to avoid GH App creds in marimohub for people who care, and 2) to work with other git providers as a break glass. Pull is probably nicer long-term with GH; I just haven't built out all the features.

@philipmassouh what is wrong with a user-bound PAT? Would it be ok to just put that in the GH Action secrets? Can you define your "repo token"? We can improve the endpoints (or add CLI) with a custom github action to bulk upload. I like your syntax e.g

use: marimo-team/marimohub/actions/upload
with:
project: proj-abc123
token: {{ secrets.MARIMOHUB_PAT }}
discover:
- glob: "notebooks/**/*.py"
root: dirname
Separately, I plan to add 2 more related features for the GH app only:

1. "Pull-mode" for git sources - server-side clone at session provision (GitHub App only)
2. Manual "Sync now" - server-initiated pull sync for git sources (GitHub App only)

I agree with your summary of push vs. pull and that pull is a nicer workflow for most of your users.

the main nuisance w/ PAT is that the its personal. If i understand correctly, it doesn't encode permissions in the token, it just resolves to my user. therefore if I leave the company or another admin changes my permissions, sync breaks. to get around this I would have to make the PAT using a service account. we use tailscale so I think I would have to sign out of my personal and into the service acc to mint the token. This doesnt solve permission drift risk from manual edits.