Skip to content

Move shareable authz utilities into Tiled#1008

Merged
danielballan merged 37 commits intobluesky:mainfrom
nmaytan:upstreaming_authz
Aug 27, 2025
Merged

Move shareable authz utilities into Tiled#1008
danielballan merged 37 commits intobluesky:mainfrom
nmaytan:upstreaming_authz

Conversation

@nmaytan
Copy link
Contributor

@nmaytan nmaytan commented Jul 10, 2025

  • Code pertaining to access control has been moved into an access_control subdirectory
  • The access tags compiler and db schema are made available in Tiled
  • Unit test coverage has been added for the new access policy and access control features
  • API keys can now be limited to access for only specific tags
  • Some improvements/bug fixes

This PR should mean we can close #951

Checklist

  • Add a Changelog entry
  • Add the ticket number which this PR closes to the comment section

@nmaytan nmaytan requested a review from danielballan July 11, 2025 20:19
@nmaytan nmaytan force-pushed the upstreaming_authz branch from 5c6657a to 0fae734 Compare July 11, 2025 21:25
@nmaytan nmaytan force-pushed the upstreaming_authz branch from b3e9cd5 to ac6f3f1 Compare July 25, 2025 21:06
Comment on lines 44 to +51
"database": {
"uri": "sqlite://", # in-memory
"uri": "sqlite:///file:authn_mem?mode=memory&cache=shared&uri=true", # in-memory
Copy link
Contributor Author

@nmaytan nmaytan Jul 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supporting named in-memory SQLite authentication databases is unexpectedly important for the case of creating multiple concurrently logged-in contexts in one process (i.e. for tests).

Tiled loads its configuration as a singleton. If multiple "apps" are created, the singleton is overridden each time. When a context is created for that app, it will use the current value of the singleton. So, if an app + context is created, it will initialize a config singleton (and an authN database according to that singleton). If a second app + context is created, the same thing happens again.

This means that if you gave each app + context a unique authN database URI, the last app + context to be created will have its corresponding URI in the config singleton. This is a problem because if you create contextA -> contextB but then go back to contextA, it will now be given contextB's singleton (which means contextB's database) and, for authentication specifically, contextA will be put in a weird state.

Unfortunately, this exact problem occurs if an unnamed in-memory database is used, because unnamed in-memory SQLite databases exist only for the connection that creates them. Even though the singleton settings are "identical" in this case, each app + context still has a unique database, and contextA still ends up pointing at contextB's database.

To resolve this issue, a named in-memory shareable database can be used. As long as the URI (db name) is identical each time, the config singleton will retain identical values after each recreation for the subsequent context(s). And because this database is now named in memory (and marked shareable), the same database will be used for both contexts - i.e. it will be valid for both of them to use.

@danielballan danielballan added this to the v0.1.0 release milestone Aug 5, 2025
@nmaytan nmaytan mentioned this pull request Aug 5, 2025
2 tasks
@nmaytan nmaytan force-pushed the upstreaming_authz branch 2 times, most recently from 90e9925 to 678b37d Compare August 5, 2025 23:45
@nmaytan nmaytan force-pushed the upstreaming_authz branch 2 times, most recently from 060a65a to 5562bff Compare August 14, 2025 23:47
@nmaytan nmaytan force-pushed the upstreaming_authz branch 2 times, most recently from 1fa2b57 to 2badb78 Compare August 22, 2025 00:45
@nmaytan nmaytan marked this pull request as ready for review August 22, 2025 01:20
Tags can also inherit the ACLs of other tags, using the `auto_tags` field. There is also a
`public` tag which is a special tag used to mark data as public (all users can read).

Lastly, only "owners" of a tag can apply that tag to a node. Tag owners are defined in
Copy link
Contributor

@genematx genematx Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking, is it possible to declare tag owners in the same section where we declare the main tag definitions? Something like:

tags:
  alice_tag:
    users:
      - name: "alice"
        role: "facility_admin"
        owner: true
      - name: "chris"
        scopes: ["read:data", "read:metadata"]
        owner: false   # default, doesn't need to be set
  chris_tag:
    users:
      - name: "alice"    # here alice is not the owner...
        role: "facility_admin"
      - name: "chris"
        role: "facility_admin"
        owner: true     # ...but chris is
  biologists_tag:
    users:
      - name: "alice"
        role: "facility_admin"
    groups:
      - name: "biologists"
        scopes: ["read:data", "read:metadata"]
        owner: true      # anyone from biologists can apply the tag

i think this would simplify configs and make them more readable. I keep finding myself to having scroll back and forth in the file to check who's the owner of which tag.
Are there any conceptual limitations to this design (e.g. an owner may not be a user of a tag)?

Copy link
Contributor

@genematx genematx Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's not possible for some reason, maybe adding another key-value pair "owners":{"users":[...], "groups":[...]} under each tag definition could be sensible (essentially merging the two dictionaries together). Right now we have "tags" which contains tags, and then, "tag_owners", which also contains tags (but from a different angle).
When I see a dictionary/list called "tag_owners", I'd expect to find user/group-names there with corresponding tag-names under each of them, not the other way around.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question - I agree there is a small cost in ergonomics for having them defined in separate tables. This consideration has come up in design discussions, and the reason we decided to separate these is that it's plausible that these tables may each be subject to different permissions. i.e. different roles in the organization may have rights to modify who can apply a tag but not the permissions that tag confers, and vice versa.

This is something we can discuss further, but probably deserves a separate conversation/PR.

The structure of the table just reflects that tags are the focus item, but this direction (vs the inverse) also helps with deduplication & makes it easier to understand in totality who owns the tag.

@nmaytan nmaytan force-pushed the upstreaming_authz branch from 8cdefb0 to 471dc6a Compare August 26, 2025 20:26
@danielballan danielballan merged commit 277ec5a into bluesky:main Aug 27, 2025
11 checks passed
ZohebShaikh pushed a commit that referenced this pull request Feb 21, 2026
* Move shareable authz utilities into Tiled

* Move access control code into subdirectory

* Update changelog

* Move scopes into access_control, adjust imports

* In-memory catalogs should also accept a top-level access blob

* Tag compiler can accept dict for config, parser can connect to in-memory sqlite db

* WIP removing SimpleAcccessPolicy, refactor AP unit tests

* Add other-user access attempt to user-owned node test

* Allow in-memory catalogs to be shared in-process

* Use separate apps/clients to mimic multiuser auth in tests

* Enable sqlite in-memory db for authN database

* Enable unique uri for TestClient to separate token caches

* Use unique uri for test contexts and in-mem authn db

* Correct base_uri when fed to TestClient

* Add anonymous access to TBAP

* Improve AP tests; add anon, admin, and empty blob tests

* Change tags AP and queries for SpecialUsers removal

* Add ability to restrict API keys to specific access tags

Also add mechanism to set invalid tag names

* Add missing args, default access_tags in apikey_params, minor bugfixes

* Make AccessTagsParser async

* Add authZ tests for API keys, service principals

* Fixes for access control on export

* Cover MapAdapter's lack of access control in `filter_for_access`, by
  adding an `allowed_scopes` check when transitioning MapAdapter->CatalogAdapter
* Cleanup `walk` and fix exception when export is empty for a user
  (MapAdapter does not have items_range)

* Add unit test on node export access control

* Add access test on data nested in container

* Update changelog

* Add access test for modify_node & metadata updates

* Remove redundant catalog access control test

* toy_authentication uses TagBasedAccessPolicy

Added helpers that setup the access tags and catalog databases for
use by the `toy_authentication` example server.

* Update docs that reference toy_authentication

* Add tests on tag compiler and resultant db

* Fix formatting/linting

* Fix args in zarr routes with API key access tags

* Correct the tag compiler tests

* Make AccessTagsParser URI config more user friendly

* Update changelog for new release

* Minor cleanups re: review

* Add authN database migration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rework AccessControlPolicy

3 participants