Skip to content

Add portable schema GUI, apply extensions, and deletion hardening#599

Closed
christianbur wants to merge 10 commits into
netboxlabs:featurefrom
christianbur:feature/portable-schema-gui
Closed

Add portable schema GUI, apply extensions, and deletion hardening#599
christianbur wants to merge 10 commits into
netboxlabs:featurefrom
christianbur:feature/portable-schema-gui

Conversation

@christianbur

Copy link
Copy Markdown

Summary

Adds the portable schema GUI and apply workflow on top of the existing feature-branch backend: JSON import on the Custom Object Type add page, export tab and list export, choice_sets / objects apply support, and a reference demo document (security_objects.json).

Also includes operational hardening found while testing complex cross-COT schemas: COT deletion guards, bulk-delete reliability after dynamic model regeneration, and recursion guards for circular / polymorphic M2M graphs.

Portable schema import/export makes it straightforward for the community to share, reuse, and adopt Custom Object Type definitions as portable JSON documents.

Portable schema (GUI + apply)

  • Import: JSON tab on Custom Object Types → Add (preview + apply via comparator/executor)
  • Export: per-COT export tab + list action Portable schema download
  • Apply extensions: document-level choice_sets and objects seed data (executor + validation)
  • Documentation: portable-schema.md, rest-api.md, branch-aware apply (DDL in branch schema; COT/field metadata in main)
  • Example: netbox_custom_objects/schema/examples/security_objects.json

Recommended top-level key order when authoring documents by hand:

schema_versionchoice_setstypesobjects

Custom Object Type list & deletion

  • List columns Objects and Referenced by (with bulk_load_list_stats)
  • get_deletion_blockers() — block COT delete while instances exist or other COTs reference this type in schema (AbortRequest in UI/API)
  • Safer COT delete when the dynamic backing table is already missing (drop tables before ContentType cleanup)

Bulk delete & dynamic model stability

Fixes ValueError: Must be "TableNModel" instance and related collector failures after cache_timestamp / model regeneration:

  • Realign inbound/outbound M2M through FKs on all in-process class copies (registry, M2M descriptor, through cache)
  • Polymorphic multiobject through source FK realignment (e.g. security-rb-demo1)
  • CustomObject.delete() uses the canonical registered model class
  • _netbox_private on CustomObject to avoid broken NetBox reverse-M2M changelog traversal
  • Recursion guards during model generation (get_registered_model(), _model_generation_guard in get_models())

Other

  • schedule_reindex_custom_object_type() — batch reindex enqueue on schema apply
  • Tests: schema UI/API/executor, COT list stats, extended deletion scenarios (including security_objects apply smoke tests)

Test plan

  • Apply security_objects.json via JSON tab (preview → apply)
  • Export single COT + list portable schema export; verify slugs match
  • COT list: Objects / Referenced by counts; delete blocked when > 0
  • Bulk-delete security-service, security-service-group, and security-rb-demo1 instances after schema re-apply / cache invalidation
  • COT bulk-delete with dependent types still referenced → blocking error (not silent schema break)
  • python manage.py test netbox_custom_objects.tests.test_schema_ui netbox_custom_objects.tests.schema netbox_custom_objects.tests.test_cot_list_stats netbox_custom_objects.tests.test_deletion

Notes for reviewers

  • Diff includes ~1k lines for security_objects.json (integration demo). Can trim or move to external reference if preferred.
  • Some commits may include Co-authored-by: Cursor from local authoring; happy to squash on merge.
  • Base branch: feature (not main).

Christian and others added 10 commits June 25, 2026 19:14
Apply only the portable-schema delta: import/export UI, choice_sets/objects
apply, security_objects.json demo, PROTECT COT deletion, list columns, and
reindex job batching — without replacing feature-branch branching/GraphQL code.
Align branching/portable-schema documentation with branch-aware schema
apply, document COT list columns in the REST API guide, and restore the
missing branch_bypass_warning.html referenced by the schema import UI.

Co-authored-by: Cursor <cursoragent@cursor.com>
Fix portable-schema and REST docs; add COT deletion blockers and bulk-delete
realignment with recursion guards (_netbox_private, get_registered_model,
get_models). Extend deletion tests and query-count expectations.

Co-authored-by: Cursor <cursoragent@cursor.com>
Collect every in-process through class via _iter_m2m_through_copies and
realign inbound/outbound FKs on each copy. Use get_registered_model() for
single and bulk deletes so stale class identity cannot break the collector.
Bulk-delete of rulebooks with polymorphic multiobject fields failed when
stale through copies kept a previous TableNModel class on the source FK.
Extend realign_inbound/outbound to patch all polymorphic through copies.
Unregister generated models and drop physical tables before ObjectType
deletion so Django's ContentType collector does not SET NULL against
orphaned polymorphic GFK columns. Skip field DDL when the backing table
is already gone.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document list-column hints, bulk-delete parity, orphaned-table cleanup,
and the distinction between schema-level blockers and object-field
on_delete_behavior (object fields only).
Align the Top-Level Structure table with the recommended authoring order:
schema_version, choice_sets, types, objects.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Christian seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@jnovinger

Copy link
Copy Markdown
Contributor

We require an approved issue before accepting PRs of this scope, and there isn't one linked here (or that I could find). This isn't a reflection on the quality of the work (the PR clearly represents significant effort), but taking on a large feature contribution without prior discussion means accepting maintenance responsibility for something we haven't had a chance to evaluate against our roadmap and architecture direction.

Please open an issue describing the feature and your proposed approach. Once we've had a chance to review and approve it, a PR will be welcome.

@jnovinger jnovinger closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants