Skip to content

DESIGN.md

Latest commit

 

History

History
392 lines (316 loc) · 18.4 KB

DESIGN.md

File metadata and controls

392 lines (316 loc) · 18.4 KB
Raw

Design: Unity Analytics environment-modifying scripts

Purpose

This document describes how to implement scripts in this repo that modify a Unity Analytics environment (events / parameters / schemas) via the live-ops/events/v3 API. It is derived from the patterns already established in methods.py, main_copy.py, main_copy_single.py, main_compare.py, main_add_param_to_all_schemas.py, main_create_custom_echoes_of_standard_schemas.py and main_recompile.py.

The single most important property these scripts must have is the operator can be confident that the environment about to be mutated is the one they intended. Everything in this design exists to enforce that property.

1. The threat model

A script in this repo can:

  • Create new events and parameters in an environment.
  • Modify existing event schemas (add parameters, change descriptions, toggle enabled).
  • Change the type of a parameter (admin-only endpoint — change_parameter_type).
  • Copy schemas between environments.

Every one of those actions can silently corrupt a production environment if the wrong trgtOrgId / trgtProjectId / trgtEnvId is sitting in settings.json. The IDs are opaque GUIDs — visually indistinguishable. A mis-typed character in trgtEnvId does not produce an error: it produces "successful" writes against the wrong place.

The mitigation strategy below is therefore defence in depth: settings are loaded → the API is queried for the human-readable names that those IDs resolve to → the operator must confirm both the names and the deep-link URLs before any write is issued.

2. Configuration: settings.json + token.auth

Every script uses the existing init_session() helper. New scripts MUST do the same — do not roll your own loader.

from methods import init_session
rh, settings, src_project, target_project = init_session(
    validate_src=True,      # set False for write-to-source-only scripts
    validate_target=True,   # set False for read-only-of-source scripts
)

Conventions:

  • settings.json stores a dictionary of named locations plus an active pointer selecting the current source/target:

    {
  "locations": {
    "auth_linking_and_cloud_save": {
      "orgId": 18966763638557,
      "projectId": "714657ca-...",
      "envId": "4df3ce35-..."
    },
    "ilyon - jungle jam - dev": { "orgId": ..., "projectId": ..., "envId": ... }
  },
  "active": { "source": "auth_linking_and_cloud_save", "target": "ilyon - jungle jam - dev" }
}

    The friendly name is the dict key and is assumed unique. active.source / active.target each name a location (or null). An MCP tool flips the active pair by calling set_active(settings, role, friendly_name, path).

  • Backwards-compatible flat keys. Scripts still read settings["srcOrgId"] … settings["trgtEnvId"]. init_session resolves the active source/target locations and injects those six legacy flat keys into the returned settings dict, so the skeleton in §4 (and every existing main_*.py) is unchanged. Single-environment scripts still pass validate_target=False and operate against the src* triple, which now derives from active.source.

  • Legacy migration. An old flat-shaped settings.json (top-level srcOrgId/trgt* + optional suspended block) is auto-converted to the locations/active shape on load via migrate_settings_if_legacy and rewritten to disk once. No manual edit needed.

  • token.auth holds a single Bearer token copied from the token cookie at https://cloud.unity.com/. Both files are gitignored and per-operator.

  • Any script that mutates the target environment must call init_session with validate_target=True so a 200 from get_project_metadata is recorded before the first write. _fetch_project_or_refresh_token will prompt for a fresh token once and retry; if that fails the script exits.

Why fetch project metadata first

get_project_metadata is the cheapest authoritative round-trip that proves:

  1. The token is valid.
  2. The org/project pair actually exists.
  3. The operator has access to it.

Without this, a typo'd project ID fails late — possibly after a partial write — instead of failing at startup. New scripts must never skip this step.

3. The mandatory confirmation block

Every script that writes opens with the same pattern. Do not deviate:

<TITLE LINE — what the script will do, in one sentence>
-------------------------------------------------
Here's a link to the SOURCE project <name>:
https://cloud.unity.com/home/organizations/<org>/projects/<proj>/environments/<env>
[press enter to continue]

-------------------------------------------------
Here's a link to the TARGET project <name>:
https://cloud.unity.com/home/organizations/<org>/projects/<proj>/environments/<env>
[press enter to continue — THERE WILL BE NO FURTHER CONFIRMATIONS]

Rules:

  1. Print the project's display name (src_project["name"]) alongside the IDs. init_session returns the metadata for exactly this reason — use it. IDs alone are not human-verifiable; names are.
  2. Print the deep-link URL to cloud.unity.com. The operator must be able to click it, see the environment in the dashboard, and visually confirm the project name, environment name, and event/parameter list match what they expect. URL-only or name-only is not enough — together they are.
  3. Use input() to gate progression. Do not auto-proceed. The text "If this is correct, press enter to continue... If not, terminate the script and change the settings.json file." is the established phrasing — keep it.
  4. The final confirmation must say "THERE WILL BE NO FURTHER CONFIRMATIONS". This signals to the operator that this is the last point at which they can abort cheaply. Per-record confirmations during the run are explicitly not the model — they cause confirmation fatigue and get click-through'd.
  5. If the script also requires a within-environment selection (e.g. "which event do you want to copy?", "which parameter to add?"), that selection prompt comes after the env confirmations and before the "no further confirmations" line. See main_copy_single.py and main_add_param_to_all_schemas.py for the canonical shape.

Source vs Target labelling

For clarity, scripts should label environments by their role in this script, not by the settings key:

  • A copy script reads from SOURCE and writes to TARGET — both blocks shown.
  • A recompile script writes to SOURCE only — only the source block shown, but the title makes the write-intent explicit (main_recompile.py does this correctly: " RECOMPILE EVENT SCHEMAS TO MATCH THE UI").
  • An "echoes" script writes to SOURCE but the operator's mental model is that it's the target of the operation — main_create_custom_echoes_of_standard_schemas.py handles this by labelling the source-env block with "Here's a link to the TARGET project". This is fine if the title clearly says what's happening; ambiguity here is the most dangerous failure mode.

4. Standard script skeleton

from methods import init_session
from methods import recursively_get_parameters, convert_event_from_get_to_post
from methods import check_for_prefix

# 1. Load + validate config (this also validates the token).
rh, settings, src_project, target_project = init_session(
    validate_src=True, validate_target=True,
)
org_id            = settings["srcOrgId"]
project_id        = settings["srcProjectId"]
environment_id    = settings["srcEnvId"]
target_org_id     = settings["trgtOrgId"]
target_project_id = settings["trgtProjectId"]
target_env_id     = settings["trgtEnvId"]
project_name        = src_project.get("name", "Unknown Project")
target_project_name = target_project.get("name", "Unknown Project")

# 2. Title + confirmation block (see section 3).
print(" <WHAT THIS SCRIPT DOES> ")
print("-------------------------------------------------")
print(f"Here's a link to the SOURCE project {project_name}:")
print(f"https://cloud.unity.com/home/organizations/{org_id}/projects/{project_id}/environments/{environment_id}")
input("If this is correct, press enter to continue... If not, terminate the script and change the settings.json file.")

print("-------------------------------------------------")
print(f"Here's a link to the TARGET project {target_project_name}:")
print(f"https://cloud.unity.com/home/organizations/{target_org_id}/projects/{target_project_id}/environments/{target_env_id}")
input("If this is correct, press enter to continue... If not, terminate the script and change the settings.json file. \n THERE WILL BE NO FURTHER CONFIRMATIONS.")

# 3. (optional) Within-env selection — print options, take input, confirm.

# 4. Fetch — get current state of source and target.
src_schemas = rh.get_schemas(org_id, project_id, environment_id)
if not src_schemas:
    # Treat empty as "token expired". This is the established failure-mode
    # tell for these scripts. See main_copy.py for the standard message.
    print("No schemas found in source project. ...")
    exit(1)

# 5. Plan — figure out what would change. Idempotency check goes here.

# 6. Mutate — write each change, log per-record success/failure, do not abort
#    on a single failure unless the failure could leave the env corrupt.

Step 5 (Plan) is non-negotiable for write scripts

Before any write, the script must:

  1. Fetch the target environment's current state (events, parameters).
  2. Compute the diff: what would be created, updated, or skipped.
  3. Skip anything that already exists at the target (idempotency).

Examples in the codebase:

  • main_copy.py: skips parameters whose name is already in destination_params, and events whose name is already in destination_events.
  • main_create_custom_echoes_of_standard_schemas.py: skips standard events whose _custom echo already exists.
  • main_add_param_to_all_schemas.py: compares the parameter dict before/after insertion and only PUTs the schema if len(new) > len(old).

Idempotency means re-running the script after a partial failure is safe. Without it, a flaky network or expired token mid-run leaves the operator unable to resume.

5. Restricted / predefined records

The Unity Analytics environment contains records that are owned by Unity, not the operator. New scripts must respect them.

  • isRestricted: true / isPredefined: true events — skip in copy scripts (they cannot be POSTed to a target). For modify-source scripts like main_recompile.py, also skip — the API will reject the patch.
  • Reserved-prefix names — check_for_prefix(name) returns False for names starting with rsv, ddna, deltaDNA, or unity. These cannot be created in a target environment. Copy/create scripts must call this and skip with a clear log line.
  • Type OBJECT / ARRAY parameters — cannot be added as leaf children of an event's eventParams block; main_add_param_to_all_schemas.py filters these out of the operator's selection list. Any script that lets the operator pick a parameter must do the same.

6. Error handling and logging

  • Treat an empty list response from get_schemas / get_parameters as "probably an expired token" and exit with the established message that points the operator at the token cookie. Do not retry silently.
  • The request helper in methods.py already logs non-200 responses and returns {}. Per-record write failures should be print()ed with the record name and the script should continue to the next record (unless the failure indicates a systemic problem like an auth fault).
  • Do not catch exceptions broadly. If an unexpected shape comes back from the API the script should crash loudly — silent corruption is worse than a stack trace.

7. Adding a new endpoint to methods.py

When implementing a new modify-script that needs an endpoint not yet in AnalyticsUIRequestHelper:

  1. Find the route in the live-ops/events/v3 spec.
  2. Add a method that takes org_id, project_id, environment_id, ... as the first three arguments — matching every other method in the class.
  3. Use the appropriate _request_*_and_validate helper (these centralise the logging and JSON handling — do not call requests directly).
  4. Build the URL as a relative path starting with /api/... so the helper prepends self.base_url. (A few existing methods build absolute URLs inline — that's a wart, not a pattern to copy.)
  5. Validate inputs that the API will reject (see create_parameter's type whitelist and change_parameter_type's type whitelist) so the script fails fast rather than after a round-trip.

8. What a new modify-script checklist looks like

Before merging a new main_*.py:

  • Calls init_session with the correct validate_src / validate_target flags for the side(s) it will write to.
  • Title line clearly states the write intent (verb + object).
  • Prints SOURCE and/or TARGET confirmation blocks with name + URL + input() gate, in the order the operator will think about them.
  • Final gate ends with THERE WILL BE NO FURTHER CONFIRMATIONS.
  • Fetches target state and computes a diff before writing.
  • Skips restricted / predefined / reserved-prefix records with a logged reason.
  • Re-running the script after a partial failure is safe (idempotent).
  • On per-record failure, logs the record name and continues; on systemic failure, exits with a message pointing at the likely cause (token, settings, network).
  • No new direct requests.* calls — all HTTP goes through AnalyticsUIRequestHelper.

9. Endpoint inventory (live-ops/events/v3)

Sourced from routes.yaml. Two route shapes exist for every operation — the org-scoped form (/organizations/{organizationId}/projects/...) and a "NoOrg" form (/projects/...) that infers the org from auth. Stick with the org-scoped form in this codebase: settings.json always carries srcOrgId / trgtOrgId, the URL printed in confirmation blocks contains the org segment (so the operator's visual check matches the request that's about to fire), and methods.py already uses it consistently. Mixing forms across scripts is a footgun — different URLs for the same logical action makes audit/log review harder.

Operation Method + path (org-scoped) Wrapped in methods.py?
List schemas GET /schemas get_schemas
Create schema POST /schemas create_event
Get schema GET /schemas/{eventName} get_schema_by_id
Patch schema (desc + enabled) PATCH /schemas/{eventName} patch_event
Replace schema PUT /schemas/{eventName} update_schema
Copy schemas → other env(s) POST /schemas/copy copy_schemas_in_project
List parameters GET /parameters get_parameters (also list_parameters_in_environment)
Create parameter POST /parameters create_parameter
Get parameter GET /parameters/{parameterName} get_parameter_by_id
Update parameter PATCH /parameters/{parameterName} No
Delete parameter DELETE /parameters/{parameterName} No
Change parameter type (admin) PATCH /parameters/{parameterName}/admin/type/{newType} change_parameter_type
Bulk add parameters to events POST /bulk/add-parameters-to-events No

9.1 The unwrapped endpoints, and what to do about them

PATCH /parameters/{parameterName} — wrap when first needed. Same shape as create_parameter minus the name field. Standard rules apply (reserved-prefix check, type validation).

POST /bulk/add-parameters-to-events — main_add_param_to_all_schemas.py currently does this one event at a time via repeated PUT /schemas/{name}. That's N round-trips, no transactionality, and partial-failure recovery is the operator's problem. A bulk wrapper would replace that loop with one call. When wrapping, keep the per-event diff/skip logic the script already does (only send the events that actually need the parameter added) — do not blindly forward the operator's selection to the bulk endpoint.

DELETE /parameters/{parameterName} — see section 10. Do not add a wrapper without also adding the destructive-operation guards described there.

10. Destructive operations (DELETE, type changes)

The standard confirmation flow in section 3 is calibrated for additive operations: "I'm about to create things in this environment." Two endpoints are not additive and require stronger guards:

  • DELETE /parameters/{parameterName} — irreversible. The parameter is gone, and any historical events that referenced it lose their schema binding.
  • PATCH /parameters/{parameterName}/admin/type/{newType} — reversible only by another type-change call, and only if you know the original type. Type changes can break downstream queries and dashboards silently.

Rules for any script that calls one of these:

  1. Operate on TARGET, never on a SOURCE-only flow. Destructive scripts must use validate_target=True and write only against the trgt* IDs, regardless of what src* is set to. This makes "I'm about to delete from the wrong environment" much harder — the operator has to have target pointed at the wrong place, not just source.
  2. Print a per-record manifest and require a typed confirmation. Unlike additive scripts, "press enter" is not enough. After the env confirmation blocks, print the full list of records that would be affected, then require the operator to type the environment name (not the GUID, the human-readable name from target_project["name"] or get_project_metadata's env list) verbatim before proceeding. Mismatched input → exit, no retries.
  3. Refuse to act on restricted/predefined records. Belt and braces: the API will reject these, but the script should refuse before sending.
  4. Dry-run mode by default. A --apply (or equivalent) flag must be passed for the script to actually issue the destructive calls. The default invocation prints what would happen and exits.
  5. Log every destructive call with its full URL and the API's response status. This is the audit trail.

These rules apply equally to any future endpoint that mutates existing data in place (e.g. a future DELETE /schemas/{eventName} should it appear). Treat the Admin Operations tag in the spec as a flag: anything tagged admin gets the section-10 treatment, not the section-3 treatment.