Skip to content

Split 2 of 7: episode and campaign data model - #1793

Draft
martien-wdy wants to merge 1 commit into
split/1-timelinefrom
split/2-episode-model
Draft

Split 2 of 7: episode and campaign data model#1793
martien-wdy wants to merge 1 commit into
split/1-timelinefrom
split/2-episode-model

Conversation

@martien-wdy

Copy link
Copy Markdown
Contributor

Problem

See chunk 1 for why pull request #1751 is being split. This is chunk 2 of 7.

Cause

A device that records sensor data needs a durable on-device representation of what it recorded, when, and under what uncertainty, before anything can capture into it or upload out of it.

Solution

Adds the episode and campaign data model:

  • Manager owns the on-device episode store, including its quota and eviction policy, so a device that records more than its data partition can hold evicts oldest first rather than filling the filesystem.
  • model.go and model_io.go define the episode manifest and its atomic read and write path, including recovery from a manifest interrupted mid write.
  • campaign.go defines the campaign: the description of what a device should record, which sources it resolves to, and the health checks that refuse a campaign naming an unhealthy source rather than silently recording nothing.
  • clock_linux.go and clock_other.go bracket agent receipts on CLOCK_BOOTTIME and sample the monotonic mapping, using the timeline from chunk 1.

Two campaign examples move in with this chunk rather than with the rest of the examples in chunk 7, because the package's tests parse them as fixtures and would otherwise fail here. See the seam note at the end.

Deliberately left to later chunks

Nothing imports this package yet. The agent side DataService that writes into it arrives in chunk 5, the uploader that drains it in chunk 6, and the command line tools that read it in chunk 7.

Depends on

Chunk 1 (split/1-timeline), which this pull request is based on.

Verification

  • CC=/usr/bin/clang go build ./... succeeds.
  • CC=/usr/bin/clang go test ./go/internal/agent/data/... passes.
  • gofmt -l go/ is empty and go vet ./go/internal/agent/data/... is clean.
  • Diff size: 5115 lines.

Seam note

Examples/WendyDataCampaign/campaign.yaml and Examples/WendyDataModelApp/campaign.yaml were moved forward from chunk 7 into this chunk. campaign_audio_test.go and example_campaign_test.go read them as fixtures, and four of the five tests in campaign_audio_test.go are ordinary unit tests unrelated to the fixtures. Moving the two small YAML Ain't Markup Language (YAML) files forward keeps all of that coverage in the chunk that owns the code; moving the test files back to chunk 7 instead would have dropped it.

Adds the on-device episode store, its manifest model and the campaign
description that selects what a device records. Nothing imports the
package yet; the capture, upload and tooling chunks build on it.

Part 2 of 7 in the split of the Wendy Data Platform change.
if e != nil {
return e
}
defer f.Close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant