Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nexus] webhooks #7277

Open
wants to merge 174 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
174 commits
Select commit Hold shift + click to select a range
59a10fc
[nexus] Webhook API skeleton
hawkw Dec 12, 2024
4ece574
naming consistency edits from @augustuswm
hawkw Dec 18, 2024
6e8f8ac
fix cargo workspace hack
hawkw Dec 19, 2024
478b2ce
update to match RFD 538
hawkw Jan 2, 2025
c730c63
also do event classes APIs
hawkw Jan 2, 2025
d5ed2ba
apparently this is also a thing we need to do
hawkw Jan 2, 2025
5d4f13b
[nexus] start DB model for webhooks
hawkw Dec 18, 2024
5bf2633
message queue
hawkw Dec 18, 2024
52efc08
more diesel plumbing
hawkw Dec 18, 2024
72754d7
terminology tweaks
hawkw Jan 2, 2025
fa75a2f
change tracking of delivery attempts
hawkw Jan 2, 2025
f9e5fc9
s/deliverey/delivery
hawkw Jan 2, 2025
432057a
add 'failed_timeout' delivery result
hawkw Jan 2, 2025
52d0f24
ag
hawkw Jan 2, 2025
1aa95c3
ag
hawkw Jan 2, 2025
75ac350
more models for webhook delivery
hawkw Jan 3, 2025
59c98a0
models for receivers
hawkw Jan 3, 2025
9fdd77a
s/delivery_attempts/delivery_attempt
hawkw Jan 3, 2025
b8d1efa
add probes to receiver models
hawkw Jan 3, 2025
c668252
turn event class globs into `SIMILAR TO` patterns
hawkw Jan 4, 2025
58ab5d2
rudimentary receiver create
hawkw Jan 8, 2025
e859f99
prefix all webhook UUID kinds with webhook
hawkw Jan 8, 2025
e4194dc
blargh trailing commas
hawkw Jan 8, 2025
ac82a71
fix webhook rx model not being selectable
hawkw Jan 8, 2025
1ea3a4a
bleh okay this globbing design is inefficient but does work
hawkw Jan 8, 2025
eca3b71
evaluate subscription globs on creation
hawkw Jan 9, 2025
d679139
rework subscription creation queries a bit
hawkw Jan 10, 2025
e894112
webhook dispatcher background task
hawkw Jan 10, 2025
7deece3
warnings
hawkw Jan 10, 2025
a35d51a
add empty delivery task and activate as needed
hawkw Jan 10, 2025
f5b7087
add empty delivery task and activate as needed
hawkw Jan 10, 2025
e6544e7
implement a big chunk of webhook delivery
hawkw Jan 14, 2025
71cebff
ag
hawkw Jan 14, 2025
bd8356b
blarg
hawkw Jan 14, 2025
729c245
asdf
hawkw Jan 15, 2025
0f57b33
okay there we go
hawkw Jan 15, 2025
848154c
don't select the whole event row
hawkw Jan 15, 2025
d343467
wip delivery finish attempt query
hawkw Jan 15, 2025
4ad4e04
secrets plumbing
hawkw Jan 16, 2025
ac5bf67
start api plumbing
hawkw Jan 22, 2025
80f21f2
webhook create
hawkw Jan 23, 2025
7643238
okay wow let's have a rudimentary test
hawkw Jan 23, 2025
ac71f4f
caller provides uuid
hawkw Jan 23, 2025
92e4fc3
assert there's some headers
hawkw Jan 23, 2025
8e24395
fix missing content-type, separate request build errors
hawkw Jan 23, 2025
564c8b9
capture schema versions when creating globs
hawkw Jan 27, 2025
ec661c7
wip
hawkw Jan 29, 2025
2275170
fromstr
hawkw Jan 31, 2025
3127d46
actually finish turning event classes into enums
hawkw Jan 31, 2025
d963b3d
post rebase fixup
hawkw Feb 3, 2025
e6dc20b
wip attempt dispatch query
hawkw Feb 4, 2025
d88cde7
start redoing dispatch sstuff
hawkw Feb 4, 2025
6bf2578
okay cool
hawkw Feb 5, 2025
76a484c
finish rewriting dispatcher to use unique index to dedup
hawkw Feb 6, 2025
619edcf
unstub secret_create api
hawkw Feb 6, 2025
a2106fa
[db-macros] support for children without names in `lookup_resource!`
hawkw Feb 7, 2025
ce34509
wip
hawkw Feb 7, 2025
fe518f6
[db-macros] handle DbTypedUuid in child resource parent lookup
hawkw Feb 7, 2025
63f142c
agh
hawkw Feb 7, 2025
05c4d3a
aghh
hawkw Feb 7, 2025
a6d81cf
aghhh
hawkw Feb 7, 2025
f93e8d4
aghhhh
hawkw Feb 7, 2025
7822fb3
deliverator: look up RX secrets, error handling tweaks
hawkw Feb 10, 2025
98085e9
oops i forgot to register the authz resources
hawkw Feb 10, 2025
a4952d9
goddamn sql trailing comma nonsense
hawkw Feb 10, 2025
2d13b4e
sign webhook payloads
hawkw Feb 10, 2025
e53865b
verify signatures in webhook integration tests
hawkw Feb 11, 2025
ddc648c
test receivers with multiple secrets
hawkw Feb 11, 2025
e537872
workaround for alexliesenfeld/httpmock#119
hawkw Feb 11, 2025
c8f8433
add tests for dispatching to multiple receivers
hawkw Feb 11, 2025
d9b479b
apparently reqwest will return Ok for 5xxs
hawkw Feb 13, 2025
47234f7
retry backoff!
hawkw Feb 14, 2025
cd53607
Merge branch 'main' into eliza/webhook-models
hawkw Feb 19, 2025
47cae82
Merge branch 'main' into eliza/webhook-models
hawkw Feb 24, 2025
d8540a4
lockfile 'n' stuff
hawkw Feb 24, 2025
31f2fa9
remove children
hawkw Feb 24, 2025
e79bf64
webhook receivers are lookup by name now
hawkw Feb 24, 2025
366fa2b
probe api stub
hawkw Feb 25, 2025
d9b080a
share webhook reqwest client with the API
hawkw Feb 25, 2025
890588d
factor out actually sending requests
hawkw Feb 25, 2025
aee6285
use external DNS resolver for webhook endpoints
hawkw Feb 25, 2025
5c97943
reticulating delivery triggers
hawkw Feb 25, 2025
0a85b40
remove `disable_probes`
hawkw Feb 25, 2025
6024b2d
reticulating probes
hawkw Feb 26, 2025
e668a3e
reticulating probes
hawkw Feb 26, 2025
7005048
reticulating probes
hawkw Feb 26, 2025
8cb055a
reticulating a lot of probes
hawkw Feb 26, 2025
fd706ea
add test for probes
hawkw Feb 27, 2025
ca1f998
fix probes having attempt 2
hawkw Feb 27, 2025
63ad0c8
fixup probe test
hawkw Feb 27, 2025
6d07590
reticulating delivery list
hawkw Feb 27, 2025
70032b0
reticulating delivery list query
hawkw Feb 27, 2025
3eb7368
query for listing failed resendable deliveries
hawkw Feb 27, 2025
233c7d5
resend failed deliveries on successful probes
hawkw Feb 28, 2025
e7ee2f6
fix deliverator OMDB status counting failures as ok
hawkw Feb 28, 2025
b21c1d2
query to list globs that need reprocessing
hawkw Feb 28, 2025
fa0d05c
glob reprocessing
hawkw Mar 3, 2025
86b723e
delint
hawkw Mar 3, 2025
c61f2fb
redo migration to reflect newest schema
hawkw Mar 3, 2025
16a4c14
start on a test for glob reprocessing
hawkw Mar 3, 2025
6c3e703
assert more stuff in glob-reprocessing test
hawkw Mar 4, 2025
8c93e91
use proper `IdentityMetadata` in views
hawkw Mar 4, 2025
ad4ba5b
quick test for webhook lookups
hawkw Mar 4, 2025
54236f7
ssrf prevention will be done via IP_BOUND_IF
hawkw Mar 4, 2025
440ab83
implement webhook rx delete
hawkw Mar 5, 2025
3c7f65c
flatten API routes (per @david-crespo)
hawkw Mar 5, 2025
3730f16
properly fix tests
hawkw Mar 5, 2025
46a73f2
rm experimental
hawkw Mar 5, 2025
2aaca1e
implement most of the object-list APIs
hawkw Mar 6, 2025
cdb84ed
clearer API naming (less metonymy)
hawkw Mar 6, 2025
6b0ae4e
clippy placation
hawkw Mar 6, 2025
bcc4215
reticulating deliveries
hawkw Mar 6, 2025
6422da0
reticulating deliveries
hawkw Mar 6, 2025
b72f03c
bite the bullet and give deliveries a state enum
hawkw Mar 7, 2025
0e435ed
enums in their own file
hawkw Mar 7, 2025
7adca87
stick nexus IDs in attempt records
hawkw Mar 7, 2025
7813be8
make events lookupable
hawkw Mar 7, 2025
8e7c039
add thingy for checking if an rx is subscribed
hawkw Mar 8, 2025
e147a82
implement resend api
hawkw Mar 9, 2025
0bcd73f
rename receiver update endpoint for consistency
hawkw Mar 10, 2025
4891e3f
make webhook receiver names unique
hawkw Mar 10, 2025
e0ffe66
fix webhook-receiver-delete concurrency control
hawkw Mar 10, 2025
7e6643e
redo receiver CRUD
hawkw Mar 10, 2025
ff9c213
oops forgot to update this
hawkw Mar 11, 2025
4c72109
ensure index is used when deleting secrets
hawkw Mar 11, 2025
a42546f
Merge branch 'main' into eliza/webhook-models
hawkw Mar 11, 2025
9b262a0
fix merge mistakes
hawkw Mar 11, 2025
f9c238b
post merge fixup (mostly for #7570)
hawkw Mar 11, 2025
1768140
make subscription changes optional in rx updates
hawkw Mar 11, 2025
26a029f
blech more test updates
hawkw Mar 11, 2025
9f2ae15
you have to actually implement the endpoint
hawkw Mar 11, 2025
dbfc8a7
fix authz for webhook secrets
hawkw Mar 11, 2025
6bb16ac
add authz tests for most of webhooks
hawkw Mar 11, 2025
bf822de
rustfmt & clippy placation
hawkw Mar 12, 2025
52fa346
reorganize methods in webhooks.rs
hawkw Mar 12, 2025
5a191d0
make secret-delete more like other API endpoints
hawkw Mar 12, 2025
1e4ce47
add event classes list/fetch APIs
hawkw Mar 12, 2025
43c1842
webhook event classes participate in authz now
hawkw Mar 12, 2025
b91f0c9
big theory statement
hawkw Mar 12, 2025
4854395
update OMDB tests
hawkw Mar 12, 2025
f88e4fa
that's not supposed to be there, sorry
hawkw Mar 12, 2025
b5e6a0e
rustdoc fixup
hawkw Mar 13, 2025
4d68e89
update lookup expectorate
hawkw Mar 13, 2025
3169e64
omdb background task status
hawkw Mar 13, 2025
143b34b
Merge branch 'main' into eliza/webhook-models
hawkw Mar 13, 2025
032d5ab
fix nexus-config test
hawkw Mar 14, 2025
d0b91e3
remove unnecessary event class get endpoint
hawkw Mar 14, 2025
4f8c258
improve API docs
hawkw Mar 14, 2025
28e851b
apparently markdown is okay!
hawkw Mar 14, 2025
c498066
Update nexus/external-api/src/lib.rs
hawkw Mar 14, 2025
2e7185e
update expectorate query
hawkw Mar 14, 2025
7dbf3c4
update authz tests
hawkw Mar 17, 2025
1d2c037
rm unnecessary into
hawkw Mar 17, 2025
f591ac1
i now understand which UUIDs need to be hardcoded
hawkw Mar 18, 2025
7223f2f
fix index naming
hawkw Mar 18, 2025
9190c8c
reflect docs change in openapi document
hawkw Mar 18, 2025
4151a2c
remove defunct endpoint from tests
hawkw Mar 18, 2025
e1ce3c6
fix accidentally empty migration step
hawkw Mar 19, 2025
8ebd31c
Merge branch 'main' into eliza/webhook-models
hawkw Mar 19, 2025
9addb4e
Update README.adoc
hawkw Mar 20, 2025
9908b4f
Update README.adoc
hawkw Mar 20, 2025
4ae319d
Update nexus_config.rs
hawkw Mar 20, 2025
f76ba0c
make schema.rs fields match dbinit
hawkw Mar 20, 2025
fbff139
remove spare pipe
hawkw Mar 20, 2025
0ad888b
Merge branch 'main' into eliza/webhook-models
hawkw Mar 20, 2025
4069eea
use @david-crespo's lovely timestamp pagination from #7842
hawkw Mar 20, 2025
1eb8aae
whoops forgot to update oepnapi again
hawkw Mar 21, 2025
c1e671f
rename `trigger` to `triggered_by`
hawkw Mar 21, 2025
334249f
rename `time_delivery_started` to `time_leased`
hawkw Mar 21, 2025
5b42ea5
remove unused default impl
hawkw Mar 21, 2025
8c0d0ee
Apply @smklein's suggestions
hawkw Mar 21, 2025
d44d002
also update trigger's name where clause in index
hawkw Mar 21, 2025
5c304d4
update query expectorate tests
hawkw Mar 21, 2025
0b8b1b6
use `IncompleteOnConflictExt`
hawkw Mar 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix cargo workspace hack
hawkw committed Feb 3, 2025
commit 6e8f8acd0659410d72e1fa581f2d63cd19d9f5b6
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions workspace-hack/Cargo.toml
Original file line number Diff line number Diff line change
@@ -105,7 +105,7 @@ reqwest = { version = "0.12.9", features = ["blocking", "cookies", "json", "rust
rsa = { version = "0.9.6", features = ["serde", "sha2"] }
rustls = { version = "0.23.19", features = ["ring"] }
rustls-webpki = { version = "0.102.8", default-features = false, features = ["aws_lc_rs", "ring", "std"] }
schemars = { version = "0.8.21", features = ["bytes", "chrono", "uuid1"] }
schemars = { version = "0.8.21", features = ["bytes", "chrono", "url", "uuid1"] }
scopeguard = { version = "1.2.0" }
semver = { version = "1.0.24", features = ["serde"] }
serde = { version = "1.0.217", features = ["alloc", "derive", "rc"] }
@@ -128,6 +128,7 @@ toml = { version = "0.7.8" }
toml_datetime = { version = "0.6.8", default-features = false, features = ["serde"] }
toml_edit-3c51e837cfc5589a = { package = "toml_edit", version = "0.22.22", features = ["serde"] }
tracing = { version = "0.1.40", features = ["log"] }
url = { version = "2.5.3", features = ["serde"] }
usdt = { version = "0.5.0" }
usdt-impl = { version = "0.5.0", default-features = false, features = ["asm", "des"] }
uuid = { version = "1.12.0", features = ["serde", "v4"] }
@@ -225,7 +226,7 @@ reqwest = { version = "0.12.9", features = ["blocking", "cookies", "json", "rust
rsa = { version = "0.9.6", features = ["serde", "sha2"] }
rustls = { version = "0.23.19", features = ["ring"] }
rustls-webpki = { version = "0.102.8", default-features = false, features = ["aws_lc_rs", "ring", "std"] }
schemars = { version = "0.8.21", features = ["bytes", "chrono", "uuid1"] }
schemars = { version = "0.8.21", features = ["bytes", "chrono", "url", "uuid1"] }
scopeguard = { version = "1.2.0" }
semver = { version = "1.0.24", features = ["serde"] }
serde = { version = "1.0.217", features = ["alloc", "derive", "rc"] }
@@ -251,6 +252,7 @@ toml_datetime = { version = "0.6.8", default-features = false, features = ["serd
toml_edit-3c51e837cfc5589a = { package = "toml_edit", version = "0.22.22", features = ["serde"] }
tracing = { version = "0.1.40", features = ["log"] }
unicode-xid = { version = "0.2.6" }
url = { version = "2.5.3", features = ["serde"] }
usdt = { version = "0.5.0" }
usdt-impl = { version = "0.5.0", default-features = false, features = ["asm", "des"] }
uuid = { version = "1.12.0", features = ["serde", "v4"] }