Skip to content

Jbodner/timestamp tests - #4

Closed
anuraaga wants to merge 10 commits into
anuraaga:mainfrom
jonbodner-buf:jbodner/timestamp-tests
Closed

Jbodner/timestamp tests#4
anuraaga wants to merge 10 commits into
anuraaga:mainfrom
jonbodner-buf:jbodner/timestamp-tests

Conversation

@anuraaga

@anuraaga anuraaga commented Nov 11, 2026

Copy link
Copy Markdown
Owner

No description provided.

jonbodner-buf and others added 10 commits October 16, 2026 17:05
@jonbodner-buf
update to connect-go v2
f95f680 
Signed-off-by: Jon Bodner <jbodner@buf.build>
@jonbodner-buf
updates in response to feedback
02b80cc 
Signed-off-by: Jon Bodner <jbodner@buf.build>
@timostamm
Merge branch 'main' into jbodner/connect-go-v2
5be23a7
@jonbodner-buf
properly handle grpcWeb when there's an immediate error. respect trai…
f19fe95 
…lers-only.

Signed-off-by: Jon Bodner <jbodner@buf.build>
@jonbodner-buf
Merge branch 'jbodner/connect-go-v2' of github.com:jonbodner-buf/conf…
daf4a47 
…ormance into jbodner/connect-go-v2
@jonbodner-buf
update to connect-go v2 2.0.2 for SendHeader fix. Add test to validat…
8a8e1e8 
…e headers are sent immediately now.

Signed-off-by: Jon Bodner <jbodner@buf.build>
@jonbodner-buf
in referenceclient, make sure that we don't try to modify a nil resul…
cd9ea78 
…t in a defer

Signed-off-by: Jon Bodner <jbodner@buf.build>
@jonbodner-buf
ensure timestamps are integers with no signs or decimal points.
980c20b 
Signed-off-by: Jon Bodner <jbodner@buf.build>
@jonbodner-buf
Do not check for a specific http status code. Add tests for GET. Add …
a914ad7 
…tests where timestamp key is present, but there value is not.

Signed-off-by: Jon Bodner <jbodner@buf.build>
@jonbodner-buf
Merge branch 'main' into jbodner/timestamp-tests
c9d382d 
Signed-off-by: Jon Bodner <jbodner@buf.build>
@github-actions

github-actions Bot commented Nov 11, 2026

Copy link
Copy Markdown

Implementation conformance

connect-go, the reference implementation, has newly failing tests. They predict failures in its own CI.

Implementation Newly failing test cases Newly passing test cases Known-failing list Notes
connect-dart 91b0cb4 0 0 no list
connect-es bf4f24e 3 in 3 of 6 targets 0 add 3
connect-go 297fdd6 (reference) 9 in 1 of 2 targets 0 no list
connect-kotlin a70a2d7 0 0 no changes
connect-py 5ccf5e9 15 in 1 of 2 targets 0 no list
connect-rust bd2ab67 33 in 1 of 2 targets 0 no list
connect-swift 8d52280 0 0 no list

This pull request changes code or protos, so every suite ran.

Each implementation ran the suites with and without this pull request's changes, without known-failing lists, at the commit shown. Test cases whose results differed, or that couldn't run, ran once more with and without the changes (only with them for added suites), and those whose differences didn't repeat are listed as possibly flaky instead of newly failing or passing. The new failures and passes were then checked against the known-failing list each target passes to the runner. Results for c9d382d91237 from this workflow run.

Newly failing test cases

Test cases that aren't in the known-failing list need an entry there, or a fix.

connect-es
Target Test case Permutations In known-failing list
node server Connect GET Timeout Format Requests/zero 4 no
node server Connect Timeout Format Requests/server-stream/zero 4 no
node server Connect Timeout Format Requests/unary/zero 4 no
fastify Connect GET Timeout Format Requests/zero 4 no
fastify Connect Timeout Format Requests/server-stream/zero 4 no
fastify Connect Timeout Format Requests/unary/zero 4 no
express Connect GET Timeout Format Requests/zero 2 no
express Connect Timeout Format Requests/server-stream/zero 2 no
express Connect Timeout Format Requests/unary/zero 2 no
connect-go
Target Test case Permutations In known-failing list
server Connect GET Timeout Format Requests/empty 5 no
server Connect GET Timeout Format Requests/leading-plus 5 no
server Connect GET Timeout Format Requests/zero 5 no
server Connect Timeout Format Requests/server-stream/empty 5 no
server Connect Timeout Format Requests/server-stream/leading-plus 5 no
server Connect Timeout Format Requests/server-stream/zero 5 no
server Connect Timeout Format Requests/unary/empty 5 no
server Connect Timeout Format Requests/unary/leading-plus 5 no
server Connect Timeout Format Requests/unary/zero 5 no
connect-py
Target Test case Permutations In known-failing list
server Connect GET Timeout Format Requests/empty 5 no
server Connect GET Timeout Format Requests/zero 5 no
server Connect Timeout Format Requests/server-stream/decimal-point 5 no
server Connect Timeout Format Requests/server-stream/digit-separator 5 no
server Connect Timeout Format Requests/server-stream/eleven-digits 5 no
server Connect Timeout Format Requests/server-stream/empty 5 no
server Connect Timeout Format Requests/server-stream/exponent 5 no
server Connect Timeout Format Requests/server-stream/hex 5 no
server Connect Timeout Format Requests/server-stream/leading-minus 5 no
server Connect Timeout Format Requests/server-stream/leading-plus 5 no
server Connect Timeout Format Requests/server-stream/too-large 5 no
server Connect Timeout Format Requests/server-stream/unit-suffix 5 no
server Connect Timeout Format Requests/server-stream/zero 5 no
server Connect Timeout Format Requests/unary/empty 5 no
server Connect Timeout Format Requests/unary/zero 5 no
connect-rust
Target Test case Permutations In known-failing list
server Connect GET Timeout Format Requests/decimal-point 2 no
server Connect GET Timeout Format Requests/digit-separator 2 no
server Connect GET Timeout Format Requests/eleven-digits 2 no
server Connect GET Timeout Format Requests/empty 2 no
server Connect GET Timeout Format Requests/exponent 2 no
server Connect GET Timeout Format Requests/hex 2 no
server Connect GET Timeout Format Requests/leading-minus 2 no
server Connect GET Timeout Format Requests/leading-plus 2 no
server Connect GET Timeout Format Requests/too-large 2 no
server Connect GET Timeout Format Requests/unit-suffix 2 no
server Connect GET Timeout Format Requests/zero 2 no
server Connect Timeout Format Requests/server-stream/decimal-point 2 no
server Connect Timeout Format Requests/server-stream/digit-separator 2 no
server Connect Timeout Format Requests/server-stream/eleven-digits 2 no
server Connect Timeout Format Requests/server-stream/empty 2 no
server Connect Timeout Format Requests/server-stream/exponent 2 no
server Connect Timeout Format Requests/server-stream/hex 2 no
server Connect Timeout Format Requests/server-stream/leading-minus 2 no
server Connect Timeout Format Requests/server-stream/leading-plus 2 no
server Connect Timeout Format Requests/server-stream/too-large 2 no
server Connect Timeout Format Requests/server-stream/unit-suffix 2 no
server Connect Timeout Format Requests/server-stream/zero 2 no
server Connect Timeout Format Requests/unary/decimal-point 2 no
server Connect Timeout Format Requests/unary/digit-separator 2 no
server Connect Timeout Format Requests/unary/eleven-digits 2 no
server Connect Timeout Format Requests/unary/empty 2 no
server Connect Timeout Format Requests/unary/exponent 2 no
server Connect Timeout Format Requests/unary/hex 2 no
server Connect Timeout Format Requests/unary/leading-minus 2 no
server Connect Timeout Format Requests/unary/leading-plus 2 no
server Connect Timeout Format Requests/unary/too-large 2 no
server Connect Timeout Format Requests/unary/unit-suffix 2 no
server Connect Timeout Format Requests/unary/zero 2 no

@anuraaga

anuraaga commented Nov 11, 2026

Copy link
Copy Markdown
Owner Author

/rerun connect-py connectrpc/connect-py#463

@github-actions

github-actions Bot commented Nov 11, 2026

Copy link
Copy Markdown

Implementation conformance

Rerun of connect-py at connectrpc/connect-py#463, requested in this comment.

No tests newly fail.

Implementation Newly failing test cases Newly passing test cases Known-failing list Notes
connect-py 330e271 0 0 no list

This pull request changes code or protos, so every suite ran.

Each implementation ran the suites with and without this pull request's changes, without known-failing lists, at the commit shown. Test cases whose results differed, or that couldn't run, ran once more with and without the changes (only with them for added suites), and those whose differences didn't repeat are listed as possibly flaky instead of newly failing or passing. The new failures and passes were then checked against the known-failing list each target passes to the runner. Results for c9d382d91237 from this workflow run.

@anuraaga

anuraaga commented Nov 11, 2026

Copy link
Copy Markdown
Owner Author

/rerun connect-py connectrpc/connect-py#463

@github-actions

github-actions Bot commented Nov 11, 2026 •
edited
Loading

Copy link
Copy Markdown

Implementation conformance

Rerun of connect-py at connectrpc/connect-py#463, requested in this comment.

No tests newly fail.

Implementation Newly failing test cases Newly passing test cases Known-failing list Notes
connect-py 330e271 0 0 no list

This pull request changes code or protos, so every suite ran.

Each implementation ran the suites with and without this pull request's changes, without known-failing lists, at the commit shown. Test cases whose results differed, or that couldn't run, ran once more with and without the changes (only with them for added suites), and those whose differences didn't repeat are listed as possibly flaky instead of newly failing or passing. The new failures and passes were then checked against the known-failing list each target passes to the runner. Results for c9d382d91237 from this workflow run.

@anuraaga anuraaga mentioned this pull request Nov 11, 2026
Merged
@anuraaga anuraaga closed this Nov 11, 2026
timostamm pushed a commit to connectrpc/conformance that referenced this pull request Nov 11, 2026
@anuraaga
Allow rerunning tests against a PR to an implementation (#1195)
956a721 
This is a strawman's proposal, I had the idea when implementing the
connect-py side of the new timeout spec. connect org members can post a
comment something like `/rerun connect-py
https://github.com/connectrpc/connect-py/pull/463` to run that
implementation against a different base than `main`, verifying it passes
the new tests.

Note that issue comment event trigger runs as main rather than the PR
but GitHub special cases the even to use a readonly token.

During my testing, for some reason my GHA runner had an older jq
version, so I tweaked the existing syntax to work on it / newer
versions.

Can see it in anuraaga#4

/cc @jonbodner-buf @timostamm

---------

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anuraaga @jonbodner-buf @timostamm