Skip to content

Commit ba7bbbb

Browse files
committed
Added explanation of why this tool is useful, and how it differs from running manual tests.
1 parent 458777a commit ba7bbbb

1 file changed

Lines changed: 68 additions & 0 deletions

File tree

tools/PackageCompatibility/README.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,74 @@ Specifically, it:
3333
The app is designed to run against both **published NuGet packages** and **locally-built packages**
3434
(via the `packages/` directory configured in `NuGet.config`).
3535

36+
## How This Differs From the Existing Test Suite
37+
38+
The SqlClient repository has a large suite of unit, functional, and manual tests. This tool
39+
complements — but does not replace — those tests. The key differences are:
40+
41+
### Heterogeneous package versions
42+
43+
The test projects reference sibling packages via project references or a shared `Directory.Packages.
44+
props` file. Every test run uses a **single, uniform version set** derived from whatever is
45+
currently checked out. You cannot ask the test suite to run `SqlClient 7.0.1` against
46+
`AkvProvider 7.1.0-preview1` without editing project files.
47+
48+
This tool accepts any combination of independent version numbers — including versions that have not
49+
been published yet — at the command line:
50+
51+
```bash
52+
dotnet run \
53+
-p:SqlClientVersion=7.0.1 \
54+
-p:AkvProviderVersion=7.1.0-preview1 \
55+
-- -c "<connection string>"
56+
```
57+
58+
This makes it straightforward to answer questions like *"does the new AKV provider build work
59+
against the last published SqlClient release?"* without modifying any source files.
60+
61+
### Pre-release and locally-built packages
62+
63+
Because NuGet resolves packages from the `packages/` local feed before falling back to NuGet.org,
64+
you can drop pre-release `.nupkg` files in that folder and reference them immediately — even before
65+
they have been published. The existing tests have no equivalent mechanism; they can only reference
66+
packages that are either checked out as source or already published to a configured feed.
67+
68+
### End-to-end runtime coverage across the full package graph
69+
70+
The test suite exercises individual classes and APIs in isolation. Functional and manual tests do
71+
open real connections, but they always run against the packages as built from source in the current
72+
branch.
73+
74+
This tool loads every package in the dependency graph simultaneously in a single process and then
75+
opens a live `SqlConnection`. This catches a class of failures that isolated tests miss:
76+
77+
- **Binding redirect conflicts**: two packages pulling in incompatible versions of a shared
78+
dependency (`Azure.Core`, `Microsoft.Identity.*`, etc.) that only manifest when all packages are
79+
present in the same AppDomain.
80+
- **Transitive version mismatches**: a package expecting an internal API surface that has changed in
81+
a sibling package across a version boundary.
82+
- **Registration side-effects**: authentication providers or other singleton registrations that
83+
interfere when packages are composed in an unexpected order or version combination.
84+
85+
### Diagnostic console output
86+
87+
When a connection fails, the tool can emit structured TDS-level and authentication trace output via
88+
the `--log` and `--trace` flags. This output is written directly to the console and requires no
89+
test harness or log configuration — useful for quickly diagnosing authentication failures in CI
90+
environments or on developer machines where full test infrastructure is unavailable.
91+
92+
The test suite's diagnostics are routed through `EventSource`/`DiagnosticListener` and are only
93+
visible if a listener is attached (e.g. via `dotnet-trace` or a custom test initializer).
94+
95+
### What this tool does NOT do
96+
97+
- It does not assert on individual API behaviours, query results, or error messages. For that, use
98+
the existing unit and functional tests.
99+
- It cannot run without a real SQL Server instance. The manual test suite has the same constraint
100+
for connectivity tests, but unit and functional tests run without a server.
101+
- It does not cover every authentication mode automatically. You must provide a suitable connection
102+
string for each mode you want to validate.
103+
36104
## Project Layout
37105

38106
- `src/` contains the tool source files and project file.

0 commit comments

Comments
 (0)