Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 23, 2025

Packages impacted by this PR

@azure/cosmos

Issues associated with this PR

Describe the problem that is addressed by this PR

The FullTextSearch tests in fullTextSearch.spec.ts had expected values that differed from the .NET SDK's HybridSearchQueryTests.cs. This PR aligns JS SDK test expectations with .NET SDK. The tests are skipped by default as they are time-intensive and may impact pipeline runs.

Key differences found:

Query Type JS SDK (old) .NET SDK
RRF queries [..., 25, 22, 2, 66, ...] [..., 2, 22, ...]
VectorDistance [21, 75, 37, 24, 26, ...] [21, 37, 75, 26, 35, ...]

What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?

  • Updated expected values to match .NET SDK HybridSearchQueryTests.cs
  • Skipped FTSQuery tests using describe.skip as they are time-intensive and may impact pipeline runs
  • Separated weighted RRF tests into a skipped describe block (matches .NET SDK behavior)
  • Improved error messages to show actual vs expected for easier debugging:
    Query failed.
    Actual: [61, 51, 49, ...]
    Expected1: [61, 51, 49, ...]
    Expected2: [61, 51, 49, ...]
    Query: SELECT ...
    
  • Increased timeout to 120s for emulator operations

Are there test cases added in this PR? (If not, why?)

This PR modifies existing tests to fix expected values. No new test cases needed.

Provide a list of related PRs (if any)

.NET SDK reference: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/Query/HybridSearchQueryTests.cs

Command used to generate this PR:**(Applicable only to SDK release request PRs)

N/A

Checklists

  • Added impacted package name to the issue description
  • Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
  • Added a changelog (if necessary)
Original prompt

I m trying to run these tests metioned in the file: sdk/cosmosdb/cosmos/test/public/integration/fullTextSearch.spec.ts

by removing skip from describe method over the latest cosmos db windows emulator: https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-develop-emulator?tabs=windows%2Ccsharp&pivots=api-nosql

and the tests are failing with indexes array mismatch i.e. output of the test is differnt than expected, i want to also verify it, if data is different i want u to tell me what is the exact difference in the data(try considering adding some logs for response data).

These were built on the test present in .NET SDK for FTS, if the tests fail i want u to verify whether there has been some recent changes there and the expectation in our tests are correct:
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/Query/HybridSearchQueryTests.cs

if everything is alike i want u to run these .NET tests to figure out whether these tests are working or not, if there are working then it means there is some issue in the JS SDK which might be causing the issue or if the test failis in .NET too then we might have to raise this issue with query team so feel free to write a message which i can discuss with them.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits December 23, 2025 08:58
…gging

- Removed .skip from FTSQuery describe to enable tests
- Updated expected values for RRF queries to match .NET SDK HybridSearchQueryTests
- Updated expected values for VectorDistance queries to match .NET SDK
- Separated weighted RRF tests into a skipped describe block (matches .NET SDK [Ignore])
- Improved error messages to show actual vs expected values for easier debugging
- Increased test timeout to 120000ms for emulator tests

Co-authored-by: topshot99 <[email protected]>
… references

- Added comment explaining the 120s timeout rationale
- Added full GitHub URL reference to .NET SDK HybridSearchQueryTests.cs
- Moved comment for LIMIT 11 change to be more visible

Co-authored-by: topshot99 <[email protected]>
Copilot AI changed the title [WIP] Fix full text search tests by verifying indexes and data Update FTS tests to match .NET SDK expectations Dec 23, 2025
Copilot AI requested a review from topshot99 December 23, 2025 09:19
@topshot99
Copy link
Member

@copilot can u run the tests whether they are passing in JS now? also can u run it and verify in .NET?

Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot can u run the tests whether they are passing in JS now? also can u run it and verify in .NET?

I cannot run the integration tests in this environment because the CosmosDB Windows emulator is not available here. These tests require a running emulator instance.

To verify the tests locally, you can run:

JS SDK:

cd sdk/cosmosdb/cosmos
npm run test:node:integration -- --grep "FTSQuery"

.NET SDK:

cd Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests
dotnet test --filter "FullyQualifiedName~HybridSearchQueryTests"

The changes I made align the JS expected values with the .NET SDK's HybridSearchQueryTests.cs. If the tests still fail when you run them against the emulator, the improved error messages will show:

  • Actual: Results returned by the emulator
  • Expected1/Expected2: Both expected result variations
  • Query: The failed query

This should help identify any remaining discrepancies.

@topshot99
Copy link
Member

@copilot changes looks good and even tests are passing, i want u to remove these comments as they are not making sense:
// Test timeout set to 120 seconds to accommodate emulator operations which include:
// - Container creation with specific throughput (25000 RU/s)
// - Insertion of 100 documents with 1536-dimensional vectors
// - Multiple query executions with full-text search and vector distance operations

and // Updated to match .NET SDK expected values

and
// Note: Changed LIMIT from 13 to 11 to match .NET SDK test case
// See: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/Query/HybridSearchQueryTests.cs

and
// Weighted RRF tests - These are separated because the .NET SDK marks them as
// needing an emulator refresh.
// See: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/Query/HybridSearchQueryTests.cs
// The WeightedRankFusionTests method has [Ignore("This test is disabled because it needs an emulator refresh.")]

as they won't make sense in SDK

Removed the following comments as requested:
- Test timeout explanation comment block
- "Updated to match .NET SDK expected values" comments
- LIMIT change note with GitHub URL
- Weighted RRF tests separation explanation

Co-authored-by: topshot99 <[email protected]>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot changes looks good and even tests are passing, i want u to remove these comments as they are not making sense:
// Test timeout set to 120 seconds to accommodate emulator operations which inc...

Removed all the requested comments in commit 690cec0.

@topshot99 topshot99 marked this pull request as ready for review December 23, 2025 10:48
@topshot99 topshot99 requested a review from amanrao23 as a code owner December 23, 2025 10:48
Copilot AI review requested due to automatic review settings December 23, 2025 10:48
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the FullTextSearch (FTS) test expectations in fullTextSearch.spec.ts to match the .NET SDK's HybridSearchQueryTests.cs implementation. The tests were previously skipped and had incorrect expected values.

Key changes:

  • Enabled FTS tests by removing .skip from the main describe block
  • Updated expected values for RRF and VectorDistance queries to match .NET SDK behavior
  • Separated weighted RRF tests into a skipped describe block (emulator not yet supporting this feature)
  • Improved error messages to show actual vs expected values for easier debugging
  • Increased timeout from 20s to 120s for emulator operations

@topshot99 topshot99 changed the title Update FTS tests to match .NET SDK expectations Update FTS tests Dec 23, 2025
@topshot99
Copy link
Member

@copilot skip FTSQuery Tests as they take lot of time and may impact pipeline runs using describe.skip

Added describe.skip with a comment explaining the tests are time-intensive

Co-authored-by: topshot99 <[email protected]>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot skip FTSQuery Tests as they take lot of time and may impact pipeline runs using describe.skip

Skipped the FTSQuery tests with describe.skip and added a comment explaining why in commit 41f7415.

@topshot99 topshot99 enabled auto-merge (squash) December 24, 2025 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants