Skip to content

Conversation

@gerlowskija
Copy link
Contributor

@gerlowskija gerlowskija commented Mar 31, 2025

https://issues.apache.org/jira/browse/SOLR-16395\

Description

Many v2 APIs remain on the legacy v2 API framework, including a number of APIs allowing individual and bulk modifications to collection schemas.

Solution

This PR converts a number of Solr's schema-modification APIs to JAX-RS. It also makes some tweaks to the appearance of the v2 endpoints, to make them align a little better with the overall "REST-ful" design being targeted by our v2 APIs.

The following APIs have been converted and updated.

  • "add-field"/"update-field" - now at PUT /collections/collName/schema/fields/fieldName {...}
  • "delete-field" - now at DELETE /collections/collName/schema/fields/fieldName
  • "add-field-type"/"update-field-type" - now at PUT /collections/collName/schema/fieldtypes/fieldTypeName {...}
  • "delete-field-type" - now at DELETE /collections/collName/schema/fieldtypes/fieldTypeName
  • "add-dynamic-field"/"update-dynamic-field" - now at PUT /collections/cName/schema/dynamicfieldtypes/dFName
  • "delete-dynamic-field" - now at DELETE /collections/cName/schema/dynamicfieldtypes/dFName
  • "bulk operation API" - now at POST /collections/cName/schema/bulk

See the ref-guide updates in this PR for complete details on updated v2 API appearance/operation.

JAX-RS conversion has the additional benefit of adding the APIs to our OpenAPI spec ("OAS"), and to all of the downstream artifacts (e.g. SolrJ bindings) generated from that.

Tests

See new tests (mostly serialization/deserialization related) in SchemaManagerUtilsTest and SchemaChangeOperationSerializationTest.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

Will enable SchemaChangeOperation to serve as the processing format for
both v1 and v2 requests down the line, when SchemaManager is cut over to
it.
Still more test failues to address, but progress!
@gerlowskija
Copy link
Contributor Author

This isn't quite ready for prime-time yet, but I wanted to make the broad strokes available for review if anyone's curious.

Much of the code in this PR is currently in service of making SchemaManager (which is responsible for orchestrating "bulk" schema modifications) consume a format that can be used by both v1 and v2 APIs. Code to expose the new v2 APIs is still needed.

Still needed:
  - finish "bulk" API
  - prompt PR discussion about "copyfield" APIs
  - ref-guide updates
  - remove existing Api-based v2 APIs (where present)
  - unit tests for parameter-validation
  - manual tests for error cases (to make sure errors come through
    similarly)
This cleans up most of the remaining TODOs prior to this being
"ready-for-review", including removal of the old 'Api' based Schema
binding, updates to our ref-guide, and some additional unit tests to
ensure that missing required properties trigger appropriate errors.

Still TODO is a better look at the generated v2 request objects, which
aren't picking up the request-bodies for certain endpoints for whatever
reason.  Some debugging still needed there...
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Apr 3, 2025
@gerlowskija gerlowskija marked this pull request as ready for review April 3, 2025 11:33
@gerlowskija
Copy link
Contributor Author

gerlowskija commented Apr 3, 2025

Alright, the JAX-RS code is in place so I've marked this as "Ready for Review". This has turned into a pretty big PR, but it does cover a pretty large chunk of our API surface so it represents a pretty big step forward in our v2 coverage.

One thing that I've left out of this PR, that we'll want to think through and handle separately are v2 APIs for "copy-field" modification. All of our other schema entities (fields, field-types, and dynamic-fields) have names that allow you to address them with nice clean resource-based paths like PUT or DELETE /collections/<collName>/schema/fields/<fieldName>. But copy-fields break this pattern - they don't really have an ID or name that allows them to be addressed after creation.

I wasn't sure how to handle them, so I've "punted" in this PR. The best approach I can think of currently is a POST /schema/copyfields {...}, but it deserves some more thought/discussion.

@gerlowskija
Copy link
Contributor Author

gerlowskija commented Apr 3, 2025

Still TODO (mostly reminders to self pre-merge):


createJettyAndHarness(
tmpSolrHome, "solrconfig-managed-schema.xml", "schema-rest.xml", "/solr", true, null);
if (random().nextBoolean()) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The v2 Schema APIs now differ from v1 by more than just the path prefix, so this quick-and-dirty randomization no longer works.

@gerlowskija gerlowskija merged commit e0cff13 into apache:main Apr 9, 2025
4 checks passed
@gerlowskija gerlowskija deleted the SOLR-16395-bulk-schema-api-jaxrs branch April 9, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat:api cat:schema client:solrj dependencies Dependency upgrades documentation Improvements or additions to documentation tests tool:build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant