Add Reranker service to lambda

[bamader]

Description

This PR adds the Reranker service into the TTC repo and to the lambda function handler. It creates the Reranker as a separate service class, following the pattern of the Embedder, and incorporates reranking into the record processing pipeline defined in the text to code lambda function.

It also makes a small change to the index creation variables in the index lambda to use the EF Construction and M-Value parameters we experimentally determined were best for performance.

Related Issues

Closes #307

Additional Notes

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

• I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
• I have reviewed my changes to ensure they are clear, concise, and well-documented.
• I have updated the documentation, if applicable.
• I have added or updated test cases to cover my changes, if applicable.
• I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

• The code follows best practices and conventions.
• The changes implement the desired functionality or fix the reported issue.
• The tests cover the new changes and pass successfully.
• Any potential edge cases or error scenarios have been considered.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.20%. Comparing base (941e6cf) to head (e9dd82e).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #365      +/-   ##
==========================================
+ Coverage   93.11%   93.20%   +0.09%     
==========================================
  Files          38       40       +2     
  Lines        2077     2105      +28     
==========================================
+ Hits         1934     1962      +28     
  Misses        143      143              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nickclyde]

Thanks Brandon! The overall structure looks clean and follows the existing Embedder pattern well. I flagged one critical issue with the Dockerfile that will cause the retriever model to be overwritten at build time, plus a few smaller items.

Also note: lambda_function.py:237-241 (on main) has a hardcoded test candidate "test test test" that overwrites the real selected_candidate. I think this means, deployed as is, the reranker will currently score against that dummy value rather than the actual extracted text. I already wrote an issue for this (#367), but figured I'd call it out.

[robertandremitchell]

lgtm, just wanted to confirm I understand the retriever v. reranker distinction. We are just using an "out-of-the-box" reranker right? whereas we're still figuring out the best way to load the retriever?

[nickclyde]

Thanks for addressing my comments, great work!!