Update dependencies and spaCy model loading

[puja-trivedi]

Current Changes
pyproject.toml: narrow litellm version upper bound from <1.80.0 to <1.60.0 to avoid a regression where newer versions accidentally import litellm.proxy.proxy_server, pulling in proxy-only dependencies (apscheduler) even when the proxy is not used.

Reverted Changes to Original PR- Outdated:

• ner_tool.py: Remove automatic spaCy model download at runtime; raise an explicit error if the model is not installed, prompting the user to install it manually.
• pyproject.toml: Add en_core_web_sm as a direct dependency via URL so the model is installed at project setup time rather than downloaded at runtime via spacy.cli.download(), which uses pip internally and fails in non-pip environments (e.g. uv)

[gemini-code-assist]

Summary of Changes

Hello @puja-trivedi, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines the project's dependency management and enhances the robustness of spaCy model loading. It addresses issues with runtime model downloads and prevents unintended dependency imports by adjusting version constraints, leading to a more stable and predictable development environment.

Highlights

• Dependency Management for spaCy Model: The en_core_web_sm spaCy model is now added as a direct URL dependency in pyproject.toml. This ensures the model is installed during project setup, bypassing runtime downloads that can fail in non-pip environments like uv.
• LiteLLM Version Constraint: The upper bound for the litellm dependency in pyproject.toml has been narrowed from <1.80.0 to <1.60.0. This change prevents accidental imports of litellm.proxy.proxy_server and its associated proxy-only dependencies (like apscheduler) when the proxy functionality is not in use.
• Robust spaCy Model Loading: Automatic runtime downloading of the spaCy model has been removed from ner_tool.py. Instead, if the model is not found, an explicit OSError is raised, guiding the user to manually install it, which improves clarity and control over model installation.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• pyproject.toml
  • Added en-core-web-sm as a direct URL dependency to ensure installation at project setup.
  • Updated the litellm version constraint to <1.60.0 to avoid importing proxy-only dependencies.
• src/utils/ner_tool.py
  • Removed the import of spacy.cli.download.
  • Modified the get_spacy_model function to raise an OSError if the spaCy model is not found, instead of attempting an automatic download.

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request improves dependency management by making the spaCy model a direct project dependency and removing runtime downloads, which is a great change for robustness. I've identified a potential version incompatibility issue with the spacy package and its model that could break the application in the future. Additionally, I've suggested an improvement to an error message to make it consistent with the new dependency strategy.

[tekrajchhetri]

@puja-trivedi any particular reason for including the model in pyproject.toml? Why not put it in the code, you can download the model automatically. For now we've pre-defined models, but say we decided to take it from env files then it won't work.

[puja-trivedi]

I ran into an issue when using uv; however, I was able to fix that now. I also understand your point about the env files so I have reverted that change. However the version changes I made for spacy and litellm should stay because of the non-fatal runtime error I was getting.

Further explanation about the error:
The error is happening because a newer version of LiteLLM has an internal code path in its logging/cold-storage handler that accidentally imports litellm.proxy.proxy_server, which requires apscheduler (a proxy-only dependency). This happens even when you're not using the LiteLLM proxy — it's a bug/regression in LiteLLM itself.
Key points:
1. It's a non-fatal ERROR log, not a crash. The extraction pipeline should still work — LiteLLM catches this exception internally and continues.
2. Don't need litellm[proxy] — that would pull in a bunch of proxy dependencies we don't need.
3. The real fix is to pin LiteLLM to a version that doesn't have this regression. Try tightening the upper bound.