feat: Add Indeed.com crawling and improve crawling prompts

[KevinJBoyer]

Ticket

https://navalabs.atlassian.net/browse/DST-1253

Changes

• Move "two sentence" specifier from description from Pydantic model to the extract_supports prompt (and update the prompt version)
• Add a prompt for crawl_indeed
• Update the prompt for crawl_gcta to provide a more informative description
• Add a helper utility to the makefile to easily update and add crawl jobs

Context for reviewers

See ticket

Testing

make copy-prompts
make add-crawl-jobs
make process-crawl-jobs

Preview environment for frontend

♻️ Environment destroyed ♻️

Preview environment for app

♻️ Environment destroyed ♻️

[KevinJBoyer]

Moved into the prompt for extract_supports in Phoenix

[Copilot]

Pull Request Overview

This PR updates prompt versions in the application configuration and removes field-level documentation from the SupportEntry model. The changes appear to be coordinating updates to LLM prompt versions with a simplified schema definition.

• Updated prompt version IDs for extract_supports and crawl_gcta operations
• Added new prompt version for crawl_indeed operation
• Removed the Field description metadata from the description field in SupportEntry

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
app/src/ingestion/support_entry.py	Removed Field description metadata for the description attribute in the SupportEntry model
app/src/app_config.py	Updated prompt version IDs for existing operations and added new crawl_indeed prompt configuration

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The removal of the Field description metadata ('2-sentence summary, including offerings') eliminates important guidance for LLM-based extraction. The SUPPORT_ENTRY_SCHEMA on line 15 uses model_json_schema() which includes field descriptions in the schema. This description helps the LLM understand what type of content to generate for this field. Consider restoring the Field description or documenting why this guidance was removed, especially since the prompt versions are being updated in this same PR.

Suggested change

	description: str | None
	description: str | None = Field(description="2-sentence summary, including offerings")

[yoomlam]

Missing Makefile? make add-crawl-jobs doesn't work

[yoomlam]

Nice!