Skip to content

Feat/support deep research #567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

Feat/support deep research #567

wants to merge 8 commits into from

Conversation

pftom
Copy link
Contributor

@pftom pftom commented Mar 7, 2025

Summary

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Tip

Close issue syntax: Fixes #<issue number> or Resolves #<issue number>, see documentation for more details.

Impact Areas

Please check the areas this PR affects:

  • Multi-threaded Dialogues
  • AI-Powered Capabilities (Web Search, Knowledge Base Search, Question Recommendations)
  • Context Memory & References
  • Knowledge Base Integration & RAG
  • Quotes & Citations
  • AI Document Editing & WYSIWYG
  • Free-form Canvas Interface
  • Other

Screenshots/Videos

Before After
... ...

Checklist

Important

Please review the checklist below before submitting your pull request.

  • This change requires a documentation update, included: Refly Documentation
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

…kflow

This commit adds a comprehensive deep research skill that:
- Supports multi-stage research with dynamic topic exploration
- Implements structured search, extraction, and analysis processes
- Provides detailed research synthesis and progress tracking
- Includes modular research prompts and activity logging
- Handles time-limited research with error resilience

The new DeepResearch skill enables systematic and adaptive research across various topics.
This commit introduces a new content extraction feature with several key improvements:
- Added `extract` method to RAG service for flexible content retrieval
- Implemented HTML to markdown conversion and markdown cleaning utilities
- Updated skill service to support content extraction workflow
- Enhanced OpenAPI schema with extraction request and response types
- Improved content processing with topic-focused extraction and length limiting

The new extraction capabilities enable more precise and context-aware content retrieval across different sources.
This commit moves the research-related Zod schemas from the deep-research skill file to a new dedicated schema module, improving code organization and reusability. The changes include:

- Created a new `schema.ts` file in the deep-research module
- Moved existing research-related schemas (titleSchema, searchResultSchema, etc.) to the new file
- Updated imports in the deep-research skill to use the new schema module
- Maintained existing schema definitions and descriptions
…ling

This commit improves the deep research skill's robustness and tracking capabilities by:
- Adding a class-level research state property for better tracking
- Implementing more detailed activity logging for search and analysis stages
- Introducing advanced error handling with retry mechanisms
- Adding support for optional URL-specific search
- Enhancing source tracking and progress reporting
- Cleaning up research state after completion or failure
This commit fixes import paths for LangChain message types, replacing '@langchain/core/dist/messages' with '@langchain/core/messages' in several skill-related files. The changes ensure consistent and correct imports across the project.
@mrcfps mrcfps closed this May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants