Skip to content

feat: add Keenable as a configurable internet search backend - #2072

Open
ilya-bogin-keenable wants to merge 20 commits into
MemTensor:mainfrom
keenableai:feat/keenable-web-search
Open

feat: add Keenable as a configurable internet search backend#2072
ilya-bogin-keenable wants to merge 20 commits into
MemTensor:mainfrom
keenableai:feat/keenable-web-search

Conversation

@ilya-bogin-keenable

Copy link
Copy Markdown

Summary

Adds Keenable as a new internet search backend alongside the existing Bocha / Tavily / Google / Bing / Xinyu retrievers, following the Tavily backend pattern (#1357). Additive and opt-in via INTERNET_SEARCH_BACKEND=keenable; existing backends are untouched.

Keenable is a web search API built for AI agents. Unlike the key-required backends it is keyless by default: with no key it calls the public endpoint (rate-limited), and an optional KEENABLE_API_KEY only lifts the cap.

Files changed

  • src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (new): InternetKeenableRetriever. No SDK dependency, a thin requests call. Keyless requests hit /v1/search/public; a configured key switches to /v1/search with an X-API-Key header. Attribution via X-Keenable-Title. Results map into TextualMemoryItem the same way the Tavily retriever does.
  • src/memos/configs/internet_retriever.py: KeenableSearchConfig (API key optional) + registration in InternetRetrieverConfigFactory.
  • src/memos/memories/textual/tree_text_memory/retrieve/internet_retriever_factory.py: register keenable + constructor branch.
  • src/memos/api/config.py: INTERNET_SEARCH_BACKEND=keenable branch (KEENABLE_API_KEY optional).

Testing

  • python -m py_compile on all changed files: passes.
  • ruff check on all changed files: passes.

Bryunyon and others added 2 commits July 7, 2026 16:16
Add Keenable alongside the existing bocha / tavily / google / bing / xinyu
internet retrievers, following the Tavily backend pattern.

- retrieve/keenablesearch.py: InternetKeenableRetriever. Keyless by default
  (no SDK, a thin requests call): with no key it hits /v1/search/public
  (rate-limited); a key switches to /v1/search with an X-API-Key header.
  Attribution via X-Keenable-Title. Results map into TextualMemoryItem
  exactly like the Tavily retriever.
- configs/internet_retriever.py: KeenableSearchConfig (api_key optional) and
  registration in InternetRetrieverConfigFactory.
- retrieve/internet_retriever_factory.py: register "keenable" + constructor.
- api/config.py: INTERNET_SEARCH_BACKEND=keenable branch (KEENABLE_API_KEY
  optional, keyless by default).

py_compile and ruff pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Memtensor-AI Memtensor-AI added area:memory 记忆存储、检索、更新、召回逻辑 area:api 云服务 / FastAPI / OpenAPI / MCP labels Jul 8, 2026
@Memtensor-AI
Memtensor-AI requested a review from bittergreen July 8, 2026 11:43
@ilya-bogin-keenable

Copy link
Copy Markdown
Author

Hey @bittergreen could you please take a look? Thanks!

@Memtensor-AI Memtensor-AI removed the area:api 云服务 / FastAPI / OpenAPI / MCP label Jul 13, 2026
@Memtensor-AI Memtensor-AI added area:api 云服务 / FastAPI / OpenAPI / MCP area:core MOS 编排层 / 框架底座 / 跨模块问题 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 31, 2026
@Memtensor-AI
Memtensor-AI requested a review from WeiminLee July 31, 2026 19:09
@Memtensor-AI

Memtensor-AI commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2072
Task: aa46054cf1f71848
Base: main
Head: feat/keenable-web-search

🔍 OpenCodeReview found 7 issue(s) in this PR.

⚠️ 1 warning(s) occurred during review.


1. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L268)

[Blocking] parsed_goal tags silently dropped — dead branch in _extract_tags.

_process_result receives parsed_goal but never forwards it to _extract_tags. The method has a parsed_goal=None parameter and a branch if parsed_goal and hasattr(parsed_goal, 'tags'): tags.extend(parsed_goal.tags) that is therefore unreachable in practice — goal-derived tags are silently dropped on every call.

💡 Suggested Change

Before:

            else self._extract_tags(title, content, summary)[:3]

After:

            else self._extract_tags(title, content, summary, parsed_goal)[:3]

2. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L240-L242)

Dead parameter: query is declared but never used inside _process_result.

The query argument is received and passed from _convert_to_mem_items, but it is never read within the function body. Either remove it from the signature and the executor.submit call, or use it (e.g. attach it to the metadata for provenance).


3. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L234-L235)

Traceback discarded on future failure.

str(e) alone is logged, dropping the full stack trace. If _process_result raises, there will be no indication of where or why it failed.

Suggestion: use logger.exception(...) or pass exc_info=True to preserve the traceback.

💡 Suggested Change

Before:

                except Exception as e:
                    logger.error(f"Error processing Keenable search result: {e}")

After:

                except Exception:
                    logger.exception("Error processing Keenable search result")

4. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L254-L255)

Overly broad exception on datetime.fromisoformat.

fromisoformat raises only ValueError on a malformed string. Catching Exception here also silently swallows programming errors such as AttributeError (if publish_time is not a string after the .replace() call). Narrow the catch to ValueError.

💡 Suggested Change

Before:

            except Exception:
                publish_time = datetime.now().strftime("%Y-%m-%d")

After:

            except ValueError:
                publish_time = datetime.now().strftime("%Y-%m-%d")

5. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L297)

Potential IndexError if embedder.embed returns an empty list.

The if content guard only rules out falsy strings; it does not protect against the embedder returning an empty list (e.g. on an internal error it swallows). [][0] will raise IndexError at runtime.

Suggestion: capture the result and guard the index access.

💡 Suggested Change

Before:

                    embedding=self.embedder.embed([content])[0] if content else [],

After:

                    embedding=(lambda emb: emb[0] if emb else [])(self.embedder.embed([content])) if content else [],

6. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L198)

Eager f-string formatting on logger.error call.

The f-string is formatted unconditionally even if the ERROR level is disabled. Prefer lazy %-style formatting so the string is only constructed when the level is actually enabled.

💡 Suggested Change

Before:

            logger.error(f"Keenable search error: {traceback.format_exc()}")

After:

            logger.error("Keenable search error: %s", traceback.format_exc())

7. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L195-L198)

import traceback inside an except block.

While Python caches imports so this is not incorrect, placing an import inside an exception handler is unconventional and obscures the module's actual dependencies. Move import traceback to the top-level imports.

💡 Suggested Change

Before:

        except Exception:
            import traceback

            logger.error(f"Keenable search error: {traceback.format_exc()}")

After:

# At the top of the file, alongside other stdlib imports:
import traceback

# Then in the except block:
        except Exception:
            logger.error("Keenable search error: %s", traceback.format_exc())

🧹 Filtered 7 low-confidence OCR finding(s) before posting/fix-loop (duplicate: 7).

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-33586be3c2b3f89c-20260801031509: 107/107 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 31, 2026
- gate jieba behind require_python_package, like the bocha retriever, so an
  English-only install no longer fails at construction
- skip embedding when the content is empty
- lowercase GDP/AI in the keyword lists; the text is lowercased before matching
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 1, 2026
@Memtensor-AI
Memtensor-AI requested a review from WeiminLee August 1, 2026 05:56
@ilya-bogin-keenable

Copy link
Copy Markdown
Author

Аixed 4 of the 5: jieba is now gated behind require_python_package like the bocha retriever (an English-only install used to fail at construction), empty content is no longer embedded, and GDP/AI are lowercased so they can actually match.

Will not fix for #2. Forwarding the local mode to the API would break the default path: the search endpoint returns 400 for mode: "fast", and fast is the default the caller passes. The API's mode and this method's mode are different things, so I added a comment saying so instead.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-b3c50e54d556e024-20260804132254: 86/87 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added the status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 label Aug 4, 2026
@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 18, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 19, 2026
Keenable returns both fields on every result: snippet carries the page text
and description is the page's meta description, which is empty for most
pages. Reading description alone stored memories with a title and a URL but
no text.
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-d929796b8f2bd83f-20260819141815: 136/136 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git clone --depth 1 --branch feat/keenable-web-search git@github.com:keenableai/MemOS.git /data/test-workspaces/0bb2eb3c8566c1ed/repo
Cloning into '/data/test-workspaces/0bb2eb3c8566c1ed/repo'...
Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git clone --depth 1 --branch feat/keenable-web-search git@github.com:keenableai/MemOS.git /data/test-workspaces/1a3c8d0e9d69e77a/repo
Cloning into '/data/test-workspaces/1a3c8d0e9d69e77a/repo'...
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 5s [advisory, non-gating] AI-generated tests on branch test/auto-gen-6b2de16b7c284ae6-20260828124956: 123/125 passed, 2 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 28, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 29, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-ffe1c10a605ed5dd-20260829140808: 106/107 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 29, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Sep 1, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-aa46054cf1f71848-20260901114424: 90/92 passed, 2 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:api 云服务 / FastAPI / OpenAPI / MCP area:core MOS 编排层 / 框架底座 / 跨模块问题 area:memory 记忆存储、检索、更新、召回逻辑 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.