Skip to content

Comments

[DRAFT] Chunker#2260

Draft
atroyn wants to merge 1 commit intomainfrom
05-29-anton_chunker
Draft

[DRAFT] Chunker#2260
atroyn wants to merge 1 commit intomainfrom
05-29-anton_chunker

Conversation

@atroyn
Copy link
Contributor

@atroyn atroyn commented May 30, 2024 •
edited
Loading

Description of changes

This PR adds the type, and a default implementation for built-in chunkers for Chroma. The default text chunker is a riff on LangChain's RecursiveTextSplitter but refactored a bit for our purposes.

The API surface is designed to resemble the EmbeddingFunction implementation, with the exception that they are not a property of Collections.

Test plan

TBC

Documentation Changes

TBC

TODO

  • Tests
  • Docs

@vercel
Copy link

vercel bot commented May 30, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
chroma ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 6, 2024 4:20am

@github-actions
Copy link

github-actions bot commented May 30, 2024

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@github-actions
Copy link

github-actions bot commented May 30, 2024

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

@atroyn Graphite App
Copy link
Contributor Author

atroyn commented May 30, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @atroyn and the rest of your teammates on Graphite Graphite

@atroyn atroyn mentioned this pull request May 30, 2024
Draft
2 tasks
@atroyn atroyn changed the title anton/chunker [Draft] Chunker May 30, 2024
@atroyn atroyn changed the title [Draft] Chunker [DRAFT] Chunker May 30, 2024
@atroyn atroyn marked this pull request as ready for review May 30, 2024 04:09
@atroyn atroyn marked this pull request as draft May 30, 2024 04:26
@atroyn atroyn force-pushed the 05-29-anton_chunker branch from d2b2d7f to 719c37c Compare May 31, 2024 19:56
@github-actions
Copy link

github-actions bot commented May 31, 2024

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@atroyn atroyn force-pushed the 05-29-anton_chunker branch from 719c37c to 773385a Compare June 3, 2024 20:45
@github-actions
Copy link

github-actions bot commented Jun 3, 2024

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@atroyn atroyn force-pushed the 05-29-anton_chunker branch from 773385a to d673865 Compare June 4, 2024 18:34
@github-actions
Copy link

github-actions bot commented Jun 4, 2024

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@atroyn atroyn linked an issue Jun 4, 2024 that may be closed by this pull request
[New Feature][Ease of Use] Smart Chunker #2281
Open
@atroyn atroyn force-pushed the 05-29-anton_chunker branch from d673865 to 205c06c Compare June 5, 2024 18:49
@github-actions
Copy link

github-actions bot commented Jun 5, 2024

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

@atroyn atroyn force-pushed the 05-29-anton_chunker branch from 205c06c to 514cf4d Compare June 6, 2024 04:17
@github-actions
Copy link

github-actions bot commented Jun 6, 2024

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[New Feature][Ease of Use] Smart Chunker

1 participant

@atroyn