Skip to content

Move the RapidPro ingestion to Polars#32

Merged
TuscanyBothaReach merged 11 commits into
mainfrom
move_rapidpro_to_polars
Sep 25, 2025
Merged

Move the RapidPro ingestion to Polars#32
TuscanyBothaReach merged 11 commits into
mainfrom
move_rapidpro_to_polars

Conversation

@TuscanyBothaReach

@TuscanyBothaReach TuscanyBothaReach commented Sep 23, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

  • Modified RapidPro pagination
  • Updated the RapidPro ingestion to use Polars
  • Added tests for concatenation and pagination functions

Noting potential future changes

  • Consider persisting a cursor to handle the very large send volumes instead of reducing the max_update size
  • Consider replacing the try/except logic to handle the final payload without catching an error

@TuscanyBothaReach
TuscanyBothaReach merged commit 2aa2b5a into main Sep 25, 2025
4 checks passed
@TuscanyBothaReach
TuscanyBothaReach deleted the move_rapidpro_to_polars branch September 25, 2025 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HawkiesZA HawkiesZA HawkiesZA approved these changes
@jerith jerith Awaiting requested review from jerith

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@TuscanyBothaReach @HawkiesZA