Markdown minification for scrape_batch

[tomkelsey]

Hey,

Just wondering if the same nice tweak you made for scrape_as_markdown (using remark to strip e.g. base64 images) could be made for scrape_batch too in an effort to reduce token count when batch scrapes are performed?

Thanks!

[meirk-brd]

Hi @tomkelsey,

Thanks for reaching out!

Absolutely - we’ll apply the same tweak to scrape_as_markdown and publish an updated version of the NPM package as soon as it’s ready.

If you have any additional requirements or suggestions, feel free to share - we’re happy to incorporate them.

[meirk-brd]

Hi @tomkelsey ,

We have released v2.8.6 :
scrape_batch now returns minified markdown, consistent with scrape_as_markdown.

See: #112

I will now close this issue, if you have any additional requirements, please feel free to reach out!