Feature: Remove web scraped content

[wladimiiir]

Currently, /web command scrapes the website and puts it into the message. User should be able to remove this content from the chat. I can see two approaches here:

1. Show webscraped data as message that will have action to remove it from the chat.
2. Web scrape to a temporary file and add the file to the context, so it can be removed (similar to how images are handled)

[Sigfrodr]

2 is better imo, but why temporary? For example if I scrape https://svelte.dev/llms-small.txt, I would want it to be in a .web folder in my app to be able to add/remove it from the context when needed (docs are huge, difficult to keep then in the context forever). No need to scrape it each time.

[wladimiiir]

Great idea with storing the scrape result. I would not do it implicitly though as it would create files in your project that you would not be able to gitignore (otherwise Aider would not be able to add them to the context) and it could get annoying. Instead I propose the following:

1. /web [url] will scrape the page and store as temporary file (similar to how images are handled)
2. /web [url] [path] will scrape the page and store it at [path], so you will be able to reuse it when needed.

In your case you would use /web https://svelte.dev/llms-small.txt .web/svelte.md, which would store it inside .web folder.

[Sigfrodr]

Even better would be to manage that in settings: automatic save for scraping (checkbox) + default folder (from project root).
Maybe also an UI to be able to list all *.md files from project and click on whatever we want to be on the context or not (but that's an other idea/ticket, a tree to see project files for easy context management would be nice too)