Enhancement: Scaling with Celery

[cartfisk]

I saw that you mentioned looking for assistance with scaling. Using a distributed computing library to assign work to a group of "worker" nodes would be a nice way to implement this. I'm familiar with a Python library called Celery that does just that. The main question is how to set up a release distribution that enables individuals to spin up their own workers easily on whatever computing resources they have available.

Additionally, I imagine querying for and detecting image matches is the most computationally expensive task in this project. With this in mind, a distributed system will require a way to designate "chunks of work" to individual instances/nodes so that repeat queries are avoided where possible.

I'll look deeper into the code to see if this can be easily accomplished with the current setup.

[syntax]

I think ultimately the thing that prevents this from scaling well is instagrams rate limit, which puts comment placing cool-downs on accounts for a semi-considerable time, and there isn't really any ideal get arounds to this apart from having many instagram accounts.

[cartfisk]

Maybe the set up process for a distributed instance could configure the bot to post with the host’s own account or a throwaway?