-
Notifications
You must be signed in to change notification settings - Fork 0
Patterns and Conventions
chris48s edited this page Feb 12, 2020
·
2 revisions
This suite of projects started out life as a collection of scrapers running on morph.io. When we migrated them all away from morph, we inherited some of morph's design decisions (which might seem strange viewed in complete isolation).
- The entry point for bot/scraper repos is always a file called
scraper.py, even if its not really a scraper. (TODO: improve this https://github.com/DemocracyClub/dc-bots/issues/8) - Credentials are stored in env vars. Env vars for bots are always prefixed with
MORPH_ - All bots/scrapers have access to the same shared env vars. (TODO: improve this https://github.com/DemocracyClub/dc-bots/issues/10)
- All bots/scrapers run in a single shared virtualenv. This means they all need to have a compatible set of dependencies. (TODO: improve this https://github.com/DemocracyClub/dc-bots/issues/6)
- You should be able to run your scraper locally without sending slack notifications/opening GH issues/etc if no credentials are set. e.g: