Quick Reference

Logging

https://console.cloud.google.com/logs/viewer?project=vertnet-portal&minLogLevel=0&expandAll=false&resource=gae_app%2Fmodule_id%2Ftools-usagestats&dateRangeUnbound=forwardInTime&logName=projects%2Fvertnet-portal%2Flogs%2Fappengine.googleapis.com%252Frequest_log

Entities

https://console.cloud.google.com/datastore/entities;kind=Period;ns=__$DEFAULT$__/query/kind?project=vertnet-portal

Task Queue

https://console.cloud.google.com/cloudtasks/queue/usagestatsqueue?project=vertnet-portal

1) Update Dataset entity information by querying Carto registry in table resource_staging

https://github.com/VertNet/usagestats/blob/master/admin/setup/DatasetsSetup.py

curl -i -X POST -d "" http://tools-usagestats.vertnet-portal.appspot.com/admin/setup/datasets

2) Check that the GitHub repositories referenced in the table resource_staging actually exist

https://github.com/VertNet/usagestats/blob/master/admin/tools/RepoChecker.py

http://tools-usagestats.vertnet-portal.appspot.com/admin/tools/repo_checker

3) Initialize stats or run full stats process

Full processing run, storing on GitHub and sending notification issues

Note: Does not pass test of running to completion without errors. Only step-wise processing has been tested so far. Example:

curl -i -X POST -d "period=201204&github_store=true&github_issue=true" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/init

Default usage: Initialize a new period. No override, no testing, no GitHub processing

Example:

curl -i -X POST -d "period=201204" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/init

Re-intitializing an existing period

Example, forcing a period to be re-initialized:

curl -i -X POST -d "period=201204&force=true" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/init

4) Manual processing after Initialize step

If there has been a failure in GetEvents. Example:

curl -i -X POST -d "period=201204" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/get_events

If there has been a failure in ProcessEvent. Example:

curl -i -X POST -d "period=201204" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/process_events

To run the GitHub reports generation step. Example:

curl -i -X POST -d "period=201204" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/github_store

To run the GitHub issues generation step. Example:

curl -i -X POST -d "period=201204" http://tools-usagestats.vertnet-portal.appspot.com/admin/parser/github_issue