Releases: m-lab/etl
Allow leading ./ in filenames
ndt-staging-v0.4.1-a Update ndt.go
Sandbox - try pubsub with remote subscription
Local subscription doesn't get updates from remote topic.
From pubsub branch.
Try using local subscription with 10 minute deadline.
ndt-sandbox-v0.4-e Change to use a local subscription
Trial using pubsub in sandbox
Added 10 minute ack deadline.
Note that for 50MB tar files with typical distribution of around 40 tests per file, the processing takes under a minute. However, for very slow tests, the trace files are small, so there may be 100s of tests in a tar file, and it may take more like 5 minutes to process. We will need to monitor whether some test files are resubmitted, resulting in higher dup rates.
Also, note that AppEngine appears to do autoscaling only if there are HTTP requests passed through the load balancer. Since subscription method uses pulls directly from the pubsub system, AppEngine does not autoscale. We will need to use a different scaling mechanism.
tweak queue configs + quota retry
Tweaking queue configs, max_workers, for better instance regulation and 14 hour per day processing time.
Add quota retry
ndt-sandbox-v0.3.2 Fix time arithmetic
GeoLocation with Sidestream
ss-staging-v0.2 enable staging and prod.
GeoLocation with Sidestream
Merge pull request #263 from m-lab/pt-geo activate GeoLocation in staging and prod for PT
sidestream Geo
ss-sandbox-v0.1 enable Geo with SS.
Geo in Sidestream
ss-prod-v0.1 enable staging and prod.