Releases: gem/oq-engine
OpenQuake Engine 2.0
[Michele Simionato (@micheles)]
- Quoted the taxonomies in the CSV exports
- Fixed a bug in classical_damage and added a master test for it
- Fixed the escaping of the taxonomies in the datastore
- Fixed the names of the exported risk files
- Fixed a segfault in the WebUI when exporting files with h5py >= 2.4
- Added a command
oq dbserver
to start/stop the database server - The engine exports the hazard curves one file per IMT
- Exported lon and lat with 5 digits after the decimal point
- Added a command
oq info --build-reports
- Introduced experimental support for exporting .hdf5 files
[Daniele Viganò (@daniviga)]
- Reworked substantially the engine documentation: removed obsolete pages,
updated to engine 2.0 and added instructions for Windows and Mac OS X - Remove oq_create_db script, db is created by the DbServer
- Move oq_reset_db into utils and clean old code
[Michele Simionato (@micheles)]
- Now the DbServer automatically upgrades the database if needed
- Renamed oq-lite -> oq and added a subcommand
oq engine
- Added a CSV reader for the hazard curves
- Having time_event=None in the hazard part of a calculation is now valid
- Added an exporter for the rupture data, including the occurrence rate
- Refactored the CSV exporters
- Moved celeryconfig.py; now celery must be started with
celery worker --config openquake.engine.celeryconfig
- Added a default location
~/oqdata/dbserver.log
for the DbServer log - Added an early check on the SA periods supported by the GSIMs
- Now the gsim_logic_tree file is parsed only once
- Added a document about the architecture of the engine
- The realizations are now exported as a CSV file
- Escaped taxonomies in the datastore
- The Web UI log tool is now escaping the HTML
- Moved openquake.commonlib.commands -> openquake.commands and
openquake.commonlib.valid -> openquake.risklib.valid to have a
linear tower of internal dependencies - Supported all versions of Django >= 1.5
- Provided a better error message in the absence of openquake.cfg
- Removed the check on the export_dir when using the WebUI
- Reduce the data transfer of the realization association object
- If uniform_hazard_spectra is true, the UHS curves are generated
even if hazard_maps is false; the hazard maps are not exported - Optimized the filtering of PointSources
- Initial work on the UCERF event based hazard calculator
- Added a test calculation crossing the International Date Line (Alaska)
[Daniele Viganò (@daniviga)]
- Remove the dependency from the python 'pwd' package which is not
available on Windows - Supervisord init scripts are now provided for the dbserver, celery
and the webui. Celery is not started by default, other two are.
[Michele Simionato (@micheles)]
- Another export fix: made sure it is run by the current user
- Fixed the export: if the export directory does not exists, it is created
- Introduced the configuration variable
multi_user
, false for source
installations and true for package installations - Fixed the WebUI export
- Removed the .txt outputs from the WebUI page engine/<output_id>/outputs
(they are useful only internally) - Fixed the export: first the xml exporter is tried and then the csv exporter;
if both are available, only the first is used, not both of them - Optimized the case when the epsilons are not required, i.e. all the
covariance coefficients are zero in the vulnerability functions - Added another test for event based risk (
case_miriam
) - Revisited the distribution mechanism and refined the weight of the
ruptures in the event based calculators to avoid generating slow tasks - Added an automatic help for the subcommands of oq-lite and managed
--version correctly - The event based risk calculator now use different seeds for different
realizations; also, the performance has been substantially improved - Improved the .rst reports with data transfer information
- Removed the RlzsAssoc object from the datastore
- Fixed the number of tasks generated by the risk calculators
- Refactored the serialization of CompositionInfo instances to HDF5
- Used exponential notation with 5 decimal digits in most exported XML files
- Refactored the sampling mechanics in the event based calculators
- The event_based_risk calculator infers the minimum intensity of the GMFs
from the vulnerability functions (if not specified in the job.ini) - Fixed the
avg_losses-stats
: they were not generated in absence of
loss curves - Added a command
oq-lite info --exports
- Added filtering on the mininum intensity also in the event based
hazard calculator; improved the performance and memory occupation - Added a view displaying the calculation times by source typology
- Fixed the test of GMPETable after the correction in hazardlib
- Optimized the saving of the asset loss table
- Optimized the case of multiple assets of the same taxonomy on the
same point and introduced a datastore viewassets_by_site
- Fixed HDF5 segmentation faults in the tests for Ubuntu 16.04
[Daniele Viganò (@daniviga)]
- Add support for Ubuntu 16.04 (xenial) packages
- Removed the openquake_worker.cfg file because it is not used anymore
[Michele Simionato (@micheles)]
- Replaced PostgreSQL with SQLite
- Introduced a dbserver to mediate the interaction with the database
- Restored the signal handler to manage properly
kill
signals so that
the workers are revoked when a process is killed manually - Fixed in a more robust way the duplicated log bug
- Made more robust the killing of processes by patching concurrent.futures
- Fixed a critical bug with celery not being used even when
use_celery
was true. - Improved the validation of NRML files
- Added a command
oq-engine --show-log <job_id>
[Daniele Viganò (@daniviga)]
- Use the 'postgresql' meta package as dependency of the .deb
package to support newer versions of Postgres; this makes
Trusty package installable on Ubuntu 16.04 and Debian 8
[Daniele Viganò (@daniviga), Michele Simionato (@micheles)]
- Fixed a bug in
oq-engine --export-outputs
[Daniele Viganò (@daniviga), Matteo Nastasi (@nastasi-oq)]
- Allow installation of the binary package on Ubuntu derivatives
[Matteo Nastasi (@nastasi-oq)]
- Backport of libhdf5 and h5py for ubuntu 'precise' serie
[Michele Simionato (@micheles)]
- Removed openquake/engine/settings.py
- Made the dependency on celery required only in cluster installations
- Integrated the authentication database in the engine server database
- Fixed the description in the Web UI (before it was temporarily set to
the string "A job"). - Introduced filtering on the minimum intensity of the ground shaking
- Solved the issue of serializing large SES collections, over the HDF5 limit
- The loss maps and curves XML exporters now export the coordinates
of the assets, not the coordinates of the closest hazard site - Stored the job.ini parameters into a table in the datastore
- Added a check on the IMTs coming from the risk models
- Changed the aggregate loss table exporter to export the event tags,
not the event IDs - Fixed a bug with the CSV export of the ground motion fields
- Fixed a bug with the export of UHS curves with
--exports=xml
- Reduced substantially the data transfer and the memory occupation
for event based calculations with a large number of assets: we
can run the California exposure with half million assets now - Fixed a bug in the SESCollection exporter
- Changed the asset<->epsilons association: before for a given taxonomy the
assets were ordered byasset_ref
, now they are ordered byid
. This
has a minor impact on the numbers sensitive to the epsilons, akin to a
change of seeds - Added a test on the ordering of the epsilons
- Accepted
.
and|
as valid characters for source IDs - Changed the GMF calculator to use a single seed per unique rupture
- Changed the SESCollection exporter: now a single file is exported, before
we were exporting one file per source model path per tectonic region model - Changed the event based calculators to avoid duplicating ruptures
occurring more than once - Changed the risk calculators to work in blocks of assets on the same site
- Made it possible to set different integration distances for different
tectonic region types - Optimized the aggregation by asset in the event based risk calculator
- Reporting the source_id when the filtering fails
OpenQuake Engine 1.9.1
[Michele Simionato (@micheles)]
- Fixed a bug in the Web UI when running a risk calculation starting
from a previous calculation
OpenQuake Engine 1.9
[Michele Simionato (@micheles)]
- Fixed a bug such that in some circumstances the logging stream handler
was instantiated twice, resulting in duplicated logs - Changed the default job status to 'executing' (was 'pre_executing')
- Fixed the ordering of the logs in the Web UI
- Removed the dependency from PostGIS
- Restored the monitoring which was accidentally removed
- Removed the obsolete option
--hazard-output-id
- Printed the names of the files exported by the engine, even when there
are multiple files for a single output - Introduced four new tables job, output, log, performance: all the other
60+ database tables are not used anymore
OpenQuake Engine 1.8
[Michele Simionato (@micheles)]
- Removed two
oq-engine
switches (--export-stats
and--list-inputs
)
and fixed--show-view
; unified--delete-hazard-calculation
and
--delete-risk-calculation
into a single--delete-calculation
- Updated
make_html_report.py
to extract the full report from the
datastore - If
use_celery
is true, use celery to determine a good default for
the parameterconcurrent_tasks
- Made celery required only in cluster situations
- Fixed the duplication of exported result in the classical_damage
calculator when there is more than one realization - Removed several obsolete or deprecated switches from the
oq-engine
command - Replaced all classical calculators with their lite counterparts
- Fixed the site-ordering in the UHS exporter (by lon-lat)
[Paolo Tormene (@ptormene)]
- Added API to validate NRML
[Michele Simionato (@micheles)]
- The engine can now zip files larger than 2 GB (used in the export)
- Now the loss maps and curves are exported with a fixed ordering: first
by lon-lat, then by asset ID - Replaced the old disaggregation calculator with the oq-lite one
OpenQuake Engine 1.7
[Michele Simionato (@micheles)]
- Fixed an encoding bug in
--lhc
- Fixed an export bug: it is now possible to export the outputs generated
by another user, if the read permissions are set correctly
OpenQuake Engine 1.6.0
[Daniele Viganò (@daniviga)]
- Added the oq_reset_db script. It removes and recreates the database and
the datastore
[Matteo Nastasi (@nastasi-oq)]
- Demos moved to /usr/share/openquake/risklib
[Michele Simionato (@micheles)]
- Removed the 'view' button from the Web UI
- Removed the epsilon_sampling configuration parameter
- Made customizable the display_name of datastore outputs (before it was
identical to the datastore key) - The zip files generated for internal use of the Web UI are now hidden
- Made visible to the engine only the exportable outputs of the datastore
- Closed explicitly the datastore after each calculation
- Replaced the old scenario calculators with the HDF5-based calculators
- Fixed a very subtle bug in the association queries: some sites outside
of the region constraint were not discarded in some situations - Removed the self-termination feature
terminate_job_when_celery_is_down
- Removed the epsilon sampling "feature" from the scenario_risk calculator
- Replaced the event based calculators based on Postgres with the new ones
based on the HDF5 technology
OpenQuake Engine 1.5.1
[Michele Simionato (@micheles)]
- Fixed a bug affecting exposures with multiple assets on the same site
OpenQuake Engine 1.5.0
[Michele Simionato (@micheles)]
- The event based calculators in the engine are now officially deprecated
and they raise a warning when used - Optimization: we do not generate the full epsilon matrix if all
coefficients of variation are zero - Fixed two subtle bugs in the management of the epsilons: it means that
all event based risk calculations with nonzero coefficients of variations
will produce slightly different numbers with respect to before - Removed excessive checking on the exposure attributes 'deductible' and
'insuredLimit' that made it impossible to run legitimate calculations - Changed the meaning of 'average_loss' for the aggregated curves: now it
is the sum of the aggregated losses in the event loss table,
before it was extracted from the aggregated loss curve - Changed the way the average losses (and insured average losses) are
computed by the event based risk calculator: now they are extracted
from the event loss table, before they were extracted from the loss curves - Set to NULL the stddev_losses and stddev_insured_losses for the event based
risk calculator, since they were computed incorrectly - Introduced a new experimental command
'oq-engine --show-view CALCULATION_ID VIEW_NAME'; the only view available
for the moment is 'mean_avg_losses' - Negative calculation IDs are interpreted in a Pythonic way, i.e. -1
means the last calculation, -2 the calculation before the last one, etc. - If a site parameter is more distant than 5 kilometers from its closest
site, a warning is logged - Changed the splitting of fault sources to reduce the number of generated
sources and avoid data transfer failures if rupture_mesh_spacing is too
small - Changed the event loss table export: now the CSV file does not contain
the magnitude and the rows are ordered by rupture tag first and loss second - Removed the calculator EventBasedBCR
- Longitude and latitude are now rounded to 5 digits
- Fixed a very subtle bug in the vulnerability functions, potentially
affecting calculations with nonzero coefficients of variation and nonzero
minIML; the numbers produced by the engine were incorrect; see
https://bugs.launchpad.net/oq-engine/+bug/1459926 - 'investigation_time' has been replaced by 'risk_investigation_time' in
risk configuration files - Initial support for Django 1.7
[Daniele Viganò (@daniviga)]
- Removed the bin/openquake wrapper: now only bin/oq-engine is
available
[Michele Simionato (@micheles)]
- Added parameter parallel_source_splitting in openquake.cfg
[Daniele Viganò (@daniviga)]
- setup.py improvements
- Added MANIFEST.in
- celeryconfig.py moved from /usr/openquake/engine to
/usr/share/openquake/engine
[Matteo Nastasi (@nastasi-oq)]
- Packaging system improvement
OpenQuake Engine 1.4.0
[Daniele Viganò (@daniviga)]
- Fixed debian/control: add missing lsb-release to build deps
[Matteo Nastasi (@nastasi-oq), Daniele Viganò (@daniviga)]
- Fixed dependencies version management
[Matteo Nastasi (@nastasi-oq), Daniele Viganò (@daniviga)]
- Add binary package support for both Ubuntu 12.04 (Precise)
and Ubuntu 14.04 (Trusty)
[Michele Simionato (@micheles)]
- Removed the SiteModel table: now the association between the sites and the
site model is done by using hazardlib.geo.geodetic.min_distance
[Daniele Viganò (@daniviga)]
- added authentication support to the 'engineweb' and the 'engineserver'
[Michele Simionato (@micheles)]
- the aggregate loss curves can be exported in CSV format
[Matteo Nastasi (@nastasi-oq)]
- added 'outtypes' attribute with list of possible output types for
each output item in outputs list API command - added '/v1/calc//status' API command
- added 'engineweb' django application as local web client for oq-engine
[Michele Simionato (@micheles)]
- Renamed the maximum_distance parameter of the risk calculators to
asset_hazard_distance, to avoid confusion with the maximum_distance
parameter of the hazard calculators, which has a different meaning;
is it an error to set the maximum_distance in a job_risk.ini file - Added to the API an URL /v1/calc/:calc_id/remove to hide jobs
- A new key is_running is added to the list of dictionaries returned by
the URL /v1/calc/list - Replaced the mock tests for the engine server with real functional tests
- Added a resource /v1/calc/:calc_id/traceback to get the traceback of a
failed calculation - Now the logs are stored also in the database, both for the controller node
and the worker nodes - Bypassed Django when deleting calculations from the database: this avoids
running out of memory for large calculations - Fixed an issue in the scenario calculator: the GMFs were not filtered
according to the distance to the rupture - Now critical errors appear in the log file
- Added a --run command to run hazard and risk together
- Fixed bug in the event based calculator; in the case
number_of_logic_tree_samples > 0 it was generating incorrect hazard curves.
Also improved (a lot) the performance in this case. - Fixed a tricky bug happening when some tectonic region type are filtered
away. - The event based risk calculator now save only the non-zero losses in
the table event_loss_asset. - Added a CSV exporter for the Stochastic Event Sets, for debugging purposes.
- The GMF CSV exporter now sorts the output by rupture tag.
[Matteo Nastasi (@nastasi-oq)]
- Each pull request must be accompanied by an update of the debian
changelog now.
OpenQuake Engine 1.3.1
[Michele Simionato (@micheles)]
- Fixed a tricky bug happening when some tectonic region types are filtered
away.