Skip to content

Releases: gem/oq-engine

OpenQuake Engine 2.0

21 Jun 12:56
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Quoted the taxonomies in the CSV exports
  • Fixed a bug in classical_damage and added a master test for it
  • Fixed the escaping of the taxonomies in the datastore
  • Fixed the names of the exported risk files
  • Fixed a segfault in the WebUI when exporting files with h5py >= 2.4
  • Added a command oq dbserver to start/stop the database server
  • The engine exports the hazard curves one file per IMT
  • Exported lon and lat with 5 digits after the decimal point
  • Added a command oq info --build-reports
  • Introduced experimental support for exporting .hdf5 files

[Daniele Viganò (@daniviga)]

  • Reworked substantially the engine documentation: removed obsolete pages,
    updated to engine 2.0 and added instructions for Windows and Mac OS X
  • Remove oq_create_db script, db is created by the DbServer
  • Move oq_reset_db into utils and clean old code

[Michele Simionato (@micheles)]

  • Now the DbServer automatically upgrades the database if needed
  • Renamed oq-lite -> oq and added a subcommand oq engine
  • Added a CSV reader for the hazard curves
  • Having time_event=None in the hazard part of a calculation is now valid
  • Added an exporter for the rupture data, including the occurrence rate
  • Refactored the CSV exporters
  • Moved celeryconfig.py; now celery must be started with
    celery worker --config openquake.engine.celeryconfig
  • Added a default location ~/oqdata/dbserver.log for the DbServer log
  • Added an early check on the SA periods supported by the GSIMs
  • Now the gsim_logic_tree file is parsed only once
  • Added a document about the architecture of the engine
  • The realizations are now exported as a CSV file
  • Escaped taxonomies in the datastore
  • The Web UI log tool is now escaping the HTML
  • Moved openquake.commonlib.commands -> openquake.commands and
    openquake.commonlib.valid -> openquake.risklib.valid to have a
    linear tower of internal dependencies
  • Supported all versions of Django >= 1.5
  • Provided a better error message in the absence of openquake.cfg
  • Removed the check on the export_dir when using the WebUI
  • Reduce the data transfer of the realization association object
  • If uniform_hazard_spectra is true, the UHS curves are generated
    even if hazard_maps is false; the hazard maps are not exported
  • Optimized the filtering of PointSources
  • Initial work on the UCERF event based hazard calculator
  • Added a test calculation crossing the International Date Line (Alaska)

[Daniele Viganò (@daniviga)]

  • Remove the dependency from the python 'pwd' package which is not
    available on Windows
  • Supervisord init scripts are now provided for the dbserver, celery
    and the webui. Celery is not started by default, other two are.

[Michele Simionato (@micheles)]

  • Another export fix: made sure it is run by the current user
  • Fixed the export: if the export directory does not exists, it is created
  • Introduced the configuration variable multi_user, false for source
    installations and true for package installations
  • Fixed the WebUI export
  • Removed the .txt outputs from the WebUI page engine/<output_id>/outputs
    (they are useful only internally)
  • Fixed the export: first the xml exporter is tried and then the csv exporter;
    if both are available, only the first is used, not both of them
  • Optimized the case when the epsilons are not required, i.e. all the
    covariance coefficients are zero in the vulnerability functions
  • Added another test for event based risk (case_miriam)
  • Revisited the distribution mechanism and refined the weight of the
    ruptures in the event based calculators to avoid generating slow tasks
  • Added an automatic help for the subcommands of oq-lite and managed
    --version correctly
  • The event based risk calculator now use different seeds for different
    realizations; also, the performance has been substantially improved
  • Improved the .rst reports with data transfer information
  • Removed the RlzsAssoc object from the datastore
  • Fixed the number of tasks generated by the risk calculators
  • Refactored the serialization of CompositionInfo instances to HDF5
  • Used exponential notation with 5 decimal digits in most exported XML files
  • Refactored the sampling mechanics in the event based calculators
  • The event_based_risk calculator infers the minimum intensity of the GMFs
    from the vulnerability functions (if not specified in the job.ini)
  • Fixed the avg_losses-stats: they were not generated in absence of
    loss curves
  • Added a command oq-lite info --exports
  • Added filtering on the mininum intensity also in the event based
    hazard calculator; improved the performance and memory occupation
  • Added a view displaying the calculation times by source typology
  • Fixed the test of GMPETable after the correction in hazardlib
  • Optimized the saving of the asset loss table
  • Optimized the case of multiple assets of the same taxonomy on the
    same point and introduced a datastore view assets_by_site
  • Fixed HDF5 segmentation faults in the tests for Ubuntu 16.04

[Daniele Viganò (@daniviga)]

  • Add support for Ubuntu 16.04 (xenial) packages
  • Removed the openquake_worker.cfg file because it is not used anymore

[Michele Simionato (@micheles)]

  • Replaced PostgreSQL with SQLite
  • Introduced a dbserver to mediate the interaction with the database
  • Restored the signal handler to manage properly kill signals so that
    the workers are revoked when a process is killed manually
  • Fixed in a more robust way the duplicated log bug
  • Made more robust the killing of processes by patching concurrent.futures
  • Fixed a critical bug with celery not being used even when use_celery
    was true.
  • Improved the validation of NRML files
  • Added a command oq-engine --show-log <job_id>

[Daniele Viganò (@daniviga)]

  • Use the 'postgresql' meta package as dependency of the .deb
    package to support newer versions of Postgres; this makes
    Trusty package installable on Ubuntu 16.04 and Debian 8

[Daniele Viganò (@daniviga), Michele Simionato (@micheles)]

  • Fixed a bug in oq-engine --export-outputs

[Daniele Viganò (@daniviga), Matteo Nastasi (@nastasi-oq)]

  • Allow installation of the binary package on Ubuntu derivatives

[Matteo Nastasi (@nastasi-oq)]

  • Backport of libhdf5 and h5py for ubuntu 'precise' serie

[Michele Simionato (@micheles)]

  • Removed openquake/engine/settings.py
  • Made the dependency on celery required only in cluster installations
  • Integrated the authentication database in the engine server database
  • Fixed the description in the Web UI (before it was temporarily set to
    the string "A job").
  • Introduced filtering on the minimum intensity of the ground shaking
  • Solved the issue of serializing large SES collections, over the HDF5 limit
  • The loss maps and curves XML exporters now export the coordinates
    of the assets, not the coordinates of the closest hazard site
  • Stored the job.ini parameters into a table in the datastore
  • Added a check on the IMTs coming from the risk models
  • Changed the aggregate loss table exporter to export the event tags,
    not the event IDs
  • Fixed a bug with the CSV export of the ground motion fields
  • Fixed a bug with the export of UHS curves with --exports=xml
  • Reduced substantially the data transfer and the memory occupation
    for event based calculations with a large number of assets: we
    can run the California exposure with half million assets now
  • Fixed a bug in the SESCollection exporter
  • Changed the asset<->epsilons association: before for a given taxonomy the
    assets were ordered by asset_ref, now they are ordered by id. This
    has a minor impact on the numbers sensitive to the epsilons, akin to a
    change of seeds
  • Added a test on the ordering of the epsilons
  • Accepted . and | as valid characters for source IDs
  • Changed the GMF calculator to use a single seed per unique rupture
  • Changed the SESCollection exporter: now a single file is exported, before
    we were exporting one file per source model path per tectonic region model
  • Changed the event based calculators to avoid duplicating ruptures
    occurring more than once
  • Changed the risk calculators to work in blocks of assets on the same site
  • Made it possible to set different integration distances for different
    tectonic region types
  • Optimized the aggregation by asset in the event based risk calculator
  • Reporting the source_id when the filtering fails

OpenQuake Engine 1.9.1

07 Mar 10:22
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Fixed a bug in the Web UI when running a risk calculation starting
    from a previous calculation

OpenQuake Engine 1.9

02 Mar 13:45
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Fixed a bug such that in some circumstances the logging stream handler
    was instantiated twice, resulting in duplicated logs
  • Changed the default job status to 'executing' (was 'pre_executing')
  • Fixed the ordering of the logs in the Web UI
  • Removed the dependency from PostGIS
  • Restored the monitoring which was accidentally removed
  • Removed the obsolete option --hazard-output-id
  • Printed the names of the files exported by the engine, even when there
    are multiple files for a single output
  • Introduced four new tables job, output, log, performance: all the other
    60+ database tables are not used anymore

OpenQuake Engine 1.8

15 Feb 13:16
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Removed two oq-engine switches (--export-stats and --list-inputs)
    and fixed --show-view; unified --delete-hazard-calculation and
    --delete-risk-calculation into a single --delete-calculation
  • Updated make_html_report.py to extract the full report from the
    datastore
  • If use_celery is true, use celery to determine a good default for
    the parameter concurrent_tasks
  • Made celery required only in cluster situations
  • Fixed the duplication of exported result in the classical_damage
    calculator when there is more than one realization
  • Removed several obsolete or deprecated switches from the oq-engine command
  • Replaced all classical calculators with their lite counterparts
  • Fixed the site-ordering in the UHS exporter (by lon-lat)

[Paolo Tormene (@ptormene)]

  • Added API to validate NRML

[Michele Simionato (@micheles)]

  • The engine can now zip files larger than 2 GB (used in the export)
  • Now the loss maps and curves are exported with a fixed ordering: first
    by lon-lat, then by asset ID
  • Replaced the old disaggregation calculator with the oq-lite one

OpenQuake Engine 1.7

14 Dec 10:00
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Fixed an encoding bug in --lhc
  • Fixed an export bug: it is now possible to export the outputs generated
    by another user, if the read permissions are set correctly

OpenQuake Engine 1.6.0

17 Nov 12:41
Compare
Choose a tag to compare

[Daniele Viganò (@daniviga)]

  • Added the oq_reset_db script. It removes and recreates the database and
    the datastore

[Matteo Nastasi (@nastasi-oq)]

  • Demos moved to /usr/share/openquake/risklib

[Michele Simionato (@micheles)]

  • Removed the 'view' button from the Web UI
  • Removed the epsilon_sampling configuration parameter
  • Made customizable the display_name of datastore outputs (before it was
    identical to the datastore key)
  • The zip files generated for internal use of the Web UI are now hidden
  • Made visible to the engine only the exportable outputs of the datastore
  • Closed explicitly the datastore after each calculation
  • Replaced the old scenario calculators with the HDF5-based calculators
  • Fixed a very subtle bug in the association queries: some sites outside
    of the region constraint were not discarded in some situations
  • Removed the self-termination feature terminate_job_when_celery_is_down
  • Removed the epsilon sampling "feature" from the scenario_risk calculator
  • Replaced the event based calculators based on Postgres with the new ones
    based on the HDF5 technology

OpenQuake Engine 1.5.1

25 Sep 12:42
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Fixed a bug affecting exposures with multiple assets on the same site

OpenQuake Engine 1.5.0

23 Sep 14:10
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • The event based calculators in the engine are now officially deprecated
    and they raise a warning when used
  • Optimization: we do not generate the full epsilon matrix if all
    coefficients of variation are zero
  • Fixed two subtle bugs in the management of the epsilons: it means that
    all event based risk calculations with nonzero coefficients of variations
    will produce slightly different numbers with respect to before
  • Removed excessive checking on the exposure attributes 'deductible' and
    'insuredLimit' that made it impossible to run legitimate calculations
  • Changed the meaning of 'average_loss' for the aggregated curves: now it
    is the sum of the aggregated losses in the event loss table,
    before it was extracted from the aggregated loss curve
  • Changed the way the average losses (and insured average losses) are
    computed by the event based risk calculator: now they are extracted
    from the event loss table, before they were extracted from the loss curves
  • Set to NULL the stddev_losses and stddev_insured_losses for the event based
    risk calculator, since they were computed incorrectly
  • Introduced a new experimental command
    'oq-engine --show-view CALCULATION_ID VIEW_NAME'; the only view available
    for the moment is 'mean_avg_losses'
  • Negative calculation IDs are interpreted in a Pythonic way, i.e. -1
    means the last calculation, -2 the calculation before the last one, etc.
  • If a site parameter is more distant than 5 kilometers from its closest
    site, a warning is logged
  • Changed the splitting of fault sources to reduce the number of generated
    sources and avoid data transfer failures if rupture_mesh_spacing is too
    small
  • Changed the event loss table export: now the CSV file does not contain
    the magnitude and the rows are ordered by rupture tag first and loss second
  • Removed the calculator EventBasedBCR
  • Longitude and latitude are now rounded to 5 digits
  • Fixed a very subtle bug in the vulnerability functions, potentially
    affecting calculations with nonzero coefficients of variation and nonzero
    minIML; the numbers produced by the engine were incorrect; see
    https://bugs.launchpad.net/oq-engine/+bug/1459926
  • 'investigation_time' has been replaced by 'risk_investigation_time' in
    risk configuration files
  • Initial support for Django 1.7

[Daniele Viganò (@daniviga)]

  • Removed the bin/openquake wrapper: now only bin/oq-engine is
    available

[Michele Simionato (@micheles)]

  • Added parameter parallel_source_splitting in openquake.cfg

[Daniele Viganò (@daniviga)]

  • setup.py improvements
  • Added MANIFEST.in
  • celeryconfig.py moved from /usr/openquake/engine to
    /usr/share/openquake/engine

[Matteo Nastasi (@nastasi-oq)]

  • Packaging system improvement

OpenQuake Engine 1.4.0

07 May 10:32
Compare
Choose a tag to compare

[Daniele Viganò (@daniviga)]

  • Fixed debian/control: add missing lsb-release to build deps

[Matteo Nastasi (@nastasi-oq), Daniele Viganò (@daniviga)]

  • Fixed dependencies version management

[Matteo Nastasi (@nastasi-oq), Daniele Viganò (@daniviga)]

  • Add binary package support for both Ubuntu 12.04 (Precise)
    and Ubuntu 14.04 (Trusty)

[Michele Simionato (@micheles)]

  • Removed the SiteModel table: now the association between the sites and the
    site model is done by using hazardlib.geo.geodetic.min_distance

[Daniele Viganò (@daniviga)]

  • added authentication support to the 'engineweb' and the 'engineserver'

[Michele Simionato (@micheles)]

  • the aggregate loss curves can be exported in CSV format

[Matteo Nastasi (@nastasi-oq)]

  • added 'outtypes' attribute with list of possible output types for
    each output item in outputs list API command
  • added '/v1/calc//status' API command
  • added 'engineweb' django application as local web client for oq-engine

[Michele Simionato (@micheles)]

  • Renamed the maximum_distance parameter of the risk calculators to
    asset_hazard_distance, to avoid confusion with the maximum_distance
    parameter of the hazard calculators, which has a different meaning;
    is it an error to set the maximum_distance in a job_risk.ini file
  • Added to the API an URL /v1/calc/:calc_id/remove to hide jobs
  • A new key is_running is added to the list of dictionaries returned by
    the URL /v1/calc/list
  • Replaced the mock tests for the engine server with real functional tests
  • Added a resource /v1/calc/:calc_id/traceback to get the traceback of a
    failed calculation
  • Now the logs are stored also in the database, both for the controller node
    and the worker nodes
  • Bypassed Django when deleting calculations from the database: this avoids
    running out of memory for large calculations
  • Fixed an issue in the scenario calculator: the GMFs were not filtered
    according to the distance to the rupture
  • Now critical errors appear in the log file
  • Added a --run command to run hazard and risk together
  • Fixed bug in the event based calculator; in the case
    number_of_logic_tree_samples > 0 it was generating incorrect hazard curves.
    Also improved (a lot) the performance in this case.
  • Fixed a tricky bug happening when some tectonic region type are filtered
    away.
  • The event based risk calculator now save only the non-zero losses in
    the table event_loss_asset.
  • Added a CSV exporter for the Stochastic Event Sets, for debugging purposes.
  • The GMF CSV exporter now sorts the output by rupture tag.

[Matteo Nastasi (@nastasi-oq)]

  • Each pull request must be accompanied by an update of the debian
    changelog now.

OpenQuake Engine 1.3.1

20 Mar 09:34
Compare
Choose a tag to compare

[Michele Simionato (@micheles)]

  • Fixed a tricky bug happening when some tectonic region types are filtered
    away.