Releases: epam/statgpt-sdmx-proxy
Releases · epam/statgpt-sdmx-proxy
0.3.1
0.3.0
Features
- Add a custom SDMX-JSON 2.0 data writer pipeline (
CustomSdmxJsonDataWriterEngineV2/SdmxJsonV2WriterOverridesand the flat/series writer overrides) that emitsrolesas a plural array and produces the correct value shape for non-codedTIME_PERIOD, working around sdmx-core's bean-model gaps without forking the library. - Add three opt-in fixtures that round-trip SDMX 3.0 metadata sdmx-core's information model cannot hold:
PRESERVE_METADATA_ATTRIBUTE_USAGES(structure) -- captures the rawmetadataAttributeUsagesarray before conversion and re-injects it onto the converted DSD.ANNOTATION_VALUE_TO_TEXT(structure) -- rewrites an annotation's non-localisedvalueintotextso the SDMX-JSON 2.0 reader preserves it (only whentext/textsis absent).PRESERVE_METADATA_ATTRIBUTES(data, JSON 2.0 -> JSON 2.0 only) -- captures and re-injects all four attribute buckets plusdimensionGroupAttributesfrom the upstream response.
Fixes
- Fix DSD structure conversion dropping
metadataAttributeUsages, the MSD reference, and annotation values on SDMX-JSON 2.0 structure responses (#79 / #81). - Fix SDMX-JSON 2.0 data conversion dropping MSD-derived metadata attribute usages and
dimensionGroupAttributes(#83 / #85). - Fix silent data corruption where three IMF WEO series were relocated under a different
INDICATORcode during SDMX-JSON 2.0 data conversion -- all input series are now preserved (#80). - Fix the SDMX-JSON 2.0 series-limit truncator returning more series than the requested
limitwhenseriesis a JSON object, including a path-collision bug where the truncator clobbered thedata.structures[*].attributes.seriesdefinitions array (design 028). - Fix occasional malformed response headers on the cache-hit fast path by writing security headers eagerly, avoiding a
HeaderWriterFilter/StreamingResponseBodyrace (spring-security#15510).
Chores
- Recover and re-enable the full E2E test suite; generalize the metadata-preservation and DSD-fidelity checks into
BaseRegistryTestSuiteas toggleable config blocks rather than registry-specific tests (#77, #78, #82, #86; design 029). - Add multi-registry structure fan-out, smoke, and API-key-driven review-environment E2E tests and supporting helpers (
ApiKeyProvider,ProxyConfigPusher). - Add the sdmx-core overrides catalog reference doc (design 022) and the cache pre-heat evaluation (design 021, rejected -- recorded for the decision trail).
- CI: correct the
slash-command-dispatchtest-applicationsmapping and update the build action.
Deployment Changes
- This version does not require any deployment changes.
0.2.0
Features
- Accept
allas an alias for*on the structure endpoint (#76) - Adapt e2e tests to dial workflows (#63)
- Add CONFIG_SERVER_FORCE_SEED flag to overwrite stored config on startup Fixes #50. (#51)
- Add limit emulation (#40) (#54)
- Re-introduce structure fan-out as a toggled feature (#74)
- Refactor exception hierarchy under a single base class (#71)
- rework limit emulation, BIS c[]-routing fix, shrink-filter cache (#56)
Fixes
- Agency routing failures return 500 instead of 400 (#34)
- BIS monthly time periods are not emitted in the standard YYYY-Mmm form expected for SDMX-JSON 2.0.0 ObservationalTimePeriod (#45)
- BIS registry returns no data when the key contains one * per dimension; only a single * is accepted for "match all" (#43)
- Docker step fixes (#46)
- Fix E2E tests (#62)
- IMF CSV data conversion (#39, #49) (#59)
- Only first c[TIME_PERIOD] filter is applied when multiple are provided (#44)
- Stop tripping circuit breaker on registry 4xx responses
Other
- Move from file repo to BIS Github packages (#23)
- Update gradle dependencies (#53)
- Update sdmx-core to 2.3.21 (#61)
- Update sdmx-core to 2.4.0; Pull sdmx-core from maven central (#67)
- add E2E tests sample (#66)
- update ai-dial-ci version (3.1.3 -> 3.2.0) (#35)
- update deploy review workflow (#64)
Deployment Changes
- This version does not require any deployment changes.
0.1.6
Fixes
- Stop tripping circuit breaker on registry 4xx responses
Deployment Changes
- This version does not require any deployment changes.
0.1.5
Features
- Add SDMX 3.0
limitparameter emulation for registries that do not natively support it -- the proxy uses availability probes (proportional-bisect search) to shrink the request key, then truncates the upstream response if the registry still returns more series than requested. Works across SDMX-JSON 2.0.0, SDMX-JSON 1.0.0, SDMX-ML, and CSV 2.0.0. - Add
CONFIG_SERVER_FORCE_SEEDflag to force the config server to overwrite the stored Dial Storage config from the bundledsdmx_registries_config.jsonon startup. Intended as an interim mechanism for pushing updated bundled configs into existing environments.DIAL_STORAGEsource only. - Add a third cache domain for limit-emulation shrunk-filter results, so repeat requests skip the availability-probe loop. Backed by Caffeine (local) or Redis (distributed) like the existing structure caches.
Fixes
- Fix IMF CSV data conversion bugs (missing dimensions, malformed observation rows on certain datasets).
- Fix only the first
c[TIME_PERIOD]filter being applied when multiple are provided. - Fix BIS-style filter routing so
c[]-wrapped parameters are normalized correctly for registries that do not accept thec[]form on data and availability endpoints. - Fix BIS registry returning no data when the key contains one
*per dimension -- only a single*is accepted as "match all". - Fix BIS monthly time periods not being emitted in the standard
YYYY-Mmmform expected for SDMX-JSON 2.0.0ObservationalTimePeriod. - Fix agency routing failures returning 500 instead of 400.
Chores
- Move sdmx-core (fusion v2.3.9) dependency resolution from local
lib-repo/JARs to BIS GitHub Packages. Thelib-repo/sdmxsource/directory is removed from the repo. - Update
ai-dial-cifrom 3.1.3 to 3.2.0. - Update Gradle dependencies.
- Docker build and CI workflow fixes.
Deployment Changes
- New env var
CONFIG_SERVER_FORCE_SEED(defaultfalse). Set totrueon the config server to force-overwrite the stored Dial Storage config from the bundled default on startup. Failure of a forced reseed is fatal (the pod crashloops). CombiningCONFIG_SERVER_FORCE_SEED=truewithCONFIG_SERVER_SOURCE_TYPE=FILESYSTEMis fatal at startup. - New cache TTL key
sdmxproxy.cache.ttl.limitEmulation(defaultduration: PT1H,jitter: PT10M). Controls TTL of the limit-emulation shrunk-filter cache. Has sensible defaults; override via Spring config if needed. No env var. - Build-time only:
GPR_USERNAMEandGPR_PASSWORDare now required (a GitHub PAT withread:packagesscope) to resolve sdmx-core (fusion v2.3.9) artefacts from BIS GitHub Packages. Runtime is unaffected -- the produced Docker image is self-contained.
0.1.4
Fixes
- Fix skip logic for dataset-level attributes in
CustomSdmxJsonDataReaderEngineV2-- attributes attached at the dataset level are now correctly skipped during streaming JSON conversion instead of being misread as observation attributes.
Deployment Changes
- No deployment changes required.
0.1.3
Features
- Auto-load registry config on startup -- when DIAL Storage is empty, Config Server seeds it from the bundled default configuration (#25)
Fixes
- Fix NPE when converting JSON data response with empty dataset (#27)
- Fix registry config option to replace empty key with
*for data queries (#28) - Fix data query with key returning 400 instead of expected response (#16)
Chores
- Change config server routes from
/statgpt/sdmx-proxy/api/v0/configto/statgpt/sdmx-proxy-config-server/api/v0/config - Add OpenTelemetry and logging configuration to Config Server
- Cleanup Gradle build scripts (#19)
Deployment Changes
- Config Server base path changed:
/statgpt/sdmx-proxy/api/v0/config->/statgpt/sdmx-proxy-config-server/api/v0/config.
0.1.2
Fixes
- Fix
unwrapFilterParametersset tofalsein BIS registry config
Deployment Changes
This version does not require any deployment changes.
0.1.1
0.1.0
Initial release of SDMX Proxy for the StatGPT platform.
A unified SDMX 3.0 REST API proxy that routes requests to multiple upstream SDMX registries,
translating between SDMX 2.1 and 3.0 protocols and converting between formats (JSON, XML, CSV).
Features
- SDMX 3.0 REST API facade with protocol translation (SDMX 2.1 <-> 3.0)
- Registry routing based on agencyID with hierarchical agency matching
- Format conversion between SDMX-JSON 2.0.0, SDMX-JSON 1.0.0, SDMX-ML 2.1, SDMX-ML 3.0, and CSV 2.0.0
- Response fixtures -- configurable patches to fix known defects in upstream registry responses
- Two-layer caching with Caffeine (local) or Redis (distributed)
- Resilience4j integration -- circuit breaker, retry with exponential backoff, per-registry rate limiting
- OpenTelemetry observability -- traces, metrics, logs with OTLP export
- Built-in configurations for BIS and IMF registries
- E2E test suite with pairwise combinatorics for registry integration validation
- Docker support
Fixes
- Fix BIS availability endpoint ignoring
c[]-wrapped filter parameters #11 - Fix jar missing in Docker images #12
Other
- Update permissions for Dependency Review workflow #9
Deployment Changes
Initial deployment. See README for setup instructions and environment variables.