Skip to content

Fix crash on schema invalid files #568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
"request": "launch",
"skipFiles": ["<node_internals>/**"],
"type": "pwa-node"
"type": "node"
},
{
"name": "Attach to Node Functions",
Expand Down
34 changes: 15 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
Summary
=======

Product | Validator API Endpoints
--- | ---
Description | Node.JS app that provides API end points to validate IATI XML files, is used by https://validator.iatistandard.org/
Website | [https://developer.iatistandard.org/](https://developer.iatistandard.org/)
Related | [IATI/validator-services](https://github.com/IATI/validator-services), [IATI/validator-web](https://github.com/IATI/validator-web)
Documentation | [https://developer.iatistandard.org/](https://developer.iatistandard.org/)
Technical Issues | https://github.com/IATI/js-validator-api/issues
Support | https://iatistandard.org/en/guidance/get-support/
# Summary

| Product | Validator API Endpoints |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| Description | Node.JS app that provides API end points to validate IATI XML files, is used by https://validator.iatistandard.org/ |
| Website | [https://developer.iatistandard.org/](https://developer.iatistandard.org/) |
| Related | [IATI/validator-services](https://github.com/IATI/validator-services), [IATI/validator-web](https://github.com/IATI/validator-web) |
| Documentation | [https://developer.iatistandard.org/](https://developer.iatistandard.org/) |
| Technical Issues | https://github.com/IATI/js-validator-api/issues |
| Support | https://iatistandard.org/en/guidance/get-support/ |

# IATI JavaScript Validator API

Expand All @@ -29,15 +27,15 @@ See OpenAPI specification `postman/schemas/index.yaml`. To view locally in Swagg
- [Docker Engine](https://docs.docker.com/engine/install/#server)
- [Docker Compose](https://docs.docker.com/compose/install/)
- `xmllint`
- There are two ways to run this locally: directly, or using the `docker compose` setup. The docker build includes the `xmllint` tool. If you run it directly, you may need to install this tool.
- On Ubuntu it is in `libxml2-utils`, so you'll need something like `sudo apt install libxml2-utils`.
- There are two ways to run this locally: directly, or using the `docker compose` setup. The docker build includes the `xmllint` tool. If you run it directly, you may need to install this tool.
- On Ubuntu it is in `libxml2-utils`, so you'll need something like `sudo apt install libxml2-utils`.

## Getting Started

1. Clone this repository
1. Follow instructions for nvm/node prerequisties above
1. Setup the environment variables using the `.env` file (instructions below)
* If running directly (option 1), this will mean setting up a `redis` database.
- If running directly (option 1), this will mean setting up a `redis` database.

### Option 1: running directly

Expand All @@ -50,7 +48,6 @@ See OpenAPI specification `postman/schemas/index.yaml`. To view locally in Swagg
4. Run `npm run docker:start` to run the Function inside a Docker container using docker-compose.yml to create a Redis instance.
1. The API end points will be available on `localhost:8080`


### Test setup

To test the basic setup, you can run the following command:
Expand Down Expand Up @@ -83,7 +80,6 @@ You should see something like (if the file is a valid IATI file):

If it has warnings or errors, you'll see them listed.


## Environment Variables

### Set Up
Expand Down Expand Up @@ -117,8 +113,8 @@ DATASTORE_SERVICES_AUTH_HTTP_HEADER_NAME=x-functions-key
DATASTORE_SERVICES_AUTH_HTTP_HEADER_VALUE=
DATASTORE_SERVICES_IATI_IDENTIFIERS_EXIST_MAX_NUMBER_OF_IDS=5000

- URL and API Key for datastore services, used by the advisory system to check for the
existence of IATI Identifiers in the Datastore
- URL and API Key for datastore services, used by the advisory system to check for the
existence of IATI Identifiers in the Datastore

### App config defaults (set in `config/config.js`)

Expand Down Expand Up @@ -181,7 +177,7 @@ Locally

- Install newman globally `npm i -g newman`
- Start function `npm start`
- Run Tests `npm int:test`
- Run Tests `npm run int:test`

In Docker container

Expand Down
77 changes: 77 additions & 0 deletions integration-tests/js-validator-api-tests.postman_collection.json
Original file line number Diff line number Diff line change
Expand Up @@ -2915,6 +2915,83 @@
}
},
"response": []
},
{
"name": "iati-org-schema-error-broken-root-tag",
"event": [
{
"listen": "test",
"script": {
"exec": [
"pm.test(\"Status code is 422\", function () { pm.response.to.have.status(422); });",
"const jsonData = pm.response.json();",
"pm.test(\"The file is invalid\", function () {",
" pm.expect(jsonData.valid).to.eql(false);",
"});",
"pm.test(\"fileType value is iati-organisations\", function () {",
" pm.expect(jsonData.fileType).to.eql('iati-organisations');",
"});",
"pm.test(\"version is 2.02\", function () {",
" pm.expect(jsonData.iatiVersion).to.eql('2.02');",
"});",
"pm.test(\"Two schema errors in first group\", function () {",
" pm.expect(jsonData.errors[0].errors[0].errors.length).to.eql(2);",
"});",
"pm.test(\"Error ID should be 0.3.1\", function () {",
" pm.expect(jsonData.errors[0].errors[0].errors[0].id).to.eql(\"0.3.1\");",
"});",
""
],
"type": "text/javascript",
"packages": {}
}
},
{
"listen": "prerequest",
"script": {
"exec": [
""
],
"type": "text/javascript",
"packages": {}
}
}
],
"protocolProfileBehavior": {
"followRedirects": false,
"followOriginalHttpMethod": false,
"followAuthorizationHeader": false
},
"request": {
"method": "POST",
"header": [
{
"key": "Accept",
"value": "application/json"
},
{
"key": "Content-Type",
"value": "application/xml"
}
],
"body": {
"mode": "file",
"file": {
"src": "/home/s/dev/js-validator-api/integration-tests/test-files/iati-org-schema-errors-broken-root-element.xml"
}
},
"url": {
"raw": "{{baseURL}}/pub/validate",
"host": [
"{{baseURL}}"
],
"path": [
"pub",
"validate"
]
}
},
"response": []
}
],
"event": [
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<iati-organisations generated-datetime="2025-04-30T11:08:44" version="2.02">
<iati-organisation default-currency="USD" last-updated-datetime="2025-04-30T11:08:44"
xml:lang="en">
<organisation-identifier>TEST-ORG-010</organisation-identifier>
<name>
<narrative>Office for Government Policy Coordination</narrative>
</name>
<reporting-org ref="TEST-ORG-010" type="10" secondary-reporter="0">
<narrative>Office for Government Policy Coordination</narrative>
</reporting-org>
</iati-organisation>
<iati-organisations generated-datetime="2025-05-08T10:30:11" version="2.02">
<iati-organisation default-currency="USD" last-updated-datetime="2025-05-08T00:00:00"
xml:lang="en">
<organisation-identifier>TEST-ORG-051</organisation-identifier>
<name>
<narrative>State Government International Cooperation Agency</narrative>
</name>
<reporting-org ref="TEST-ORG-010" type="10" secondary-reporter="1">
<narrative>Office for Government Policy Coordination</narrative>
</reporting-org>
</iati-organisation>
</iati-organisations>
<iati-organisation default-currency="USD" last-updated-datetime="2025-04-30T11:08:44"
xml:lang="en">
<organisation-identifier>TEST-ORG-060</organisation-identifier>
<name>
<narrative>Office of Ministry of Justice</narrative>
</name>
<reporting-org ref="TEST-ORG-010" type="10" secondary-reporter="1">
<narrative>Office for Government Policy Coordination</narrative>
</reporting-org>
</iati-organisation>
</iati-organisations>
Loading
Loading