Skip to content

Adding new documentation files (English) #1079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lucpretti wants to merge 1 commit into from
Closed

Adding new documentation files (English) #1079

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions docs/source/documentation/contributing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Contributing {#contributing}

## Contribution Guidelines {#contribution-guidelines}

You don’t need to know how to code to contribute to Open Data Editor, you just need time and attitude. There are many possible ways to contribute to the project:

* Use the Open Data Editor and give us feedback
* Spread the word about it\!
* Improve the documentation
* Add a translation
* Report issues
* Contribute to the code

Please read this guide for more details on the contribution process.

### How can you help? {#how-can-you-help?}

#### Giving us feedback {#giving-us-feedback}

Use the Open Data Editor and give us feedback. You can try out all the features and report any issues you might encounter. You can also suggest improvements to the Open Data Editor’s look and layout. For example, *if something feels confusing or hard to find in the tool’s layout, you can suggest changes to make it easier to use.*

Feedback from a beginner’s perspective is particularly welcome as it helps us improve usability.

To give us feedback, you can either [open an issue in the GitHub repository](https://github.com/okfn/opendataeditor) or reach out by email at [email protected].

#### Spread the word {#spread-the-word}

If you like the application, tell others about it\! Spreading the word about the Open Data Editor is also a valuable contribution to the project. Whether you share it on social media, write a blog post, create a tutorial video, or simply tell a friend, you’re helping more people discover and use the tool. Every bit of awareness and engagement makes a difference\!

#### Documentation {#documentation-1}

##### Improving documentation {#improving-documentation}

Documentation is always the best place to start contributing. If you spot a typo or you’ve ever thought, “I wish the documentation explained this more clearly,” you can help make it easier to understand. You can either flag that, or propose a modification. You can also add missing steps or explanations.

##### Adding examples {#adding-examples}

You can also contribute by adding examples\! If you notice something is not covered in the documentation, try creating a helpful example and sharing it. You can also provide real-world use cases to show how Open Data Editor can be used. Every improvement makes Open Data Editor easier for everyone to use\!

All documentation files are contained [in this folder](https://github.com/okfn/opendataeditor/tree/main/portal/content/docs/documentation). You can just propose a change or add examples by opening a PR. Please **always include a short description of what you changed and why**, so it is easier to review. If you don’t feel comfortable using GitHub, you can also send us an email with the proposed changes at info\[at\]okfn.org.

##### Use cases {#use-cases-1}

If you have started using Open Data Editor in your daily work and would like to share with the world how the tool is helping you, you can always write a blog about it. You can find a few examples [here](https://blog.okfn.org/tag/ode-use-cases/).

If possible, try to include photos and screenshots. If you need guidance, you can always reach out to us at [email protected]. You can send us your blog at that same email address.

#### Reporting a bug {#reporting-a-bug}

We use GitHub as a code and issues hosting platform. To report a bug or propose a new feature, please open an issue [in the repository](https://github.com/okfn/opendataeditor/issues). Please provide a detailed description of the bug and include screenshots. There are a few predefined issue templates to help you get started.

If you don’t feel comfortable using GitHub, you can also send us an email at [email protected].

#### Code contributions {#code-contributions}

##### Pull requests {#pull-requests}

First, please check the [issue tracker](https://github.com/okfn/opendataeditor/issues). Look for issues with “help wanted” or “good first issue.” **If you want to submit a PR, there needs to be a corresponding issue in the issue tracker.** Please link the issue in your PR, and provide a brief explanation of the changes you are pushing.

### What is the review process for your contribution? {#what-is-the-review-process-for-your-contribution?}

Your contribution will be **reviewed by the Open Data Editor core team at Open Knowledge Foundation**.

In case of documentation and code contributions, a member of the core team will check if it is relevant, helpful, and easy to understand. If small changes are needed, they suggest edits. Once everything looks good, the contribution is approved and added to the project, and the contributor is acknowledged (because every contribution matters\!).

In case of bugs and feedback contributions, the core team will discuss if and how they can be implemented. We can include you in the discussion if you wish. Once we have a clear pathway, we will notify you.

### Do you need help? {#do-you-need-help?}

If at any point you need help, feel free to contact us via email at [[email protected]](mailto:[email protected]).
105 changes: 105 additions & 0 deletions docs/source/documentation/introduction.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Introduction {#introduction}

## What is Open Data Editor {#what-is-open-data-editor}

[Open Data Editor (ODE)](https://okfn.org/opendataeditor), developed by the [Open Knowledge Foundation (OKFN)](https://okfn.org/en/), is a free, open-source tool designed to help nonprofits, data journalists, activists, and public servants detect errors in their datasets. It is designed for people working with tabular data (Excel, Google Sheets, CSV) who don't know how to code or don't have the programming skills to automatise the data exploration process.

The technical mission of ODE is to provide a free, open-source, no-code, cross-platform desktop application that empowers non-technical users working with tabular data to quickly detect and correct errors, enforce data-validation and metadata standards (notably the [FAIR principles](https://www.go-fair.org/fair-principles/)), and output clean, interoperable datasets ready for publication – while preserving privacy (local-first architecture), remaining lightweight (suitable for low-resource or offline contexts), and employing open standards (e.g., the [Frictionless framework](https://framework.frictionlessdata.io/)) for maximum reuse and integration.

Since 2025, the Digital Public Goods Alliance (DPGA) has recognised Open Data Editor as a [digital public good](https://blog.okfn.org/2025/10/22/open-data-editor-recognised-as-a-digital-public-good/) (DPG), which means it meets high standards of openness and supports sustainable development globally.

## FAIR data {#fair-data}

The Open Data Editor (ODE) improves data quality based on the [FAIR principles](https://www.go-fair.org/fair-principles/).

As described by the [GO FAIR initiative](https://www.go-fair.org/), FAIR data refers to data that adheres to the **Findable**, **Accessible**, **Interoperable**, and **Reusable** principles, designed to make research data more discoverable and usable by people and machines, enhancing its value and reuse.

**Findable**
The first step in (re)using data is to find it. Metadata and data should be easy to find for both humans and computers.

**Accessible**
Once the user finds the required data, they need to know how it can be accessed, possibly including authentication and authorisation.

**Interoperable**
The data usually need to be integrated with other data. In addition, the data needs to interoperate with applications or workflows for analysis, storage, and processing.

**Reusable**
The ultimate goal of FAIR is to optimise the reuse of data. To achieve this, metadata and data should be well-described so that they can be replicated and/or combined in different settings.

Learn all about the FAIR principles in [Module 4](https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/lessons/introduction-6/) of the ‘Quality and Consistent Data with ODE’ course, available on [School of Data](https://schoolofdata.org/).

## Responsible AI integration {#responsible-ai-integration}

The Open Data Editor (ODE) has an AI component to help users better understand their data. The AI component is powered by local models. **No data is sent to the cloud, and all operations take place on the user’s computer.**

The introduction of these AI-assisted data quality features significantly expands the tool’s capabilities, enabling faster identification of anomalies, smarter suggestions for corrections, and improved handling of large, dense datasets while retaining a privacy-preserving, local-first design.

This deepens Open Knowledge Foundation's commitment to developing simple, sustainable and long-lasting technologies that solve people’s real problems, in line with [The Tech We Want](https://okfn.org/en/projects/the-tech-we-want/) initiative.

In order to use the AI feature, ODE will guide users to download the model onto their machine first.

:::{note}
As the model is local and all computing will be happening in the user’s computer, performance will be affected by the machine’s hardware, and the response quality sometimes will be minor than typical cloud LLMs like ChatGPT, Deepseek or Claude.
:::

ODE is trying to balance user experience for non-technical audiences and performance and privacy.

## Free data literacy course {#free-data-literacy-course}

Open Data Editor's focus on improving digital literacy and preparing non-technical users to work with data, led us to develop a free course available on School of Data. [‘Quality and Consistent Data with the Open Data Editor’](https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/) is an essential open educational resource for anyone who wants to generate knowledge from data. The course is now available in English and Portuguese, and will soon be translated into Spanish and French.

It is specially designed for non-technical users working with tabular data (Excel, Google Sheets, CSV) but without advanced technical knowledge. It's 100% online, free, and gamified.

You can check the contents and enrol here: [https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/](https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/)

## Similar tools and differentiators {#similar-tools-and-differentiators}

The tools currently available with functions similar to those of the Open Data Editor were created for a specific purpose and have a more technical profile. This makes them difficult for people unfamiliar with code, standards or programming languages to access.

The main differences in relation to ODE are listed in each subsection below:

### Data Check {#data-check}

Available at: [https://data.humdata.org/tools/datacheck/import](https://data.humdata.org/tools/datacheck/import)

Main differences:

* Maximum file size: 20 MB.
* Works only with the HXL standard.
* The table view after the error check is limited, and the user needs to navigate through several tabs if the file has many lines.
* Does not include a publication feature.

### IATI Validator {#iati-validator}

Available at: [https://validator.iatistandard.org/](https://validator.iatistandard.org/)

Main differences:

* Works only with the IATI standard.
* Targets a specific sector, the international aid community.
* The tool offers five levels of qualification regarding data quality (Success, Success with Advisories, Warning, Error, and Critical), rather than a list of all errors and how to correct them.

### CSV Lint.io {#csv-lint.io}

Available at: [https://csvlint.io/](https://csvlint.io/)

Main differences:

* Works only with CSV files, informing the user if the file “is readable” or not.
* Agnostic tool; the schema can also be ingested.

### 360Giving Data Quality Checker {#360giving-data-quality-checker}

Available at: [https://dataquality.threesixtygiving.org/](https://dataquality.threesixtygiving.org/)

Main differences:

* Works only with the 360Giving standard.

## Acknowledgements {#acknowledgements}

We are grateful for the support and partnership of the [Patrick J. McGovern Foundation (PJMF)](https://www.mcgovern.org/), without which the development of the Open Data Editor would not have been possible. Learn more about its funding programmes [here](https://www.mcgovern.org/grants/).

## Latest updates {#latest-updates}

Open Data Editor is being built in the open. Follow the progress, from feature updates to community stories on the Open Knowledge Blog: [https://blog.okfn.org/category/open-data-editor/](https://blog.okfn.org/category/open-data-editor/)
91 changes: 91 additions & 0 deletions docs/source/documentation/technical-documentation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Technical Documentation / Development {#technical-documentation-/-development}

## Prerequisites {#prerequisites}

We are using 3.13. To start working on the project, you need the following dependencies on your machine:

* Python 3.13
* python3.13-dev (For PyInstaller)

We are using [uv](https://docs.astral.sh/uv/) as a package manager, so make sure you have it installed.

## Environment {#environment}

Use `uv` to create a virtualenv and activate it:

```bash
uv sync
source venv/bin/activate
```

## Start the application {#start-the-application}

```bash
uv run ode
```

or

```bash
# With the virtual environment activated
python src/ode/main.py
```

## Running tests {#running-tests}

```bash
uv run pytest tests/
```

or

```bash
# With the virtual environment activated
pytest tests/
```

## Building the application {#building-the-application}

```bash
uv run build.py build
```

or

```bash
# With the virtual environment activated
python build.py build
```

This will create a distributable file for the application in the ‘dist/’ folder.

## Documentation {#documentation}

Documentation is written with [Sphinx](https://www.sphinx-doc.org/en/master/) (in the `docs` directory). The source files are in the `docs/source/` directory. To locally build the documentation, you can execute:

```bash
uv run build.py docs
```

or

```bash
# With the virtual environment activated
python build.py docs
```

It will be automatically published on CloudFlare when merged to the `main`branch, with previews available for pull requests.

## Making a release {#making-a-release}

To make a release, follow the following checklist:

* Check with the Product Owner that the `main` branch is code complete.
* Check that the distributables built on `main` are working by installing them on your machine.
* Sometimes PyInstaller cannot compile new dependencies, and the application will fail at runtime.
* Create a new PR bumping the version of the application in the `pyproject.toml` file and merge it to main.
* Create a New Github Release with a new tag matching the new version number of the application.
* Fill in the Release notes.
* Create the Release.
* Wait until the GitHub Action for the new tag finishes, and then upload the distributable files to the new Release.
* Notify the Communications Team to make the announcement and changes to the [OKFN’s Website](https://okfn.org/opendataeditor/).
35 changes: 35 additions & 0 deletions docs/source/documentation/translations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Translations {#translations}

ODE supports several languages following the [Qt Framework practices for](https://doc.qt.io/qt-6/internationalization.html) internationalisation.

## Internationalisation workflow {#internationalisation-workflow}

:::{note}
ODE provides a `python build.py update-translations` and `python build.py compile-translations` command for all the supported languages. Check the project’s `build.py` file for reference.
:::

1. Create or update translation files by running `python build.py update-translations` (or the `pyside6-lupdate` command directly if you are on macOS)
2. Update the translation files:
1. Complete `unfinished` translations (this is the actual addition of translated text).
2. Clean `vanished` translations.
3. Compile the translation files by running `python build.py compile-translations` (or the `pyside6-lrelease` command directly if you are on macOS)
4. Commit both the `.ts` and `.qm` files.
5. Create a PR with the changes.

All translation files are located in the `ode/assets/translations/` folder.

## Translation Tools {#translation-tools}

For updating translations, you can use either:

1. A text editor to directly update the translation files (\*.ts)
2. Install Qt and use [Qt Linguist](https://doc.qt.io/qt-6/qtlinguist-index.html) application to do the translation using a UI.

## Adding new languages {#adding-new-languages}

When adding a new language, two extra changes are required:

1. Add the new language to both `python build.py update-translations` and `python build.py compile-translations` commands of the `build.py` file. (after doing this, you can run the `python build.py update-translations` and it will create the `.ts` file for you.)
2. Update the `language` QComboBox of the `main.py` file so the new language appears as an option to the user.

Here is a reference Pull Request of what is expected when adding a new language: [https://github.com/okfn/opendataeditor/pull/750](https://github.com/okfn/opendataeditor/pull/750)
Loading