Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/CD.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Deploy to GitHub Pages

on:
push:
branches: [main]

jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Use Node.js 20.x
uses: actions/setup-node@v4
with:
node-version: 20

- name: Install pnpm
run: npm install -g pnpm

- name: Install dependencies
run: pnpm install

- name: Build and export
run: |
pnpm build
pnpm run export

- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./out
43 changes: 43 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: CI

on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]

jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Use Node.js 20.x
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- name: Install pnpm
run: npm install -g pnpm

- name: Install dependencies
run: pnpm install

- name: Lint
run: pnpm lint

- name: Format check
run: pnpm format --check || true

- name: Build
run: pnpm build

- name: Install Playwright Browsers
run: npx playwright install --with-deps

- name: Run unit tests
run: pnpm test

- name: Run Playwright e2e tests
run: pnpm test:e2e
4 changes: 4 additions & 0 deletions .husky/pre-push
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

pnpm test
201 changes: 77 additions & 124 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,171 +5,124 @@
[![Ajv](https://img.shields.io/badge/Schema%20Validation-Ajv-orange?style=flat-square)](https://ajv.js.org/)
[![UI](https://img.shields.io/badge/UI-Shadcn/ui-black?style=flat-square)](https://ui.shadcn.com/)

A web-based tool to validate CSV data against JSON schemas directly in your browser. Edit your data on the fly and ensure compliance with predefined or custom structures.
A modern web-based tool to validate CSV data against JSON schemas, edit CSVs, and ensure compliance with custom or predefined structures. Built with Next.js 15+, TypeScript, and a robust UI/UX.

---

## ✨ Features

* **CSV Upload:** Easily upload your CSV files via drag-and-drop or file selection.
* **Schema Selection:** Choose from a list of predefined JSON schemas tailored for specific data types.
* **Custom Schema Upload:** Validate against your own JSON schema by uploading it.
* **In-Browser Validation:** Performs validation locally using Web Workers for a smooth UI experience.
* **Detailed Results:** View a clear list of validation errors and warnings, sorted with errors first.
* **Specific Value Highlighting:** Error messages pinpoint the exact value that caused the issue.
* **Live CSV Editing:** Modify your CSV data directly within the application using a code editor interface.
* **Save Modified CSV:** Download the edited CSV data back to your local machine.
* **Auto Re-validation:** Automatically re-validates the CSV data after saving changes.
- **CSV Upload & Editing:** Upload, edit, and validate CSV files in-browser with a code editor interface.
- **Schema Selection & Upload:** Choose from predefined JSON schemas or upload your own custom schema.
- **Live Validation:** Real-time validation using Ajv, with errors and warnings shown per row and value.
- **Detailed Results:** Errors and warnings are clearly displayed, with row numbers and schema paths.
- **Save & Download:** Download your edited CSV at any time.
- **Auto Re-validation:** CSV is re-validated after every save or edit.
- **Human-Readable Schema Docs:** View a Markdown-rendered, human-friendly version of the selected schema.
- **Responsive UI:** Panels resize and animate smoothly; results always visible.
- **Dark Mode:** Full support for light/dark themes, including logo adaptation.
- **E2E Testing:** Playwright-based end-to-end tests for reliability.
- **CI/CD:** Automated lint, test, build, and deploy via GitHub Actions.

---

## 🚀 Getting Started

### Prerequisites

* Node.js (v18 or later recommended)
* pnpm (or npm/yarn)
- Node.js (v18 or later recommended)
- pnpm (or npm/yarn)

### Installation & Running Locally

1. **Clone the repository:**
```bash
git clone https://github.com/FabienDostieIT/CSV_Data_Validator.git
cd CSV_Data_Validator
```
```bash
# Clone the repository
git clone https://github.com/FabienDostieIT/CSV_Data_Validator.git
cd CSV_Data_Validator

2. **Install dependencies:**
```bash
pnpm install
# or npm install / yarn install
```
# Install dependencies
pnpm install
# or npm install / yarn install

3. **Run the development server:**
```bash
pnpm dev
# or npm run dev / yarn dev
```
# Run the development server
pnpm dev
# or npm run dev / yarn dev

4. Open [http://localhost:3000](http://localhost:3000) in your browser.
# Open http://localhost:3000 in your browser
```

---

## 🛠️ How to Use

### 1. Upload CSV File

* Drag and drop your CSV file onto the designated area.
* Alternatively, click the upload area to open a file selection dialog.
* The raw CSV content will be displayed in the editor pane.

![Alt text](img/image.png "CSV Content")

### 2. Select or Upload Schema

* **Predefined Schema:** Choose a schema from the dropdown list (e.g., `productSchema`, `userSchema`). The validation will use this structure.
* **Custom Schema:**
* Click the "Upload Custom Schema" button.
* Select your JSON schema file (`.json`).
* The application will use your uploaded schema for validation.

![Alt text](img/image2.png "JSON Schema")

### 3. Validate Data

* Click the "Validate CSV" button.
* The validation process runs in the background.
* A loading indicator will show while processing.

### 4. Review Results

* Once validation is complete, the "Validation Results" section will appear below the editor.
* Results are categorized into **Errors** and **Warnings**.
* Errors are listed first, followed by warnings.
* Each result shows:
* Row Number (in the original CSV)
* Error/Warning Message (including the specific problematic value)
* Schema Path (`instancePath`) indicating the location of the issue within the data structure.
* Counts for total errors and warnings are displayed.

![Alt text](img/image3.png "Results section")
## 🛠️ Usage Overview

### 5. Edit CSV Data
1. **Upload CSV:** Drag-and-drop or select a CSV file. The content appears in the editor.
2. **Select/Upload Schema:** Choose a schema from the dropdown or upload your own JSON schema.
3. **Validate:** Click "Validate Data". Errors and warnings appear below, with details per row.
4. **Edit & Save:** Edit CSV directly, then save/download. Validation runs automatically after edits.
5. **View Schema Docs:** Toggle the schema panel to see a human-readable Markdown version of the schema.

* You can directly edit the text in the CSV editor pane on the left.
* Make corrections based on the validation results or perform other modifications.

![Alt text](img/image4.png "Optional title")
---

### 6. Save Modified CSV
## 🧪 Testing & Quality

- **Lint:**
```sh
pnpm lint
```
- **Format:**
```sh
pnpm format --check
```
- **Unit Tests:**
```sh
pnpm test
```
- **E2E Tests (Playwright):**
```sh
pnpm e2e
```

* Click the "Save CSV" button.
* This will trigger a download of the currently displayed CSV content as a `.csv` file.
---

### 7. Automatic Re-validation After Save
## ⚙️ CI/CD & Deployment

* After saving the CSV, the application automatically re-parses the data from the editor and runs the validation again with the selected schema.
* The results panel will update to reflect the validity of the *saved* data.
- **CI:** Automated via GitHub Actions (`.github/workflows/CI.yml`). Runs lint, format, unit tests, build, and Playwright e2e tests on every push/PR.
- **CD:** Static export and deploy to GitHub Pages via `.github/workflows/CD.yml`.
- **Static Export:**
- Next.js 15+ uses `output: 'export'` in `next.config.mjs`.
- Build with:
```sh
pnpm build
# Output is in the out/ directory
```
- **GitHub Pages:**
- Deployment is automatic from the `main` branch to the `gh-pages` branch.
- Configure GitHub Pages in repo settings to serve from `gh-pages` branch.

---

## 💻 Tech Stack

* **Frontend:** Next.js (React Framework), TypeScript
* **UI Components:** Shadcn/ui, Radix UI, Tailwind CSS
* **CSV Parsing:** PapaParse
* **Schema Validation:** Ajv (Another JSON Schema Validator)
* **Code Editor:** CodeMirror
* **State Management:** React Hooks (useState, useCallback, useRef)
* **Asynchronous Operations:** Web Workers



---

<div align="center">
✔️ Made with ❤️ by the fine people @ [F]it 👌
</div>
- **Frontend:** Next.js 15+, React 19, TypeScript
- **UI:** Shadcn/ui, Radix UI, Tailwind CSS
- **CSV Parsing:** PapaParse
- **Schema Validation:** Ajv
- **Code Editor:** Monaco/CodeMirror
- **E2E:** Playwright
- **CI/CD:** GitHub Actions

---

## 🤝 Contributing

Contributions are welcome! If you find a bug or have a feature request, please open an issue on GitHub.

If you'd like to contribute code:

1. Fork the repository.
2. Create a new branch for your feature or bug fix (`git checkout -b feature/your-feature` or `git checkout -b fix/your-bug`).
3. Make your changes and commit them with clear messages.
4. Push your branch to your fork (`git push origin feature/your-feature`).
5. Open a Pull Request back to the main repository's `develop` branch.

Please ensure your code adheres to the existing style and that any relevant tests pass (if applicable).
Contributions are welcome! Please open issues or pull requests. Ensure your code passes lint, format, and all tests before submitting.

---

## 📄 License

This project is licensed under the MIT License.
MIT License © 2025 Fabien Dostie, [F]it

**MIT License**

Copyright (c) 2025 Fabien Dostie, [F]it

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
---

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
## 🔒 Security

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
See [SECURITY.md](./SECURITY.md) for vulnerability reporting and supported versions.
43 changes: 24 additions & 19 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,33 +9,38 @@ We are committed to ensuring the security of the CSV Data Validator. Security up
| >= 1.0.0 (Latest `main`) | :white_check_mark: |
| < 1.0.0 | :x: |

*(**Note:** Based on your `package.json`, the version is `1.0.0`. This table assumes you only actively support the latest code. Adjust this table if you maintain older versions with security patches).*
> **Note:** Only the latest code on `main` is actively supported. Older versions do not receive security patches unless otherwise stated.

## Security Features & Practices

- **Automated Dependency Checks:** All dependencies are regularly updated and checked for vulnerabilities via GitHub Actions.
- **CI/CD Security:** All code changes are tested (unit, integration, E2E) and statically analyzed before deployment.
- **E2E Security Testing:** Playwright E2E tests help ensure that critical user flows are protected against regressions and common web vulnerabilities.
- **Static Export:** The app is deployed as a static site (Next.js `output: 'export'`), minimizing server-side attack surface.

## Reporting a Vulnerability

We take security vulnerabilities seriously. If you discover a security issue, please report it responsibly. **Please do not report security vulnerabilities through public GitHub issues.**
We take security vulnerabilities seriously. If you discover a security issue, please report it responsibly. **Do not report security vulnerabilities through public GitHub issues.**

Instead, please report vulnerabilities using one of the following methods:

1. **(Recommended) GitHub Security Advisories:** You can privately report a vulnerability directly through GitHub by navigating to the "Security" tab of the repository (`https://github.com/FabienDostieIT/CSV_Data_Validator/security/advisories/new`). This is the preferred method as it allows for secure communication and tracking.
2. **(Alternative) Email:** You can email the report directly to `[Your Security Contact Email Address - e.g., [email protected] or your personal email if appropriate]`. Please use a clear subject line like "Security Vulnerability Report: CSV Data Validator".

**What to Include in Your Report:**
1. **(Recommended) GitHub Security Advisories:** Privately report a vulnerability via the "Security" tab of the repository ([link](https://github.com/FabienDostieIT/CSV_Data_Validator/security/advisories/new)).
2. **(Alternative) Email:** Email the report to `[email protected]` (replace with your actual contact email). Use a clear subject like "Security Vulnerability Report: CSV Data Validator".

Please include as much detail as possible to help us understand and reproduce the issue:
### What to Include in Your Report

* Type of vulnerability (e.g., Cross-Site Scripting, dependency vulnerability, improper validation).
* Steps to reproduce the vulnerability.
* Any proof-of-concept code or examples.
* Potential impact of the vulnerability.
* Any suggested mitigation, if known.
- Type of vulnerability (e.g., XSS, dependency, improper validation)
- Steps to reproduce
- Proof-of-concept code or examples
- Potential impact
- Suggested mitigation (if known)

**Our Process:**
### Our Process

1. **Acknowledgement:** We aim to acknowledge receipt of your vulnerability report within **[e.g., 2-3 business days]**.
2. **Assessment:** We will investigate the report to confirm the vulnerability and assess its severity and impact.
3. **Communication:** We will maintain communication with you regarding the status of the report and our findings.
4. **Resolution:** If the vulnerability is confirmed and requires action, we will work on a fix. Security patches will typically be applied to the `develop` branch first and then merged into the `main` branch for release.
5. **Disclosure:** Once a fix is available, we may coordinate disclosure with you. We typically aim to credit reporters who responsibly disclose vulnerabilities unless they prefer to remain anonymous.
1. **Acknowledgement:** We aim to acknowledge your report within 2-3 business days.
2. **Assessment:** We investigate and assess severity/impact.
3. **Communication:** We keep you updated on status and findings.
4. **Resolution:** If confirmed, we work on a fix and apply patches to `main`.
5. **Disclosure:** We may coordinate disclosure and credit you unless you prefer anonymity.

We appreciate your efforts in responsibly disclosing security vulnerabilities and helping us keep the CSV Data Validator secure.
We appreciate your efforts in responsibly disclosing vulnerabilities and helping us keep the CSV Data Validator secure.
Loading
Loading