Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
{
"name": "Search-a-licious Development",
"dockerComposeFile": [
"../docker-compose.yml",
"../docker/dev.yml"
],
"service": "api",
"workspaceFolder": "/opt/search",
"shutdownAction": "stopCompose",
"overrideCommand": true,

// Use docker-outside-of-docker feature to allow running docker commands inside the container
"features": {
"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
"moby": true,
"installDockerBuildx": true,
"version": "latest",
"dockerDashComposeVersion": "v2"
},
"ghcr.io/devcontainers/features/node:1": {
"version": "lts"
}
},

// Configure tool-specific properties
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"charliermarsh.ruff",
"ms-azuretools.vscode-docker",
"esbenp.prettier-vscode",
"dbaeumer.vscode-eslint"
],
"settings": {
"python.defaultInterpreterPath": "/opt/pysetup/.venv/bin/python",
"python.testing.pytestEnabled": true,
"python.testing.unittestEnabled": false,
"[python]": {
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
}
},
"[typescript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[javascript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"files.watcherExclude": {
"**/node_modules/**": true,
"**/.venv/**": true,
"**/data/**": true
}
}
}
},

// Forward ports for local development
"forwardPorts": [
8000,
9200,
8080
],

"portsAttributes": {
"8000": {
"label": "FastAPI Backend",
"onAutoForward": "notify"
},
"9200": {
"label": "Elasticsearch",
"onAutoForward": "silent"
},
"8080": {
"label": "ElasticVue",
"onAutoForward": "silent"
}
},

// Lifecycle scripts
"postCreateCommand": "pip install pre-commit && pre-commit install && cd frontend && npm install",
"postStartCommand": "echo 'Dev container ready! All services running. Use make commands for development.'",

// Set environment variables
"remoteEnv": {
"ELASTICSEARCH_URL": "http://es01:9200",
"CONFIG_PATH": "/opt/search/data/config/openfoodfacts.yml",
"USER_UID": "${localEnv:USER_UID}",
"USER_GID": "${localEnv:USER_GID}"
},

// Mount the workspace
"mounts": [
"source=${localWorkspaceFolder},target=/opt/search,type=bind,consistency=cached"
],

// Run as the same user that runs the container
"remoteUser": "off",

// Specify which services to start
"runServices": ["es01", "es02", "elasticvue", "api", "search_nodejs"]
}
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,16 @@ Ready to use it ? Jump to the [documentation](https://openfoodfacts.github.io/se

This is an Open Source project and [contributions are very welcome](https://openfoodfacts.github.io/search-a-licious/#contributing) !

## 🚀 Quick Start for Developers

Want to contribute? Get started in minutes with our [Developer Installation Guide](https://openfoodfacts.github.io/search-a-licious/devs/how-to-install/)

## 🎨 Design

- [![Figma](https://img.shields.io/badge/figma-%23F24E1E.svg?logo=figma&logoColor=white) Search-a-Licious logo](https://www.figma.com/design/gNnI56OF91Ugdd4fBaq5F8/Search-A-Licious-Logo?m=auto&t=bEHMawF1eequkY9w-6)
- [![Figma](https://img.shields.io/badge/figma-%23F24E1E.svg?logo=figma&logoColor=white) Wireframes & Mood board](https://www.figma.com/design/d1tuhFa0qQLqqJP2xbwBqZ/Website-%E2%80%93%3E-Search?node-id=1-2&t=bEHMawF1eequkY9w-0)


## Thank you to our sponsors !
## Thank you to our sponsors!

This project has received financial support from the NGI Search (New Generation Internet) program, funded by the 🇪🇺 European Commission. Thank you for supporting Open-Souce, Open Data, and the Commons.

Expand Down
79 changes: 56 additions & 23 deletions docs/devs/how-to-install.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,42 @@
# How to install for local development

## Pre-requisites
## Option 1: Dev Containers (Recommended)

The easiest way to get started is using Dev Containers, which provides a fully-configured development environment.

### Prerequisites

- [Docker](https://www.docker.com/products/docker-desktop) installed and running
- [VS Code](https://code.visualstudio.com/) with the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
- Or use [GitHub Codespaces](https://github.com/features/codespaces) for browser-based development

### Getting Started

1. Clone the repository
2. Open the folder in VS Code
3. When prompted, click "Reopen in Container" (or use the command palette: `Dev Containers: Reopen in Container`)
4. Wait for the container to build (first time takes a few minutes)
5. You're ready to develop! All dependencies are pre-installed.

The dev container automatically:

- Sets up Python 3.11 with Poetry and all backend dependencies
- Installs Node.js LTS with npm and all frontend dependencies
- Configures Docker-in-Docker for running compose commands
- Installs helpful VS Code extensions
- Starts Elasticsearch, Redis, and other services
- Sets up pre-commit hooks

You can now use all `make` commands and start developing immediately. Skip to the [Importing data](#importing-data-into-your-development-environment) section to load sample data.

## Option 2: Manual Installation

### Pre-requisites

First, follow same [prerequisite as for normal installation](../users/how-to-install.md#prerequisites):
* configuring mmap count
* installing docker and docker compose

- configuring mmap count
- installing docker and docker compose

## Installing Pre-commit

Expand All @@ -16,10 +47,9 @@ to install pre-commit on your machine.

### Auto-fixing linting issues in Pull Requests

For Pull Requests, you can automatically fix linting issues by commenting `/fix-linting` on the PR.
For Pull Requests, you can automatically fix linting issues by commenting `/fix-linting` on the PR.
This will trigger a GitHub Action that runs the linting tools and commits any fixes directly to the PR branch.


## Installing Direnv

Direnv is a tool to automatically set environment variables depending on the current directory.
Expand All @@ -30,6 +60,7 @@ For Linux and macOS users, You can follow our tutorial to install [direnv](https
## Setting up your environment

You have several options to set up your environment:

1. use direnv, and thus use the `.envrc` file to set up your environment
2. add a .envrc that you source in your terminal.
3. modify the .env file directly, in which case you should be careful to not commit your changes
Expand All @@ -51,10 +82,10 @@ export OFF_API_URL=https://world.openfoodfacts.org

[^winEnvrc]: For Windows users, the .envrc is only taken into account by the `make` commands.


## Building containers

To build the containers, you can run the following command:

```bash
make build
```
Expand All @@ -69,38 +100,40 @@ After that run the following command on another shell to compile the project: `m

Do this for next installation steps and to run the project.


> [!NOTE]
> * You may encounter a permission error if your user is not part of the `docker` group, in which case you should either [add it](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) or modify the Makefile to prefix `sudo` to all docker and docker compose commands.
> * Update container crash because we are not connected to any Redis, this is not a problem
>
> - You may encounter a permission error if your user is not part of the `docker` group, in which case you should either [add it](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) or modify the Makefile to prefix `sudo` to all docker and docker compose commands.
> - Update container crash because we are not connected to any Redis, this is not a problem

Docker spins up:

- Two elasticsearch nodes, one being exposed on port 9200 [^localhost_expose]
* test it going to http://127.0.0.1:9200
- test it going to http://127.0.0.1:9200
- [Elasticvue](https://elasticvue.com/) on port 8080
* test it going to http://127.0.0.1:8080
- test it going to http://127.0.0.1:8080
- The search service on port 8000
* test the API going to http://search.localhost:8000/docs
* test the UI going to http://search.localhost:8000/
- test the API going to http://search.localhost:8000/docs
- test the UI going to http://search.localhost:8000/

[^localhost_expose]: by default we only expose on the localhost interface.
This is driven by the `*_EXPOSE` variables in `.env`.
[^localhost_expose]:
by default we only expose on the localhost interface.
This is driven by the `*_EXPOSE` variables in `.env`.

Congratulations, you have successfully installed the project !
Congratulations, you have successfully installed the project !

You will then need to import from a JSONL dump (see instructions below).

## Importing data into your development environment

- Import Taxonomies: `make import-taxonomies`
- Import products :
```bash
# get some sample data
curl https://world.openfoodfacts.org/data/exports/products.random-modulo-10000.jsonl.gz --output data/products.random-modulo-10000.jsonl.gz
gzip -d data/products.random-modulo-10000.jsonl.gz
# we skip updates because we are not connected to any redis
make import-dataset filepath='products.random-modulo-10000.jsonl' args='--skip-updates'
```
```bash
# get some sample data
curl https://world.openfoodfacts.org/data/exports/products.random-modulo-10000.jsonl.gz --output data/products.random-modulo-10000.jsonl.gz
gzip -d data/products.random-modulo-10000.jsonl.gz
# we skip updates because we are not connected to any redis
make import-dataset filepath='products.random-modulo-10000.jsonl' args='--skip-updates'
```

Verify you have data by going to http://search.localhost:8000/

Expand Down