Skip to content
This repository was archived by the owner on Nov 21, 2023. It is now read-only.
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 29 additions & 4 deletions shared_tasks/dynalab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,19 +86,44 @@ Note that the "signed" field will be obtained by calling `self.taskIO.sign_respo
Also note that you can edit the [requirements.txt](./requirements.txt) file,
based your model's specific dependencies.

Once you've implemented the handler you'll need to test it locally.
### Project installation and dependencies

First install `dynalab` using instructions from their [repo](https://github.com/facebookresearch/dynalab#installation).
#### 1. Create & activate virtual venv
```
python3 -m venv venv
source venv/bin/activate
```
#### 2. Install Dynalab, fairseq & sentencepiece
```
git clone https://github.com/facebookresearch/dynalab.git
cd dynalab
pip install -e .
pip install sentencepiece
pip install fairseq
```
#### 3. Clone flores repo & put your model files inside the shared_task/dynalab folder
you can get your model files from (https://dl.fbaipublicfiles.com/flores101/pretrained_models/flores101_mm100_175M.tar.gz) and extract the contents of the zip and put it in "shared_task/dynalab"
```
git clone https://github.com/facebookresearch/flores
cd shared_task/dynalab
```
Once you've implemented the handler you'll need to test it locally.

The simplest test is to run `python handler.py`.
You'll need to update the `local_test` function to use the task you want.
Then you can move to running more involved tests using Dynalab.

Afterwards, from this directory run:
`dynalab-cli init -n <name-of-your-model>`
```
dynalab-cli init -n <name_of_your_model> --model-checkpoint <model.pt_file> -t <flores_task_code> --model-files <your_comma_seperated_model_files>

example:

dynalab-cli init -n asimafrican --model-checkpoint model.pt -t flores_african --model-files "model_generation.json","dict.txt","sentencepiece.bpe.model"
```
Note that the model name needs to be lower-kebab-case.

Chose the track you want to apply to: "flores_small1", "flores_small2" or "flores_full".
Chose the track you want to apply to: "flores_african"
Note that the input format is same for all the tracks.
Then follow the prompt instruction and point to your model path, handler path ...

Expand Down
Loading