These taggers are part of the project textlens , but can also be run in standalone mode.

This repository refers to tagger models that have already been trained and are ready to be used in production. To train models, see galahad-train-battery.

Quick start

Clone this repository and its submodules.

git clone https://github.com/CCL-KULeuven/taggers-dockerized

Pull builds from Docker Hub

Do you have docker and docker compose? Do you have access to the public Docker Hub cclkuleuven? Then you can clone this repository and run

docker compose up

To run the taggers locally. The taggers are avaible on localhost with port equal to their devport. (You can find out the devport by looking at the port-forwards defined in docker-compose.yml)

Alternatively you can start specific taggers with:

docker compose up [SPECIFIC_TAGGER_1] [SPECIFIC_TAGGER_2]

Local builds

Build the docker images: see buildall.sh.

Connecting to a textlens/Galahad-like endpoint

If you want to connect the taggers to an endpoint outside of a docker network, you can specify a .env.dev file like

CALLBACK_SERVER=http://host.docker.internal:8010/internal/jobs

and use it instead of the default .env file like

docker compose --env-file .env.dev

Running the spaCy tagger

For instance: to run only the small Dutch spaCy tagger:

source dev.env
docker compose --env-file dev.env up spacy-nl-sm

Creating your own tagger

To create your own tagger, use the base tagger as a starting point and overwrite process.py. I.e., start your Dockerfile with:

FROM cclkuleuven/taggers-dockerized-base:$TAG
COPY --link process.py /

And fill out the process() and (optionally) init() functions of base/process.py.

The in_file points to a plain text file. Currently, your tagger is expected to produce tsv as output. The output tsv must contain a header with at least the columns 'token', 'lemma', 'pos' defined in any order.

Running your own tagger

1. Define your tagger as a service in a docker compose file, say your-tagger-dockerized.yml . (You can use docker-compose.yml as guidance)
2. Make sure the tagger is in the taggers-dockerized_taggers-network network. your-tagger-dockerized.yml should specify it as an external network and add your tagger to it:

   services:
    my-tagger:
     ...
     ports:
      - 8091:8080 # optional devport
     networks:
      - taggers-dockerized_taggers-network
   networks:
    taggers-dockerized_taggers-network:
     external: true

3. Launch your tagger

   docker compose -f your-tagger-dockerized.yml up
   

   (add the optional -d flag to run in detached mode)

If you specified a devport, you can now find your tagger at localhost port devport.

Make textlens aware of your tagger

All that is left, is to add a yaml metadata file in the server/data/taggers/ folder of textlens. See the textlens repository for more details.