Pinned Loading

1. jax-llm jax-llm Public

   JAX implementation of Large Language Models. You can train GPT-2-like model with 青空文庫 (aozora bunko-clean dataset) or any other text dataset.

   Python 13 2

2. fast-near-duplicate-matching fast-near-duplicate-matching Public

   Fast near-duplicate matching is a method for quickly finding near-duplicate spans in a document by utilizing the Rabin-Karp algorithm.

   Rust 1

3. llm-jp/text2dataset llm-jp/text2dataset Public

   Easily turn large English text datasets into Japanese text datasets using open LLMs.

   Python 20 1

4. llm-jp/llm-jp-eval-mm llm-jp/llm-jp-eval-mm Public

   A lightweight framework for evaluating visual-language models.

   Python 29 4

5. llm-jp/clip-eval llm-jp/clip-eval Public

   clip-eval is a tool for evaluating CLIP models on various image classification and image-text retrieval tasks in Japanese.

   Python

6. llm-jp/llm-jp-modernbert llm-jp/llm-jp-modernbert Public

   This repository contains the training and evaluation code for llm-jp-modernbert-base.

   Python 8