Skip to content

syonghua/pdf2md

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Run

python3 -m venv venv
pip3 install -r requirements.txt
  • activate venv
source activate venv/bin/activate
  • install requirements.txt
pip3 install -r requirements.txt
  • prepare config.json
{
    "llm" : {
        "apiKey" : "your-api-key",
        "baseUrl" : "https://open.bigmodel.cn/api/paas/v4/",
        "model" : "glm-4v-flash"
    },
    "yolo": {
        "modelPath" : "path-to-yolo/doclayout_yolo_docstructbench_imgsz1024.pt"
    }
}

download yolo model https://huggingface.co/juliozhao/DocLayout-YOLO-DocStructBench/

export HF_ENDPOINT="https://hf-mirror.com"
huggingface-cli download juliozhao/DocLayout-YOLO-DocStructBench \
                        --local-dir ~/DocLayout-YOLO-DocStructBench \
                        --local-dir-use-symlinks False \
                        --resume-download

Run

python3 main.py -c config.json ~/Downloads/bitter_lesson.pdf

convert markdowns to epub

cd ~/Downloads/bitter_lesson_md

# prepare metadata for 
cat > final.md <<EOF
---
title: "Bitter Lesson"
author: "Rich Sutton"
language: "en"
cover-image: "cover.jpeg"
---
EOF

cat all.md >> final.md

# convert to markdown to epub
pandoc -s final.md -t epub2 -o final.epub

About

use vlm model and yolo to help parse pdf to markdowns

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages