Skip to content

Commit acf9ea6

Browse files
bubbleganbubblegan
authored
Update README.md
1 parent d0af4f9 commit acf9ea6

File tree

1 file changed

+201
-53
lines changed

1 file changed

+201
-53
lines changed

‎README.md

Lines changed: 201 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,81 +1,229 @@
1-
# Turborepo starter
21

3-
This is an official starter Turborepo.
2+
## Important
43

5-
## Using this example
4+
- This project is still in alpha, and there are constant changes happening.
5+
- There may be bugs and breaking changes. Feedback is deeply appreciated.
6+
- Please do not leak your OpenAI key or database string, as they contain sensitive information.
67

7-
Run the following command:
8+
## Motivations
89

9-
```sh
10-
npx create-turbo@latest
11-
```
10+
I started this open-source project because I found that most existing solutions, like those using SaltEdge, have limitations. For example, SaltEdge can only pull data from banks for a limited timeframe, making it hard to track expenses over longer periods.
1211

13-
## What's inside?
12+
I realized that leveraging AI to read and extract text from credit card PDF statements is a more efficient way to store and categorize expenses. By storing this data in a database, we can streamline the entire process. My goal is to create a self-hosted, local-first solution—much like Firefly or ActualBudget—where users have more control over their data.
1413

15-
This Turborepo includes the following packages/apps:
1614

17-
### Apps and Packages
15+
## Features
1816

19-
- `docs`: a [Next.js](https://nextjs.org/) app
20-
- `web`: another [Next.js](https://nextjs.org/) app
21-
- `@repo/ui`: a stub React component library shared by both `web` and `docs` applications
22-
- `@repo/eslint-config`: `eslint` configurations (includes `eslint-config-next` and `eslint-config-prettier`)
23-
- `@repo/typescript-config`: `tsconfig.json`s used throughout the monorepo
17+
The main feature to upload credit card PDFs or zip files containing credit card PDFs will work as follows:
2418

25-
Each package/app is 100% [TypeScript](https://www.typescriptlang.org/).
19+
[Video Demo](https://streamable.com/cjh94w)
2620

27-
### Utilities
21+
- Additional features like tagging, adding notes, and full CRUD functionality for categories and statements.
2822

29-
This Turborepo has some additional tools already setup for you:
23+
- I will take some time to document everything, so stay tuned.
3024

31-
- [TypeScript](https://www.typescriptlang.org/) for static type checking
32-
- [ESLint](https://eslint.org/) for code linting
33-
- [Prettier](https://prettier.io) for code formatting
25+
## Architecture
3426

35-
### Build
27+
### Stack
3628

37-
To build all apps and packages, run the following command:
29+
- Next.js: For server-side rendering and frontend development.
30+
- TurboRepo: To manage the monorepo for both frontend and backend.
31+
- Hono.js + BullMQ: Hono handles the backend, with BullMQ managing background jobs.
32+
- Prisma.js: Type-safe database ORM for managing relational data.
33+
- pdf2pic + Tesseract OCR: Converts PDFs to images and runs OCR to extract text from images.
34+
- TRPC: Provides a type-safe API between client and server.
35+
- shadCN: For accessible and customizable UI components.
3836

39-
```
40-
cd my-turborepo
41-
pnpm build
42-
```
37+
### Architecture Diagram
4338

44-
### Develop
39+
![architecture-map](https://github.com/user-attachments/assets/289fe6e6-17c8-4941-9cee-0cafcd84c225)
4540

46-
To develop all apps and packages, run the following command:
41+
### Parsing workflow
4742

48-
```
49-
cd my-turborepo
50-
pnpm dev
51-
```
43+
1. **Background Job Processing**
44+
PDF statements are sent to the background job server for processing.
5245

53-
### Remote Caching
46+
2. **Parsing Workflow**
47+
- **Convert PDFs to Images**: The PDF files are converted into images.
48+
- **OCR (Optical Character Recognition)**: OCR is run on the images to extract text from the statement.
49+
- **OpenAI API**: The extracted text is sent to the OpenAI API, which returns parsed expenses, categorized according to custom rules.
5450

55-
Turborepo can use a technique known as [Remote Caching](https://turbo.build/repo/docs/core-concepts/remote-caching) to share cache artifacts across machines, enabling you to share build caches with your team and CI/CD pipelines.
51+
3. **Temporary Storage in Redis**
52+
The parsed expenses are temporarily stored in Redis.
5653

57-
By default, Turborepo will cache locally. To enable Remote Caching you will need an account with Vercel. If you don't have an account you can [create one](https://vercel.com/signup), then enter the following commands:
54+
4. **Frontend Options**
55+
On the frontend, users can:
56+
- Store the parsed data in the database
57+
- Download it as a CSV file
58+
- Edit the data before taking any further action
5859

59-
```
60-
cd my-turborepo
61-
npx turbo login
62-
```
6360

64-
This will authenticate the Turborepo CLI with your [Vercel account](https://vercel.com/docs/concepts/personal-accounts/overview).
61+
## Database and OpenAI key
6562

66-
Next, you can link your Turborepo to your Remote Cache by running the following command from the root of your Turborepo:
63+
### Open AI Key
6764

68-
```
69-
npx turbo link
70-
```
65+
1. Visit the [OpenAI website](https://platform.openai.com/signup/) and sign up for an account. If you already have one, log in instead.
7166

72-
## Useful Links
67+
2. Once logged in, go to [Api Key Dashboard](https://platform.openai.com/api-keys)
7368

74-
Learn more about the power of Turborepo:
69+
3. Click the Create new secret key button.
7570

76-
- [Tasks](https://turbo.build/repo/docs/core-concepts/monorepos/running-tasks)
77-
- [Caching](https://turbo.build/repo/docs/core-concepts/caching)
78-
- [Remote Caching](https://turbo.build/repo/docs/core-concepts/remote-caching)
79-
- [Filtering](https://turbo.build/repo/docs/core-concepts/monorepos/filtering)
80-
- [Configuration Options](https://turbo.build/repo/docs/reference/configuration)
81-
- [CLI Usage](https://turbo.build/repo/docs/reference/command-line-reference)
71+
4. Make sure to copy and securely store your API key, as you will not be able to view it again once you leave the page. You can always create another key if needed.
72+
73+
This [medium](https://medium.com/@lorenzozar/how-to-get-your-own-openai-api-key-f4d44e60c327) guide here should be quite useful.
74+
75+
Make sure you have sufficient credit in your OpenAI account for billing purposes. It costs approximately $0.03 USD per credit card statement. Detailed cost calculations will be provided here.
76+
77+
78+
### Postgresql DB Connection
79+
80+
You can setup postgresql locally, but here is the guide on setting up in supabase
81+
82+
1. Create a Supabase Account
83+
- Go to the [Supabase website](https://supabase.com/) and sign up for an account.
84+
- If you already have an account, simply log in.
85+
86+
1. Create a New Project
87+
- After logging in, click the **New Project** button on your dashboard.
88+
- Enter your project details:
89+
- **Name**: Choose a unique name for your project.
90+
- **Organization**: Select your organization or create a new one if necessary.
91+
- **Database Password**: Create a secure password for your database. Keep this password safe since you'll need it to connect to your database later.
92+
- **Region**: Choose the closest server location to optimize performance.
93+
94+
Once done, click **Create new project**.
95+
96+
3. Wait for the Database to be Set Up
97+
- It may take a few minutes for Supabase to set up your PostgreSQL database. Once ready, you’ll be taken to your project dashboard.
98+
99+
4. Retrieve Your Database Connection String
100+
- From your project dashboard, navigate to the **Settings** tab.
101+
- Under **Database**, you’ll see details for connecting to your database.
102+
- Look for **Connection String** or **Database URL**. It will look something like this:
103+
104+
```plaintext
105+
postgres://username:password@host:port/database
106+
```
107+
108+
You will need this connection string to connect your application to the Supabase database.
109+
110+
5. Save Your Connection String
111+
- Make sure to copy and securely store the connection string. You will need it to set up the database in your app.
112+
113+
114+
115+
## Docker Setup
116+
117+
### Prerequisite
118+
119+
1. **Docker**
120+
- Install docker
121+
- Run Docker
122+
123+
### Installation
124+
125+
1. Clone the repo into a public GitHub repository.
126+
127+
```sh
128+
git clone https://github.com/bubblegan/self-hosted-expense-tracker.git
129+
```
130+
131+
1. Go to the app folder
132+
133+
```sh
134+
cd self-hosted-expense-tracker
135+
```
136+
137+
1. Run npm install
138+
139+
```sh
140+
npm install
141+
```
142+
143+
1. Set up your `.env` file
144+
145+
- Duplicate `.env.example` in `/apps/web` and `/apps/background-job` to `.env`
146+
- Use `openssl rand -base64 32` to generate a key and add it under `NEXTAUTH_SECRET` in the `/apps/web/.env` file.
147+
- Add Open AI key to `OPENAI_API_KEY` in `/apps/web/.env` and `/apps/background-job/.env`
148+
- Add Postgresql database connection string to `DATABASE_URL` in `/apps/web/.env`
149+
150+
1. Run prisma migrate for the first time to migrate database.
151+
152+
```sh
153+
npx prisma migrate deploy --schema apps/web/prisma/schema.prisma
154+
```
155+
156+
1. Run Docker Compose and build
157+
158+
```sh
159+
docker compose up --build
160+
```
161+
162+
163+
## Development Setup
164+
165+
166+
### Prerequisite
167+
1. NodeJs
168+
1. Ghostscript and graphicsmagick
169+
- Both Ghostscript and Graphicsmagick are require to PDF parsing on the background.
170+
- Follow this [guide](https://github.com/yakovmeister/pdf2image/blob/HEAD/docs/gm-installation.md) to install both of them
171+
1. Redis
172+
- Setup and run redis on port 6379. Guide [here](https://redis.io/docs/latest/operate/oss_and_stack/install/install-redis/).
173+
174+
### Installation
175+
176+
1. Clone the repo into a public GitHub repository.
177+
178+
```sh
179+
git clone https://github.com/bubblegan/self-hosted-expense-tracker.git
180+
```
181+
182+
1. Go to the app folder
183+
184+
```sh
185+
cd self-hosted-expense-tracker
186+
```
187+
188+
1. Run npm install
189+
190+
```sh
191+
npm install
192+
```
193+
194+
1. Set up your `.env` file
195+
196+
- Duplicate `.env.example` in `/apps/web` and `/apps/background-job` to `.env`
197+
- Use `openssl rand -base64 32` to generate a key and add it under `NEXTAUTH_SECRET` in the `/apps/web/.env` file.
198+
- Add Open AI key to `OPENAI_API_KEY` in `/apps/web/.env` and `/apps/background-job/.env`
199+
- Add Postgresql database connection string to `DATABASE_URL` in `/apps/web/.env`
200+
201+
1. Run prisma migrate for the first time to migrate database.
202+
203+
```sh
204+
npx prisma migrate deploy --schema apps/web/prisma/schema.prisma
205+
```
206+
207+
1. Run in development mode
208+
209+
```sh
210+
npm run dev
211+
```
212+
213+
214+
## Start Up Guide
215+
216+
1. Go to `/sign-up` to create account with `username` and `password` after everything has set up.
217+
218+
1. The categories will be seeded from `/apps/web/prisma/categoryList.json`.
219+
220+
221+
## Roadmap
222+
223+
Will put this into proper format soon :
224+
225+
- Support for different self-hosting databases like SQLite and MySQL.
226+
- Hosting guide for various VPS providers.
227+
- Support for different LLM models beyond those provided by OpenAI.
228+
- Calendar view for tracking expenses.
229+
- And much more.

0 commit comments

Comments
 (0)