Skip to content

Commit 1a1bcb0

Browse files
authored
mintable.js Refactor (#26)
* Fix progress indicator for account setup * Refactor Google & Plaid code to support multiple months & pagination * Refactor Google & Plaid code to support multiple months & pagination * Batch format & clear requests for sheets * Documentation improvements * Documentation improvements * Documentation improvements * Documentation improvements * Documentation improvements * Bump Version
1 parent 51a87a3 commit 1a1bcb0

13 files changed

+500
-288
lines changed

README.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -56,11 +56,11 @@ You can see a full list of options in the **[Config Docs](./docs/CONFIG.md)**.
5656
**How is this different from [build-your-own-mint](https://github.com/yyx990803/build-your-own-mint)?**
5757

5858
- **[build-your-own-mint](https://github.com/yyx990803/build-your-own-mint)** is a set of scripts which solely facilitates the integration between Plaid and Google Sheets. It makes no assumptions about what you want your spreadsheet to look like, and you have to define your own logic to map transactions to spreadsheet updates.
59-
- **[Mintable](#)** is and end-to-end system that works out of the box. It comes with a setup wizard, a web-based configuration server, pluggable providers (you're not limited to just Plaid & Google Sheets), and a spreadsheet template.
59+
- **[Mintable](#)** is and end-to-end system that works out of the box. It comes with a setup wizard, a web-based configuration server, [pluggable providers](./docs/PROVIDERS.md) (you're not limited to just Plaid & Google Sheets), and a spreadsheet template.
6060

6161
**Do I have to give my data to Plaid and Google? Are there any completely self-hosted alternatives I can use?**
6262

63-
- It's pluggable! Plaid & Google Sheets are working right now – contributions are welcome for other providers!
63+
- It's [pluggable](./docs/PROVIDERS.md)! Plaid & Google Sheets are working right now – contributions are welcome for [other providers](./docs/PROVIDERS.md)!
6464

6565
**Do I have to manually run this every time I want new transactions in my spreadsheet?**
6666

docs/CONFIG.md

+125-22
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
## Configuration
1+
# Configuration
22

33
All configurations below can be made using the web configuration framework or by editing `mintable.config.json`.
44

5-
`mintable.config.json` is the secret sauce – it contains all of your private tokens and is never sent to third-party servers. This file is ignored by Git – keep a backup somewhere safe as you only have 100 Plaid accounts on the free version.
5+
`mintable.config.json` is the secret sauce – it contains all of your private tokens and is never sent to third-party servers. This file is ignored by Git – keep a backup somewhere safe.
66

77
> **Pro Tip:** You can use Dropbox or another trusted service to sync `mintable.config.json` across your machines. Run `ln -s <path_to_cloud_folder>/mintable.config.json .` from the repo root to symlink Mintable to the cloud version.
88
9-
#### Automate Updates with a CI Provider
9+
### Automate Updates with a CI Provider
1010

1111
This repo includes config files for both [CircleCI](https://circleci.com/) and [Travis CI](https://travis-ci.com) to run builds automatically.
1212

@@ -18,53 +18,156 @@ yarn export
1818

1919
Run this command and paste the result into an environment variable called `MINTABLE_CONFIG` in your CI provider of choice. Mintable will handle the rest.
2020

21-
> **Note:** Some CI providers (like Travis) require you to wrap this variable in single quotes, i.e. `'{ PLAID_TOKEN: "" ...}'`. If you get an error similar to `Unable to parse JSON...` when you run your CI build, give this a try.
21+
> **Note:** Some CI providers (like Travis) require you to wrap this variable in single quotes, i.e. `'{ "ACCOUNT_PROVIDER": "plaid", ...}'`. If you get an error similar to `Unable to parse JSON...` when you run your CI build, give this a try.
2222
2323
> **Warning:** If you choose to use CircleCI, you should turn off **Pass secrets to builds from forked pull requests** under **Build Settings** > **Advanced Settings**.
2424
25-
#### Transaction Columns
25+
### Start Date
2626

27-
`TRANSACTION_COLUMNS` specifies a list of [Plaid transaction properties](https://plaid.com/docs/#transactions) (using [`_.get()` syntax](https://lodash.com/docs/4.17.11#get)) to override the default automated columns. Each time you run Mintable, all the contents of these columns will be cleared and overwritten.
27+
`START_DATE` specifies the lower bound for fetching transactions in `YYYY.MM.DD` format.
2828

29-
For example, if you only want to auto-populate the name and amount for each transaction, you could add the following line to your `mintable.config.json` file:
29+
**Default:**
30+
31+
```javascript
32+
"START_DATE": undefined // If end date is not specified, Mintable will fetch the last 2 months of transactions
33+
```
34+
35+
For example, if you only want to fetch transactions which occur after or on December 1, 2018, you could add the following line to your `mintable.config.json` file:
36+
37+
```javascript
38+
"START_DATE": "2018.12.01"
39+
```
40+
41+
### End Date
42+
43+
`END_DATE` specifies the upper bound for fetching transactions in `YYYY.MM.DD` format.
44+
45+
**Default:**
46+
47+
```javascript
48+
"END_DATE": undefined // If end date is not specified, Mintable will fetch up until the current date
49+
```
50+
51+
For example, if you only want to fetch transactions which occur before or on December 1, 2018, you could add the following line to your `mintable.config.json` file:
3052

53+
```javascript
54+
"END_DATE": "2018.12.01"
3155
```
32-
TRANSACTION_COLUMNS=["name", "amount"]
56+
57+
### Transaction Columns
58+
59+
`TRANSACTION_COLUMNS` specifies a list of transaction properties (using [`_.get()` syntax](https://lodash.com/docs/4.17.11#get)) to automatically update in your spreadsheet. All the contents of these columns will be cleared and overwritten each time you run Mintable.
60+
61+
**Default:**
62+
63+
```javascript
64+
"TRANSACTION_COLUMNS": [ 'date', 'amount', 'name', 'account_details.official_name', 'category.0', 'category.1', 'pending' ]
3365
```
3466

35-
> **Warning:** Your mileage may vary if you choose to use additional properties outside the tested defaults (`date`, `amount`, `name`, `account`, `category.0`, `category.1`, `pending`). Proceed at your own risk, you're in uncharted territory.
67+
For example, if you only want to auto-populate the name and amount for each transaction, you could add the following line to your `mintable.config.json` file:
68+
69+
```javascript
70+
"TRANSACTION_COLUMNS": ["name", "amount"]
71+
```
3672

37-
#### Reference Columns
73+
### Reference Columns
3874

3975
`REFERENCE_COLUMNS` specifies a list of additional, non-automated columns for your reference/bookkeeping purposes. Each time you run Mintable, the contents of these columns will be preserved.
4076

41-
For example, if you want to add one column to track work expenses, and another to track joint expenses shared with a partner, you could add the following line to your `mintable.config.json` file:
77+
**Default:**
4278

79+
```javascript
80+
"REFERENCE_COLUMNS": ['notes', 'work', 'joint']
4381
```
44-
REFERENCE_COLUMNS=["work", "joint"]
82+
83+
For example, if you want to add one column to track work expenses, and another to track joint expenses shared with a partner, you could add the following line to your `mintable.config.json` file:
84+
85+
```javascript
86+
"REFERENCE_COLUMNS": ["work", "joint"]
4587
```
4688

4789
> **Warning:** Since reference columns are not automated by Mintable, they have the potential to get out of sync with transaction data (for example, if your bank deletes a transaction, causing a row to get removed in `TRANSACTION_COLUMNS`)
4890
49-
#### Category Overrides
91+
### Account Provider
92+
93+
`ACCOUNT_PROVIDER` specifies which service to use to fetch transactions.
5094

51-
`CATEGORY_OVERRIDES` specifies a list of overrides to handle transactions that are routinely miscategorized by Plaid's servers. Overrides take the following format:
95+
**Default:**
96+
97+
```javascript
98+
"ACCOUNT_PROVIDER": "plaid"
99+
```
100+
101+
### Spreadsheet Provider
102+
103+
`SHEET_PROVIDER` specifies which service to use to automate spreadsheet updates.
104+
105+
**Default:**
106+
107+
```javascript
108+
"SHEET_PROVIDER": "sheets" // "sheets" = Google Sheets
109+
```
110+
111+
# Provider-Specific Configuration
112+
113+
You can see the API definitions for account & spreadsheet providers in the **[provider docs](./docs/PROVIDERS.md)**.
114+
115+
## Plaid
116+
117+
### Category Overrides
118+
119+
`CATEGORY_OVERRIDES` specifies a list of overrides to handle transactions that are routinely miscategorized by Plaid's servers.
120+
121+
**Default:**
122+
123+
```javascript
124+
"CATEGORY_OVERRIDES": []
125+
```
126+
127+
Overrides take the following format:
52128

53129
* `pattern`: [JavaScript Regular Expression](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp#Syntax) to test transaction names against
54130
* `flags`: [JavaScript Regular Expression flags](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp#Syntax) (i.e. `i` for case insensitive)
55131
* `category.0`: Override for first (top-level) category
56132
* `category.1`: Override for second (level-2) category
57133

58-
For example, if you want anything matching `autopay` or `e-payment` to get categorized as `Credit Card Payment`, you could add the following line to your `mintable.config.json` file:
59-
60-
```
61-
CATEGORY_OVERRIDES=[{ "pattern": ".*(autopay|e.payment).*", "flags": "i", "category.0": "Transfer", "category.1": "Credit Card Payments" }]
134+
For example, if you want anything matching `autopay` or `e-payment` to get categorized as `Credit Card Payment`, you could add the following lines to your `mintable.config.json` file:
135+
136+
```javascript
137+
"CATEGORY_OVERRIDES": [
138+
{
139+
"pattern": ".*(autopay|e.payment).*",
140+
"flags": "i",
141+
"category.0": "Transfer",
142+
"category.1": "Credit Card Payments"
143+
}
144+
]
62145
```
63146

64-
#### Transaction Provider
147+
## Google Sheets
65148

66-
`ACCOUNT_PROVIDER` specifies which service to use to fetch transactions. At this time, the only possible value is `"plaid"`, but we plan to add other providers in the future.
149+
### Template Sheet
67150

68-
#### Spreadsheet Provider
151+
`TEMPLATE_SHEET` specifies the template spreadsheet to use when creating a _new_ sheet for a month.
69152

70-
`SHEET_PROVIDER` specifies which service to use to automate spreadsheet updates. At this time, the only possible value is `"sheets"`, but we plan to add other providers in the future.
153+
**Default:**
154+
155+
```javascript
156+
"TEMPLATE_SHEET": {
157+
// Public template: https://docs.google.com/spreadsheets/d/10fYhPJzABd8KlgAzxtiyFN-L_SebTvM8SaAK_wHk-Fw
158+
"SHEET_ID": "10fYhPJzABd8KlgAzxtiyFN-L_SebTvM8SaAK_wHk-Fw",
159+
"SHEET_TITLE": "Template"
160+
}
161+
```
162+
163+
* `SHEET_ID`: Google Sheets spreadsheet ID (from the URL: `docs.google.com/spreadsheets/d/`**`sheet_id`**`/edit`)
164+
* `SHEET_TITLE`: Title of the sheet (along the bottom row of the document)
165+
166+
For example, you could add the following lines to your `mintable.config.json` file:
167+
168+
```javascript
169+
"TEMPLATE_SHEET": {
170+
"SHEET_ID": "10fYhPJzABd8KasbqiyFN-L_SebTvM8SaAK_wHk-Fw",
171+
"SHEET_TITLE": "My Template Sheet"
172+
}
173+
```

docs/PROVIDERS.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Providers APIs
2+
3+
Mintable is designed to be pluggable, i.e. you can swap out Plaid or Google Sheets for another service of your choice.
4+
5+
This document outlines what Mintable expects of a provider, and what constraints/functionality you need to adhere to/implement if you want to add a new provider.
6+
7+
## Transactions (`ACCOUNT_PROVIDER`) API
8+
9+
Account providers should provide an exported function which takes in a startDate and endDate, and returns a `Promise` of a raw list of transaction objects, i.e.:
10+
11+
```javascript
12+
transactions = await require('../lib/providerName').fetchTransactions(startDate, endDate)
13+
```
14+
15+
At minimum, we expect `name`, `date`, and `amount` to be defined.
16+
17+
For example, the following would an acceptable response (after promise resolution):
18+
19+
```javascript
20+
[
21+
{ "name": "Amazon.com", "date": "2019-04-16T07:00:00.000Z", "amount": -40.22 },
22+
{ "name": "United Airlines", "date": "2019-04-06T07:00:00.000Z", "amount": -500 },
23+
{ "name": "Uber", "date": "2019-04-04T07:00:00.000Z", "amount": -6.33 }
24+
]
25+
```
26+
27+
## Spreadsheets (`SHEET_PROVIDER`) API
28+
29+
Spreadsheet providers should provide an exported function which takes in a map of sheet name to a list of transactions for that sheet, and return a `Promise` which resolves when all necessary operations to update that sheet are complete, i.e.:
30+
31+
```javascript
32+
await require('../lib/providerName').updateSheets(updates, options)
33+
```
34+
35+
where `updates` come in the following format:
36+
37+
```javascript
38+
{
39+
"2019.04": [
40+
{ "name": "Amazon.com", "date": "2019.04.16", "amount": -40.22 },
41+
{ "name": "United Airlines", "date": "2019.04.26", "amount": -500 }
42+
],
43+
"2019.05": [
44+
{ "name": "Uber", "date": "2019.05.11", "amount": -6.33 }
45+
]
46+
}
47+
```

package.json

+3-2
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
"main": "mintable.js",
55
"author": "Kevin Schaich",
66
"license": "MIT",
7-
"version": "1.1.0",
7+
"version": "1.2.0",
88
"scripts": {
99
"circleci:dry-run": "circleci local execute -e MINTABLE_CONFIG=$MINTABLE_CONFIG",
1010
"export": "node ./src/scripts/export.js",
@@ -18,18 +18,19 @@
1818
"@zeit/next-sass": "^1.0.1",
1919
"body-parser": "^1.18.3",
2020
"clipboardy": "^1.2.3",
21+
"date-fns": "^1.30.1",
2122
"dotenv": "^7.0.0",
2223
"express": "^4.16.4",
2324
"googleapis": "27",
2425
"indent-string": "^3.2.0",
2526
"isomorphic-unfetch": "^3.0.0",
2627
"lodash": "^4.17.11",
2728
"log-symbols": "^2.2.0",
28-
"moment": "^2.23.0",
2929
"next": "^8.0.3",
3030
"node-sass": "^4.11.0",
3131
"opn": "^6.0.0",
3232
"ora": "^3.4.0",
33+
"p-each-series": "^2.1.0",
3334
"p-map-series": "^2.1.0",
3435
"plaid": "^2.10.0",
3536
"promise-fs": "^2.1.0",

src/components/configPropertyInput.jsx

+2-2
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,10 @@ import { FiCheckCircle, FiHelpCircle } from 'react-icons/fi'
22

33
const ConfigPropertyInput = props => (
44
<div className='config-property-input'>
5-
<code title={props.tooltip} >
5+
<code title={props.tooltip}>
66
<FiCheckCircle className='icon' style={{ color: `#FFFFFF${props.modified ? 'FF' : '00'}` }} />
77
{props.displayName}
8-
{props.tooltip && <FiHelpCircle className='icon' style={{paddingBottom: '4px'}}/>}
8+
{props.tooltip && <FiHelpCircle className='icon' style={{ paddingBottom: '4px' }} />}
99
</code>
1010
<input
1111
type='text'

src/components/progressSidebar.jsx

+4-5
Original file line numberDiff line numberDiff line change
@@ -4,24 +4,23 @@ import * as _ from 'lodash'
44
const ProgressSidebar = props => {
55
return (
66
<div className='progress-sidebar'>
7-
<ProgressItem tab={props.tab} number={1} title='Welcome' href='/' />
7+
<ProgressItem tab={props.tab} number={1} title='Welcome' href='/welcome' />
88
<ProgressItem
9-
completed={props.config.accountsSetupCompleted}
9+
completed={props.config.accountProviderSetupComplete}
1010
tab={props.tab}
1111
number={2}
1212
title={`${_.startCase(_.lowerCase(props.config.ACCOUNT_PROVIDER))} Setup`}
1313
href='/account-provider-setup'
1414
/>
1515
<ProgressItem
16-
// completed={_.some(props.config, (value, key).plaid} TODO
17-
completed={false}
16+
completed={props.config.accountSetupComplete}
1817
tab={props.tab}
1918
number={3}
2019
title='Account Setup'
2120
href='/account-setup'
2221
/>
2322
<ProgressItem
24-
completed={props.config.sheetsSetupCompleted}
23+
completed={props.config.sheetProviderSetupComplete}
2524
tab={props.tab}
2625
number={4}
2726
title={`${_.startCase(_.lowerCase(props.config.SHEET_PROVIDER))} Setup`}

0 commit comments

Comments
 (0)