Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion MANIFEST.in

This file was deleted.

74 changes: 52 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,8 @@ rules:
account: 565444499
currency: USD
ruleset:
- Replace_Asset
- Replace_Expense
- ReplaceAsset
- ReplaceExpense
```

### Structure of a configuration file
Expand Down Expand Up @@ -198,9 +198,9 @@ The `_ic` indicates `ignore case`.

The following sections provide a detailed explanation of the rules available in Beanborg.

#### Replace_Payee
#### ReplacePayee

The `Replace_Payee` rule is used to modify the name of a transaction’s counterparty. This is useful when you want to standardize or adjust the names in your financial records.
The `ReplacePayee` rule is used to modify the name of a transaction’s counterparty. This is useful when you want to standardize or adjust the names in your financial records.

This rule requires a lookup file named `payee.rules`, which should be placed in the directory defined by the `rules.rules_folder` option in the configuration file.

Expand All @@ -214,7 +214,7 @@ Given the following CSV transaction:

You would follow these steps:

1. Add the `Replace_Payee` rule to the list of rules in the configuration file for the relevant financial institution.
1. Add the `ReplacePayee` rule to the list of rules in the configuration file for the relevant financial institution.
2. In the `payee.rules` lookup file, add the following entry:

```
Expand All @@ -224,9 +224,9 @@ Fresh Food Inc.;equals;FRESH FOOD
This will ensure that the counterparty "Fresh Food Inc." is replaced with "FRESH FOOD" in your Beancount ledger.


#### Replace_Expense
#### ReplaceExpense

The `Replace_Expense` rule is used to assign an account to a transaction based on the value of the `counterparty` index from the CSV file. This rule is particularly helpful for categorizing transactions into the appropriate expense accounts.
The `ReplaceExpense` rule is used to assign an account to a transaction based on the value of the `counterparty` index from the CSV file. This rule is particularly helpful for categorizing transactions into the appropriate expense accounts.

This rule requires a lookup file named `account.rules`, which should be located in the directory defined by the `rules.rules_folder` option in the configuration file.

Expand All @@ -240,7 +240,7 @@ Given the following CSV transaction:

You would follow these steps:

1. Add the `Replace_Expense` rule to the list of rules in the configuration file for the relevant financial institution.
1. Add the `ReplaceExpense` rule to the list of rules in the configuration file for the relevant financial institution.
2. In the `account.rules` lookup file, add the following entry:

```
Expand All @@ -250,13 +250,13 @@ Fresh Food Inc.;equals;Expenses:Groceries
This will ensure that any transaction with "Fresh Food Inc." as the counterparty will be assigned to the `Expenses:Grocery` account in your Beancount ledger.


#### Replace_Asset
#### ReplaceAsset


The `Replace_Asset` rule assigns an "origin" account to a transaction based on the value of the `account` index in a CSV file.
The `ReplaceAsset` rule assigns an "origin" account to a transaction based on the value of the `account` index in a CSV file.
This rule is useful for ensuring that transactions are recorded with the correct source account in Beancount.

The `Replace_Asset` rule is automatically added to the ruleset, even if it is not explicitly declared in the configuration file.
The `ReplaceAsset` rule is automatically added to the ruleset, even if it is not explicitly declared in the configuration file.

##### Origin Account Resolution

Expand Down Expand Up @@ -305,7 +305,7 @@ rules:
origin_account: Assets:Jim:Current
```

#### Set_Accounts
#### SetAccounts

Assigns an "origin" account to a transaction, based on value of the `account` index of a CSV file row.
This rule is useful to assign the correct source account of a CSV transaction. This rule is **implicitly added** to the ruleset, even if it doesn't get declared
Expand All @@ -321,7 +321,7 @@ As an example, let's take this CSV transaction. We want to import the transactio
04.11.2020;04.11.2020;Direct Debit;"Fresh Food Inc.";-21,30;EUR;0000001;UK0000001444555
```

Add the `Replace_Asset` to the `ruleset` and create an `asset.rules` file. Add the following snippet to the `asset.rules` file:
Add the `ReplaceAsset` to the `ruleset` and create an `asset.rules` file. Add the following snippet to the `asset.rules` file:

```
value;expression;result
Expand All @@ -348,9 +348,9 @@ rules:
origin_account: Assets:Jim:Current
```

#### Set_Accounts
#### SetAccounts

The `Set_Accounts` sets both the **origin** and **destination** account for a given transaction, based on one or more values of a given CSV index.
The `SetAccounts` sets both the **origin** and **destination** account for a given transaction, based on one or more values of a given CSV index.
This rule is useful for transactions like ATM withdrawals, where both accounts need to be defined.

As an example, consider the following CSV transaction representing an ATM withdrawal:
Expand Down Expand Up @@ -381,7 +381,7 @@ For example, if you want the rule to apply to different forms of "withdrawal" in


```
- name: Set_Accounts
- name: SetAccounts
from: Assets:Jim:Current
to: Assets:Jim:Cash
csv_index: 2
Expand All @@ -392,16 +392,16 @@ For example, if you want the rule to apply to different forms of "withdrawal" in
- Wildcards are supported using `fnmatch`. In the example above,
the wildcard * is used to match any string that contains `Retiro` or `Ritiro`.

#### Ignore_By_Payee
#### IgnoreByPayee

The `Ignore_By_Payee` rule can be used to ignore transactions based on the value of the `counterparty` index in a CSV file.
The `IgnoreByPayee` rule can be used to ignore transactions based on the value of the `counterparty` index in a CSV file.
This is useful when you want to exclude specific transactions from being imported into the ledger.


Suppose you want to ignore any transactions where the counterparty is "Mc Donald" or "Best Shoes". You can configure the rule as follows:

```
- name: Ignore_By_Payee
- name: IgnoreByPayee
ignore_payee:
- Mc Donald
- Best Shoes
Expand All @@ -410,22 +410,52 @@ Suppose you want to ignore any transactions where the counterparty is "Mc Donald
The names of counterparties in the `ignore_payee` list are case-insensitive. This means both "Mc Donald" and "mc donald" would be matched and ignored.


#### Ignore_By_StringAtPos
#### IgnoreByStringAtPos

The `Ignore_By_StringAtPos` rule allows you to ignore a transaction based on the value found at a specific index in the CSV file. This is useful for filtering out transactions that meet specific criteria in a particular column.
The `IgnoreByStringAtPos` rule allows you to ignore a transaction based on the value found at a specific index in the CSV file. This is useful for filtering out transactions that meet specific criteria in a particular column.

### Example

To ignore transactions where the value in index 4 (fifth column) matches `abc0102`, configure the rule like this:

```yaml
- name: Ignore_By_StringAtPos
- name: IgnoreByStringAtPos
ignore_string_at_pos:
- abc0102;4
```
- The index in the CSV file starts from `0`, so `4` refers to the fifth column.
- The values specified in `ignore_string_at_pos` are case-insensitive, meaning `abc0102` and `ABC0102` would both be matched and ignored.

#### IgnoreByContainsStringAtPos

Ignores a transaction if the specified value is present in the specified index of the CSV file.
The comparison is case-insensitive (meaning "mega" and "MEGA" will be matched).

For instance, given this csv entry:

`10.12.2022,mega supermarket,20US$`

and this rule:

```
- name: Ignore_By_ContainsStringAtPos
ignore_string_contains_at_pos:
- mega;1
```

The row will be ignored, because the string "mega" is part of the index at position 1.

Note that this rule supports multiple string specifications.

Example:
```
- name: Ignore_By_ContainsStringAtPos
ignore_string_contains_at_pos:
- val;3
- another val;6
```


### Custom rules

TODO
Expand Down
28 changes: 0 additions & 28 deletions beanborg/arg_parser.py

This file was deleted.

142 changes: 92 additions & 50 deletions beanborg/bb_archive.py
Original file line number Diff line number Diff line change
@@ -1,68 +1,110 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

__copyright__ = "Copyright (C) 2023 Luciano Fiandesio"
__copyright__ = "Copyright (C) 2024 Luciano Fiandesio"
__license__ = "GNU GPLv2"

import csv
import os
import shutil
import sys
from datetime import datetime
from typing import Optional

import fire
from fire.decorators import SetParseFns
from rich import print as rprint

from beanborg.arg_parser import eval_args
from beanborg.config import init_config


def main():

args = eval_args("Archives imported CVS file")
config = init_config(args.file, args.debug)

target_csv = os.path.join(config.csv.target, config.csv.ref + ".csv")

if not os.path.isfile(target_csv):
rprint(f"[red]file: {target_csv} does not exist![red]")
sys.exit(-1)

if not os.path.isdir(config.csv.archive):
os.mkdir(config.csv.archive)
class BeanborgArchiver:
"""Archive imported CSV files with date range in filename.
Synopsis:
bb_archive --config_file=CONFIG_FILE [--debug=true]
Description:
Archives processed CSV files by moving them to an archive directory.
The archived filename includes the date range of transactions contained
in the file.
Arguments:
config_file: Path to the YAML configuration file
debug: Enable debug output (default: False)
Examples:
bb_archive --config_file=config.yaml
bb_archive --config_file=/path/to/config.yaml --debug=true
"""

@SetParseFns(f=str, config_file=str, debug=bool)
def __call__(
self,
f: Optional[str] = None,
config_file: Optional[str] = None,
debug: bool = False,
):
"""Archive the processed CSV file.
Args:
f: Path to the configuration file (shorthand for --config-file)
config_file: Path to the configuration file
debug: Enable debug mode
Returns:
int: 0 for success, 1 for failure
"""
final_config_file = f or config_file
if not final_config_file:
raise ValueError(
"Configuration file must be specified using either -f or --config-file"
)
config = init_config(final_config_file, debug)

target_csv = os.path.join(config.csv.target, config.csv.ref + ".csv")

if not os.path.isfile(target_csv):
rprint(f"[red]file: {target_csv} does not exist![red]")
sys.exit(-1)

if not os.path.isdir(config.csv.archive):
os.mkdir(config.csv.archive)

dates = []
print("\u2713" + " detecting start and end date of transaction file...")
with open(target_csv) as csv_file:
csv_reader = csv.reader(csv_file, delimiter=config.csv.separator)
for _ in range(config.csv.skip):
next(csv_reader) # skip the line

for row in csv_reader:
try:
dates.append(
datetime.strptime(
row[config.indexes.date].strip(), config.csv.date_format
)
)
except Exception as ex:
print("error: " + str(ex))

print("\u2713" + " moving file to archive...")
os.rename(
target_csv,
config.csv.archive
+ "/"
+ config.csv.ref
+ "_"
+ str(min(dates).date())
+ "_"
+ str(max(dates).date())
+ ".csv",
)

print("\u2713" + " removing temp folder")
shutil.rmtree(config.csv.target)

dates = []
print("\u2713" + " detecting start and end date of transaction file...")
with open(target_csv) as csv_file:
csv_reader = csv.reader(csv_file, delimiter=config.csv.separator)
for _ in range(config.csv.skip):
next(csv_reader) # skip the line

for row in csv_reader:
try:
dates.append(
datetime.strptime(
row[config.indexes.date].strip(), config.csv.date_format
)
)
except Exception as ex:
print("error: " + str(ex))

print("\u2713" + " moving file to archive...")
os.rename(
target_csv,
config.csv.archive
+ "/"
+ config.csv.ref
+ "_"
+ str(min(dates).date())
+ "_"
+ str(max(dates).date())
+ ".csv",
)

print("\u2713" + " removing temp folder")
shutil.rmtree(config.csv.target)


if __name__ == "__main__":
main()
def main():
"""Main function to run the beanborg archiver."""
fire.Fire(BeanborgArchiver)
Loading