Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,15 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Added

- **Manual edits made in Bitwarden are no longer silently reverted on re-run** (issue #30). kp2bw is
KeePass-authoritative, so any difference on an existing item used to be overwritten with the KeePass value -- quietly
undoing a title you fixed, a note you added or a URI you corrected in Bitwarden. Every item kp2bw writes now carries a
hidden-from-noise `KP2BW_SYNC` content signature; a re-run that finds an item's current content no longer matching
that stamp knows a *user* edited it (kp2bw's own writes restamp, so they never self-trip) and **preserves the edit**,
reporting it as `protected` in the summary instead of clobbering it. `--force-update` (env `KP2BW_FORCE_UPDATE`) makes
KeePass win regardless. Unchanged re-runs stay idempotent (the stamp is excluded from the content diff), legacy/
unstamped items keep current behaviour, and `--strip-ids` now removes `KP2BW_SYNC` alongside `KP2BW_ID`.

- **`--report-uris keepass|bitwarden` -- a read-only URI collision report** (env `KP2BW_REPORT_URIS`). Groups every
login URL by registrable domain (a curated two-level public-suffix heuristic, so `10bis.co.il` stays whole) and lists
the domains with more than one host -- exactly the logins that all surface together under Bitwarden's base-domain
Expand Down
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ with advantages over the built-in Bitwarden importer:
- <details open><summary><b>Passkey migration</b></summary> KeePassXC FIDO2/passkey credentials (<code>KPEX_PASSKEY_*</code>) are converted to Bitwarden <code>fido2Credentials</code>.</details>
- <details><summary><b>Custom properties &amp; attachments</b></summary> Imported as Bitwarden custom fields or attachments (values &gt;10k chars auto-upload as files).</details>
- <details><summary><b>Long notes handling</b></summary> Notes exceeding 10k chars are uploaded as <code>notes.txt</code> attachments.</details>
- <details><summary><b>Idempotent re-runs that sync changes</b></summary> Safe to run repeatedly; existing entries are updated in place when their KeePass content changed (notes, credentials, URIs, fields) and never duplicated. Each item is stamped with its KeePass UUID in a <code>KP2BW_ID</code> field, so distinct entries that share a title stay separate and a re-run is matched by identity rather than title.<br> Disable updates with <code>--no-update</code>.</details>
- <details><summary><b>Idempotent re-runs that sync changes</b></summary> Safe to run repeatedly; existing entries are updated in place when their KeePass content changed (notes, credentials, URIs, fields) and never duplicated. Each item is stamped with its KeePass UUID in a <code>KP2BW_ID</code> field, so distinct entries that share a title stay separate and a re-run is matched by identity rather than title. Edits you make in Bitwarden are <b>protected</b>: a re-run preserves them instead of reverting to KeePass (a <code>KP2BW_SYNC</code> content stamp tells your edit apart from kp2bw's own writes); <code>--force-update</code> overrides.<br> Disable updates with <code>--no-update</code>.</details>
- <details><summary><b>Nested folders</b></summary> KeePass folder hierarchy is recreated in Bitwarden.</details>
- <details><summary><b>Recycle Bin filtering</b></summary> Deleted entries are automatically excluded.</details>
- <details><summary><b>Expiry awareness</b></summary> Expired entries are marked <code>[EXPIRED]</code> in notes; optionally skip them entirely with <code>--skip-expired</code>.</details>
Expand Down Expand Up @@ -84,7 +84,7 @@ kp2bw [-h] [-V] [-k PASSWORD] [-K FILE] [-b PASSWORD] [-o ID]
[-t TAG [TAG ...]] [-c ID] [--path-to-name | --no-path-to-name]
[--path-to-name-skip N] [--skip-expired | --no-skip-expired]
[--include-recycle-bin | --no-include-recycle-bin]
[--metadata | --no-metadata] [--update | --no-update]
[--metadata | --no-metadata] [--update | --no-update] [--force-update]
[--include-oversize-secrets] [--uri-match MODE]
[--interpret-uri-syntax | --no-interpret-uri-syntax]
[--migrate-uris] [--report-uris SOURCE] [--strip-ids] [-y] [-v] [-d]
Expand All @@ -106,12 +106,13 @@ kp2bw [-h] [-V] [-k PASSWORD] [-K FILE] [-b PASSWORD] [-o ID]
| `--include-recycle-bin` | Include Recycle Bin entries (excluded by default) | `KP2BW_INCLUDE_RECYCLE_BIN` |
| `--metadata` / `--no-metadata` | Toggle KeePass tags/expiry as a `KP2BW_META` field (default: on) | `KP2BW_MIGRATE_METADATA` |
| `--update` / `--no-update` | Update existing entries changed in KeePass (default: on) | `KP2BW_UPDATE` |
| `--force-update` | Overwrite items even if edited in Bitwarden since the last run (default: protect such edits) | `KP2BW_FORCE_UPDATE` |
| `--include-oversize-secrets` | Offload over-limit secret fields[^offload] to a `.txt` attachment instead of dropping them (default: off) | `KP2BW_INCLUDE_OVERSIZE_SECRETS` |
| `--uri-match MODE` | Match mode for plain URLs: `default`(default, account default)/`domain`/`host`/`startswith`/`exact`/`regex`/`never` | `KP2BW_URI_MATCH` |
| `--interpret-uri-syntax` | Honor KeePassXC quote/wildcard URL syntax on additional URLs (default: on; `--no-…` for literal) | `KP2BW_INTERPRET_URI_SYNTAX` |
| `--migrate-uris` | Upgrade existing items: re-fold legacy `KP2A_URL*`/`AndroidApp` fields into login URIs, then exit (no KeePass) | `KP2BW_MIGRATE_URIS` |
| `--report-uris SOURCE` | Print a read-only URI collision report (`keepass` or `bitwarden`) and exit; lists registrable domains with multiple hosts | `KP2BW_REPORT_URIS` |
| `--strip-ids` | Finalize: remove the `KP2BW_ID` dedup stamp from migrated items, then exit (no migration; no KeePass db) | `KP2BW_STRIP_IDS` |
| `--strip-ids` | Finalize: remove the `KP2BW_ID`/`KP2BW_SYNC` stamps from migrated items, then exit (no migration; no KeePass db) | `KP2BW_STRIP_IDS` |
| `-y, --yes` | Skip the Bitwarden CLI setup confirmation prompt | `KP2BW_YES` |
| `-v, --verbose` | Verbose output | `KP2BW_VERBOSE` |
| `-d, --debug` | Debug output — includes third-party library logs | `KP2BW_DEBUG` |
Expand Down Expand Up @@ -142,6 +143,13 @@ Too many subdomains autofilling together? Under base-domain matching, every logi
registrable domains with multiple hosts — so you can see which entries pile up and switch those to **Host** match (or
flip your Bitwarden account's default URI match detection to Host). It changes nothing; it just lists.

Because kp2bw is KeePass-authoritative, a re-run would normally overwrite any difference on an existing item — quietly
undoing a title, note or URI you fixed in Bitwarden. It doesn't: every item kp2bw writes carries a `KP2BW_SYNC` content
signature, and a re-run that finds an item's current content no longer matching that stamp knows *you* edited it
(kp2bw's own writes restamp, so they never self-trip) and **preserves your edit**, listing it as `protected` in the
summary. Pass `--force-update` (env `KP2BW_FORCE_UPDATE`) to make KeePass win regardless. Unchanged re-runs stay
idempotent, and items imported before this existed are adopted normally on their first re-run.

Every migrated item carries a plain-text `KP2BW_ID` custom field — the KeePass UUID kp2bw uses to match entries on
re-runs so nothing duplicates. Once you're satisfied the migration is complete and you're ready to fully adopt
Bitwarden, run
Expand Down
25 changes: 19 additions & 6 deletions src/kp2bw/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,13 +78,26 @@ src/kp2bw/
failing that it creates a new item. This stops distinct same-titled entries from collapsing onto one item (data loss)
and keeps re-runs idempotent across title/folder edits. The legacy adoption is a one-time path for vaults imported
before stable identity.
- `--strip-ids` (`KP2BW_STRIP_IDS`, default off) is the finalize-mode inverse of the stamp above: it short-circuits
- Manual-edit protection (issue #30, `--force-update` / `KP2BW_FORCE_UPDATE`, `Converter(force_update=...)` →
`self._force_update_all`): every written item carries a `KP2BW_SYNC` plain-text field holding
`_content_signature(item)` — a sha256 over exactly the surface `_content_differs` compares (name, notes,
`_fields_signature` with managed stamps excluded, `_login_signature`), stamped in `_add_bw_entry_to_entries_dict`,
read by `bw_serve.item_kp2bw_sync`, and itself excluded from the signature (in `_MANAGED_FIELD_NAMES`) so it never
causes a spurious diff. `_reconcile_existing_item` computes `_is_user_modified(existing)` =
`stamp != _content_signature(existing)`; when content differs **and** the item was user-edited **and** not
`force_update`/`_force_update_all`, it returns `"protected"` (no PUT, attachments skipped) instead of clobbering.
kp2bw's own writes restamp, so they never self-trip; an unstamped (legacy/first-run) item returns `False` and updates
normally to establish the stamp. The signature mechanism is deliberate over comparing Bitwarden's `revisionDate` to a
client timestamp — the server bumps `revisionDate` *after* kp2bw generates a stamp, so a timestamp compare would
false-trip every kp2bw-written item. The `protected` count is reported in the summary. Tests: `protect_edits_test.py`.
- `--strip-ids` (`KP2BW_STRIP_IDS`, default off) is the finalize-mode inverse of the stamps above: it short-circuits
migration entirely (`main()` returns before any KeePass read or password prompt) and only touches Bitwarden, removing
the `KP2BW_ID` field from every in-scope item via `_run_strip_ids` (`cli.py`) → `strip_field_from_items`
(`bw_serve.py`, one full `update_item` `PUT` per stamped item). Scope mirrors a migration (`-o`/`-c`), so only items
kp2bw could have stamped are touched. It is **irreversible** and degrades future re-runs (they fall back to
`(folder, name)` matching), so an interactive run confirms first (skippable with `-y`/`KP2BW_YES`); a declined prompt
exits `0` (clean abort), Ctrl+C exits `130`. Re-runnable: a second pass finds nothing.
the `KP2BW_ID` **and** `KP2BW_SYNC` fields from every in-scope item via `_run_strip_ids` (`cli.py`) →
`strip_field_from_items(*field_names)` (`bw_serve.py`, one full `update_item` `PUT` per stamped item). Scope mirrors a
migration (`-o`/`-c`), so only items kp2bw could have stamped are touched. It is **irreversible** and degrades future
re-runs (they fall back to `(folder, name)` matching), so an interactive run confirms first (skippable with
`-y`/`KP2BW_YES`); a declined prompt exits `0` (clean abort), Ctrl+C exits `130`. Re-runnable: a second pass finds
nothing.
- `--metadata` (default on) folds the KeePass metadata Bitwarden has no native slot for — **tags and expiry** — into a
single readable **YAML** `KP2BW_META` text field (`_build_metadata_field`, via PyYAML
`safe_dump(allow_unicode=
Expand Down
48 changes: 36 additions & 12 deletions src/kp2bw/bw_serve.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,14 @@
# collapse onto one item, and re-runs stay idempotent across title/folder edits.
KP2BW_ID_FIELD_NAME: str = "KP2BW_ID"

# Hidden custom-field name carrying a content signature of what kp2bw last wrote
# to the item -- the basis for protecting Bitwarden-side manual edits on re-run.
# A re-run that finds the item's current managed content no longer matching this
# stamp knows a *user* edited it (kp2bw's own writes restamp it), and preserves
# the edit instead of clobbering it. Excluded from the content diff, like
# KP2BW_ID, so it never makes a re-run look "changed".
KP2BW_SYNC_FIELD_NAME: str = "KP2BW_SYNC"


def item_kp2bw_id(item: BwItemResponse) -> str | None:
"""Return *item*'s KeePass-UUID stamp, or ``None`` if it is unstamped.
Expand All @@ -104,6 +112,19 @@ def item_kp2bw_id(item: BwItemResponse) -> str | None:
return None


def item_kp2bw_sync(item: BwItemResponse) -> str | None:
"""Return *item*'s ``KP2BW_SYNC`` content-signature stamp, or ``None``.

Absent on legacy items and on anything kp2bw has not written since the
feature shipped; such items are treated as un-protected (the next write
establishes the stamp). See :meth:`Converter._is_user_modified`.
"""
for field in item.get("fields") or []:
if field.get("name") == KP2BW_SYNC_FIELD_NAME:
return field.get("value") or None
return None


class StripResult(NamedTuple):
"""Outcome of a :meth:`BitwardenServeClient.strip_field_from_items` pass.

Expand Down Expand Up @@ -985,16 +1006,17 @@ def update_item(self, item_id: str, item: BwItemResponse) -> None:
self._request("PUT", f"/object/item/{item_id}", json_body=body)
logger.log(VERBOSE, f"Updated item {item.get('name', '?')!r} ({item_id})")

def strip_field_from_items(self, field_name: str) -> StripResult:
"""Remove a custom field from every in-scope item that carries it.
def strip_field_from_items(self, *field_names: str) -> StripResult:
"""Remove the named custom field(s) from every in-scope item carrying any.

The finalize step for users adopting Bitwarden: drops kp2bw's
``KP2BW_ID`` dedup stamp once a migration is trusted, leaving clean
items behind. Scope mirrors a migration -- the configured
organisation/collection when set, otherwise the personal vault -- so
only items kp2bw could have stamped are touched. Items lacking the
field are skipped; each match is rewritten via a full :meth:`update_item`
``PUT``. Returns the scanned/stripped counts for the caller to report.
The finalize step for users adopting Bitwarden: drops kp2bw's managed
stamps (``KP2BW_ID`` dedup key and the ``KP2BW_SYNC`` edit-protection
signature) once a migration is trusted, leaving clean items behind.
Scope mirrors a migration -- the configured organisation/collection when
set, otherwise the personal vault -- so only items kp2bw could have
stamped are touched. An item carrying none of *field_names* is skipped;
each match is rewritten once via a full :meth:`update_item` ``PUT`` that
drops every named field present. Returns the scanned/stripped counts.

The strip itself is re-runnable (a second pass finds nothing), but it is
**irreversible** and degrades future migrations: the stamp is the stable
Expand All @@ -1004,22 +1026,24 @@ def strip_field_from_items(self, field_name: str) -> StripResult:
It is therefore a deliberate final step, gated by a confirmation in the
CLI (skippable with ``-y`` for callers who know what they want).
"""
targets = frozenset(field_names)
items = self.list_items(
organization_id=self._org_id,
collection_id=self._collection_id,
)
stripped = 0
for item in items:
fields = item.get("fields") or []
if not any(field.get("name") == field_name for field in fields):
if not any(field.get("name") in targets for field in fields):
continue
item["fields"] = [
field for field in fields if field.get("name") != field_name
field for field in fields if field.get("name") not in targets
]
self.update_item(item["id"], item)
stripped += 1
logger.info(
f"Stripped {field_name} from {stripped} of {len(items)} scanned items"
f"Stripped {', '.join(field_names)} from {stripped} of "
f"{len(items)} scanned items"
)
return StripResult(scanned=len(items), stripped=stripped)

Expand Down
Loading
Loading