Since jj automatically logs all filesaves, can it pose a risk of exposing secrets accidentally, even if the file is eventually discarded?

[p-avital]

Hello,

The title says it all. Let's go through an example to highlight specifically the situation I'm referring to:

cd repo # repo was previously `jj init`'d
echo "THIS IS A SECRET, I DON'T WANT IT IN REFLOGS" > key.pem
# `git add -A` now would leak `key.pem` to reflogs. I suppose `jj` would leak it to its ops log?
echo "key.pem" >> .gitignore # Whoops, don't want to accidentally leak my key to reflog
# From now on, `git add -A` is safe, but am I already doomed because of `jj`?
jj new # Or maybe I'm wrong in my assumption, and `jj` shakes off the ops regarding `key.pem`?

Suppose I did as such, would key.pem:

• Be safe and sound?
• Be exposed in the reflogs?
  • If so, is there an "easy" way to ensure key.pem is no longer exposed?
• What if I had deleted key.pem before that jj new instead of .gitignore-ing it?
• What about the case where I copied and then erased a similar secret into a tracked file? Would the version of the file that contained the secret still be reachable in some way?

I've just started using jj, and I'm loving it. But a big part of why I love it is that it's essentially "Stacked Branches, but automatically", which helps a lot with my inattentiveness; which I can very easily see causing the kind of goof I just described. I also want to know if I should beware people of that when I tell them about jj and how nice it feels.

Adding some keywords for future searches, as I was only able to find #323 regarding this question, and the thread is so long that I couldn't really find my answer in there: leak, expose, key, secret.

[jennings]

You can select all revsets that contain a particular file with the files function.

$ jj log -r files(key.pem)

You can also use string patterns like files(root-glob:"**/*.pem")

To find secrets inside a tracked file, you can use diff_contains("p@ssw0rd").

[p-avital]

I don't think this addresses the issue, as it only would look into visible commits.

But since I'm new to jj, these commands help grok the UI, so thanks :)

[ilyagr]

@jennings answer is good for checking that the secret file is not in one of the visible commits. For normal levels of paranoia, if the important thing is not to push the secret to a remote, that should be sufficient. You can just edit commits until all traces of the secret are gone.

For heightened levels of paranoia, you should be aware that even if commands like jj log -r files(key.pem) return nothing, the secret file is likely still recoverable by anyone with access to the local repo on your computer, using jj op log and jj log --at-op. For example, you can run jj op log --summary or jj op log --patch to see all the changes to the repo recorded in jj's history (it'll be long, you might want to combine it with grep).

But then, if the secret is merely in the .gitignore, it's right there on your local disk anyway.

Still, if you do in fact want to get rid of all copies of the file on disk, jj unfortunately does not currently provide a good way to eliminate a secret from all of the local history (or otherwise modify the op log history). The simplest reliable solution would be to push everything you want to preserve to a remote (making sure not to push the secret), and then clone a fresh repo from that remote. You might want to back up the old repo until you are sure you have everything.

[emilazy]

Does jj op abandon not work for this use case? (It wouldn’t handle predecessors in the past, but I guess the recent change to move them to the op log should solve that.)

[ilyagr]

Ah, good point, thanks for mentioning it! I forgot we had this. TBH, I'm not sure.

As you mentioned, it'll probably not work for removing old operations (if you want to keep newer operations) because of the predecessor issue. Yuya is clearly working on this; I'm guessing that his latest change is a part of that, but probably there's more to do.

---

However, as you suggested, for removing recent operations (if you just committed the secret to jj), something like the following might work (following jj op abandon --help):

jj op restore <operation ID>
jj op abandon <operation ID>..@-
jj util gc # Or possibly `jj util gc --expire now`

If I committed a huge file to the repo accidentally, I might try that. (I haven't experimented much with it) If I committed a Very Important Secret That Should Never Have Been Saved to My Hard Drive, I'm not sure I'd trust it sufficiently.

[p-avital]

The attack vector that worried me was not "My disk is compromised, can someone find it in local logs" (which I agree is very very paranoid).

Instead, it was "Does jj implicitly does git add whenever I change a file? Because if so, couldn't anyone with access to the repo (even remote versions) use git fsck and git rev-list to recover versions of files that were never in any commits".

I've run a little experiment. And here are the conclusions:

• The implicit git add does happen, and does let someone with access to the local copy of the repo find intermediate states .
• However, git push only pushes objects that correspond to a commit, shaking off the offending object which cannot be recovered from the remote.

In conclusion: unless you have some weird config where git will push all objects to your remote and not just those associated with actually committed data, jj will not leak your secrets to remote repos unless you actually commit them.

Here's the aforementioned log on how intermediate states can be found only locally:

> jj st
Working copy changes:
A secret.pem
Working copy  (@) : nmlylyvp 482aa8e1 (no description set)
Parent commit (@-): urkvputl 659548b8 main

# Edit `.gitignore` to include `secret.pem`
> jj st
Working copy changes:
A .gitignore
A secret.pem
Working copy  (@) : nmlylyvp a70ce93c (no description set)
Parent commit (@-): urkvputl 659548b8 main

> jj file untrack secret.pem
> jj st
Working copy changes:
A .gitignore
Working copy  (@) : nmlylyvp ccb403db (no description set)
Parent commit (@-): urkvputl 659548b8 main

> jj commit -m "Hopefully, no secrets leaked"
Working copy  (@) now at: ozwprnsm f787d012 (empty) (no description set)
Parent commit (@-)      : nmlylyvp 58b3abe9 Hopefully, no secrets leaked

> git rev-list --all --objects
# sparing you the full list
0430c868420655e13f5a724397dff150f8209a8c .gitignore
746055eaee7c956e1a0cbaa446b78f26a5ea3124 
f79a16b76be307186f55f52d17e7b65f7bce451c secret.pem

> git cat-file -p f79a16b76be307186f55f52d17e7b65f7bce451c
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----%

[ilyagr]

The remote versions of the repo only have the commits that the branches you pushed there and their ancestors. jj cannot and does not push the op log to git remotes (they have no way of storing it). Normally, you only push visible commits, so if your secret is not in those whenever you push, you are fine.

This will become a more serious issue once there exist remotes that could store the op log, but this is theoretical for now.

[p-avital]

Well good news everyone! Git saves the day! (No shots at jj, just saying that it looks like git's core functionality prevents the leak from spreading to remotes)

The experiment's original repo had 16 objects. However, only 6 of them were pushed to the repo, and they didn't include the object that allowed recovering secret.pem.

They also didn't show intermediate versions of a second file where I explicitly called git add multiple times between commits to see if these intermediate revisions would be recoverable.

I'm guessing some tree shaking happened somewhere on the way.

So indeed, it looks like access to the local version of the repo is still necessary to extract information that wasn't explicitly committed. This is perfectly fine under my threat model (and I believe, most reasonable models).

I'll edit the above comment to indicate that the threat is mitigated unless the attacker does have access to your local repo, and mark it as answer :)

[ilyagr]

If you want to be really careful, push with --dry-run first to check what commits would be pushed and check that your secret is not in their ancestors and/or that none of them are hidden. I think it's currently difficult but might be possible to push a hidden commit.

Maybe we should explicitly have an error if the user is pushing a hidden commit, just in case, with an option to override it. I think if people ran into it often, we would have had it by now, but hopefully I'll remember to double-check.