(very much a work in progress)
- Introduction
- Assumptions
- Requirements
- Use cases
- Developer machine
- add a key/value pair
- list secrets
- create a .env file
- export a secret to another user
- explore
- Server
(very much a work in progress)
How do you communicate the set of secrets (passwords, API keys, etc.) that your code needs to run? You can't just write them in the code, because that would expose them to anyone who can read the code. You can't just send them in an email, because that would expose them to anyone who can read your email. You can't just write them on a sticky note, because that would expose them to anyone who can read your sticky note.
There are many ways to store and use secrets, e.g.:
-
Environment variables: It is popular, in e.g. kubernetes and javascript, to read secrets from environment variables. For development these are stored in .env files that are not checked into git/svn. These files contain the secrets in plain-text. Transferring secrets to a new server/devloper is a manual process where you create a new file and somehow get access to the secrets.
-
Key vaults: E.g. Secureden or HashiCorp Vault (can be self hosted). These are usually expensive or complex to set up - or both. Key vaults usually have an API that you can use to get the secrets (setting this up is also expensive/complex/both).
-
Secret management services: These are hosted key vaults, e.g. AWS Secrets Manager, Google Secret manager, Azure Key Vault, or hosted HashiCorp Vault. There is always an associated api to read the secrets. This can incur significant costs for large numbers of secrets and/or high usage.
-
Encrypted files: This is usually a key/value file (.json/.yaml), where only the values are encrypted (e.g. SOPS. Every developer must have the same key to be able to use the file, but the file can be stored in git/svn.
Seeqret stores "passwords", encrypted, in a sqlite database. On windows this database is stored in an encrypted folder and permissions are removed from anyone but the developer.
A cli, seeqret, is provided to add/remove/edit/etc. the key/value pairs.
When transferring secrets to another developer they are first encrypted with a public/private key-pair, thus ensuring encryption both at rest and in transit.
An api is provided for use in Python (seeqret.get("key")). This api can do 7000+ fetches/sec on my machine, and
is faster than a fixed record file format.
Please read and understand the docstring in __init__.py before using!
We make the following assumptions:
- https is secure
- the encryption algorithms are secure
- encrypted directories (Windows) are secure (
seeqretwill use the following receipe when setting up its data directory):attrib +I %1 icacls %1 /grant %USERDOMAIN%\%USERNAME%:(F) /T icacls %1 /inheritance:r cipher /e %1
- encrypted private directories (Ubuntu) are secure.
0600(read/write by owner only) directories (linux) are safe provided the user is not compromised.
- different users/systems should have access to different subsets of the secrets.
- the secrets should not exist in plain-text when they are not used (e.g. a database password is only used when logging into the database - it shouldn't exist in memory outside of this process1).
- a subset of secrets needs to be shared with new developers in a secure way.
- there should be a command line utility (
seeqret) toseeqret add key <key> <value>: set a secretseeqret get <key>: get a secretseeqret export <user> <keys..>export a subset of the secrets for transmission (e.g. by email) to a new developerseeqret import <keys..>import keys received by e.g. email
- and a library/API to
seeqret.get(key): get a secret (this should be a fast O(1) operation)
- the secrets should be easy to update2.
- it should be possible to backup the secrets.
- (bonus): the secrets should be easy to rotate3.
- (bonus): the secrets should be auditable4.
- Starting from scratch: How do you set up the secrets system?
- Inviting users: How do you invite users and how do you communicate the secrets to them?
- Adding a secret: How do you communicate a new secret to the users?
- Updating a secret: How do you communicate an update to a secret?
- New user: How does a new user get access to the secrets?
- Backup: How do you backup the secrets and what is needed to restore the backup?
- Developer leaves: How do you revoke access to the secrets for a developer that leaves?
You must have at least Python 3.10+ to run seeqret!
To install the vault under c:\home (the directory must not be on a network drive or inside a version controlled directory):
pip install seeqret
seeqret init c:\home --email <your-email>
seeqret usersseeqret add key KEY secret --app=myapp --env=devList all dev secrets that start with POSTGRES_:
seeqret list -f :dev:POSTGRES_*See the filter docs for more info (https://thebjorn.github.io/seeqret/filter-strings/).
❱ seeqret add key FOO bar --env dev
Adding a new key: FOO, value: bar, app: *, env: dev
❱ cat env.template
# all FOO* keys from the dev environment
:dev:FOO*
❱ seeqret env
FOO="bar"
Created .env file with 1 secrets
❱ cat .env
FOO="bar"Have the other user run
❱ seeqret owner
┏━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ username ┃ email ┃ publickey ┃
┡━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ bjorn │ [email protected] │ MBiGKmtpckXspJkmijIPXd8GrIAgAdLOoM4pZNOyDzw= │
└───┴──────────┴─────────────────┴──────────────────────────────────────────────┘and send the information to you (you should directly verify the public key).
Add the other user as a known person to your vault:
❱ seeqret add user -h
Usage: seeqret add user [OPTIONS]
Add a new user to the vault from a public key.
Options:
--username TEXT Username to record
--email TEXT Email for the user
--pubkey TEXT Public key for the user
-h, --help Show this message and exit.
❱ seeqret add user --username bjorn --email [email protected] --pubkey MBiGKmtpckXspJkmijIPXd8GrIAgAdLOoM4pZNOyDzw=
...❱ seeqret export bjorn -f :dev:FOO
{
"from": {
"email": "[email protected]",
"pubkey": "MBiGKmtpckXspJkmijIPXd8GrIAgAdLOoM4pZNOyDzw=",
"username": "bjorn"
},
"secrets": [
{
"app": "*",
"env": "dev",
"key": "FOO",
"type": "str",
"value": "W4Fuverw5Mw//ccvYVm40ysVdSxQ4ZwpNTkBuyXtPvtmIQLkd2cl9veG+w=="
}
],
"signature": "0e34f5934979a9deb02ab0c584b39a4d128188df36ec61a0d591ffd9e4e65f70",
"to": {
"email": "[email protected]",
"pubkey": "MBiGKmtpckXspJkmijIPXd8GrIAgAdLOoM4pZNOyDzw=",
"username": "bjorn"
},
"version": 1
}The other user can load this file with
> seeqret load -f message.jsonNOTE: only the intended recipient can decrypt/load the file.
There command serializer makes it convenient to send a "few" secrets e.g. in chat:
❱ seeqret export bjorn -f :dev:FOO -s command
seeqret load -ubjorn -scommand -v1::84efd:*:dev:FOO:str:jsPI3HbN7zLNpogELFbOYZaFR4wFGs2+f6m6ZJCQ2ey88fovih5ZHzIESg==the recipient only has to copy and paste the command into their terminal to import the secret.
NOTE: only the intended recipient can import the secret with this command line.
List all commands:
> seeqret info
cli
add Add a new secret, key or user
file Add a new FILE to the vault (.env file format).
key Add a new NAME -> VALUE mapping.
user Add a new user to the vault from a public key.
backup Backup the vault to a file.
edit Edit a secret or user in the vault.
value Update the secret (FILTER) to the value (VAL)
env Read filters from env.template and export values from the vault to an .env file.
export Export the vault to a user (use `seeqret load` to import)
get Get the value of a secret (specified by FILTER).
info List hierarchical command structure.
init Initialize a new vault in DIR
keys List the admins keys.
list List the contents of the vault
load Save exported secrets to local vault.
owner List the owner of the vault
rm Remove a secret or user from the vault.
key Remove a secret from the vault.
serializers List available serializers.
server Server commands.
init Initialize a server vault
upgrade Upgrade the database to the latest version
users List the users in the vaultTo get information on any command add the -h flag, e.g.:
> seeqret export -h
Usage: seeqret export [OPTIONS] TO
Export the vault TO a user (use `seeqret load` to import)
Options:
-f, --filter TEXT A seeqret filter string (see XXX)
-s, --serializer TEXT Name of serializer to use (`seeqret serializers` to
list).
-o, --out TEXT Output file (default: stdout).
-w, --windows Export to windows format.
-l, --linux Export to linux format.
-h, --help Show this message and exit.TBD
seeqret server init...
Footnotes
-
This is a defense-in-depth measure (in case the sanitizer fails to remove the secret from any traceback/logs/etc.) ↩
-
Updating means manually changing the secret (both in the storage and the service it protects), e.g. when a password expires/is compromised/a devloper leaves/etc. ↩
-
Rotation is the process of periodically updating a secret. Ideally this is an automatic process that e.g. changes both the secret storage and the service it protects. ↩
-
Auditing means checking who has access to the secrets, which secrets were accessed, and when. ↩