Allow claims, scopes, or user metadata with JWT / Session cookies

[badlydrawnrob]

A JWT contains a payload of some kind, with the most basic info being the user.id. It's also very helpful to have information about a current user in the frontend as storable json, such as claims (eg: a name), preferences (e.g: light/dark theme), scopes, or in my case, using the shortUUID of a user rather than it's incremental Int id (for security and preventing data scraping).

These would get stored in localStorage or a cookie, depending on the authentication method. Having user data readily available saves having to ping the database whenever we need info, and currently I don't think I'd be able to use the default auth methods (from what I understand they only return basic data about the user?).

Auth0's JWT looks something like this:

{
  "endpoint": "https://endpoint.auth0.com/",
  "userid": "authmethod|1234",
  "iat": 1759761022,
  "exp": 1759768222,
  "scope": "openid email update:current_user _metadata",
  "clientid": "yzMIDmaXAW"
}

You then have to ping a getProfile url to grab user data.

{ app_metadata = Nothing, email = "email@email.com", email_verified = False, name = Nothing, nickname = Nothing, picture = Nothing, user_id = "1234", updated_at = Nothing, user_metadata = Just { json = "light theme", prefs = ["one","two","three"] } }

I'm not sure what's the best practice for JWTs and Session cookies, but ideally a feature that combines these two things into one API call, that we can extend with user details we'd need. I'm not a fan of Auth0's API or their docs, which seem overly complicated for authentication.

If other Piccolo users are anything like me they're not confident with security (or prefer to outsource the task), and "subclassing the class and implementing it yourself" isn't really an option.

Seems like a handy addition for frontend apps to me?

[Skelmis]

I'd agree, the addition of custom claims would be very handy. TBH I haven't looked at JWT's in Piccolo prior to the original question as I just use sessions.

[badlydrawnrob]

@Skelmis Cool. It seems you're customising your sessions if your MFA template is anything to go by. My main point is not everybody has a computer science degree (me included) to help them understand how to safely and securely implement certain function calls or concepts without books, tutorials or examples. I'm seeing more and more people vibe coding and leaning heavily on Ai and frameworks (programming is changing for good or bad, e.g: a friend of mine works for Nuxt which has just been bought out by Vercel aka v0) to build their startups. It's quite amazing what can be done, but it's leaving apps vulnerable.

So in my opinion, it's nice if senior guys make this easier for us where possible. Auth0 and AWS style docs are just awful, overly academic/complex/verbose, zero fun to work with: I think we can do better! 100% understand Piccolo and open source software is free and we can only expect so much, but I don't have the time (or the brains) for that stuff 😔. I'm pretty sure I'm not the only one!

So yeah JWTs are really cool (as are sessions) but as an average programmer I don't know what I'm doing here. I just want to write working code I can depend on and lean on other's experience. I'm doing my part, too :)

I'll get back to trying other parts of Piccolo, it's looking promising!

[sinisaos]

@badlydrawnrob I think it is hard to expect changes to the basic JWT authentication that Piccolo has (Piccolo API with FastAPIWrapper, MFA and session auth are mostly used for Piccolo Admin) because someone has to do it and it has nothing to do with the ORM.

So in my opinion, it's nice if senior guys make this easier for us where possible. Auth0 and AWS style docs are just awful, overly academic/complex/verbose, zero fun to work with: I think we can do better! 100% understand Piccolo and open source software is free and we can only expect so much, but I don't have the time (or the brains) for that stuff 😔. I'm pretty sure I'm not the only one!

If you need custom FastAPI authentication, you have to do it yourself (you have to find the time and skills for it) or use some paid service. I'm sorry to say it, but I find it hard to believe that someone will write the code for you, but I'll try. After a short search online I found this library that I think meets your needs (custom claims etc.). Here is a working example of using it with Piccolo ORM. You can modify it however you want.

Example code

import asyncio
import uuid
from typing import Any

import uvicorn
from fastapi import Depends, FastAPI
from fastapi.responses import JSONResponse
from fastapi_jwt_auth3.jwtauth import (
    FastAPIJWTAuth,
    JWTPresetClaims,
    KeypairGenerator,
    generate_jwt_token,
)
from jwcrypto import jwk
from piccolo.apps.user.tables import BaseUser
from pydantic import BaseModel, ConfigDict, EmailStr


# Define the token claims to be projected to when decoding JWT tokens
class TokenClaims(BaseModel):
    model_config = ConfigDict(extra="forbid")

    user_id: int
    username: str
    email: EmailStr
    iss: str
    aud: str
    exp: int
    sub: str
    iat: int
    jti: str
    # You can pass whatever you want here, e.g. roles: str or roles: list[str]
    # for authorization etc. and add that to the claims in the login route
    # role: str
    # roles: list[str]


# Payload for our logins
class LoginIn(BaseModel):
    username: str
    password: str


# FastAPI app instantiation
app = FastAPI(title="FastAPI JWT Auth Example")

# For the purpose of this example, we will generate a new RSA keypair
private_key, public_key = KeypairGenerator.generate_rsa_keypair()

# Create a JWK key from the public key
jwk_key = jwk.JWK.from_pem(public_key.encode("utf-8"))
public_key_id = jwk_key.get("kid")

jwt_auth = FastAPIJWTAuth(
    algorithm="RS256",
    base_url="http://localhost:8000",
    secret_key=private_key,
    public_key=public_key,
    public_key_id=public_key_id,
    issuer="https://localhost:8000",
    audience="https://localhost:8000",
    expiry=60 * 15,
    refresh_token_expiry=60 * 60 * 24 * 7,
    leeway=0,
    project_to=TokenClaims,
)

jwt_auth.init_app(app)


# Protected route
@app.get("/protected")
async def current_user(claims: TokenClaims = Depends(jwt_auth)) -> TokenClaims:
    return claims


# Login route
@app.post("/login")
async def login(payload: LoginIn) -> dict[str, Any]:
    # Use Piccolo login method to compare the username and hashed password
    # with the payload data and obtain the registered user id
    user = await BaseUser.login(
        username=payload.username, password=payload.password
    )
    # get user
    result: Any = await BaseUser.objects().where(BaseUser.id == user).first()

    # preset claims
    preset_claims = JWTPresetClaims.factory(
        issuer=jwt_auth.issuer,
        audience=jwt_auth.audience,
        expiry=jwt_auth.expiry,
        subject=str(uuid.uuid4()),
    )

    # additional claims
    claims = {
        "user_id": result.id,
        "username": result.username,
        "email": result.email,
    }

    # generate token
    token = generate_jwt_token(
        header=jwt_auth.header,
        secret_key=jwt_auth.secret_key,
        preset_claims=preset_claims,
        claims=claims,
    )

    # This is optional but good practice to generate refresh token
    refresh_token = jwt_auth.generate_refresh_token(access_token=token)

    return {"access_token": token, "refresh_token": refresh_token}


# Public route
@app.get("/")
async def public() -> JSONResponse:
    return JSONResponse({"data": "Public Page"})


async def main():
    # Tables creating
    await BaseUser.create_table(if_not_exists=True)

    # Creating example user
    if not await BaseUser.exists().where(BaseUser.email == "admin@test.com"):
        user = BaseUser(
            username="piccolo",
            password="piccolo123",
            email="admin@test.com",
            admin=True,
            active=True,
            superuser=True,
        )
        await user.save()


if __name__ == "__main__":
    asyncio.run(main())

    uvicorn.run(app, host="127.0.0.1", port=8000)

I hope this helps and sorry if I missed your point.

[badlydrawnrob]

@sinisaos Appreciate the response and the example, I was going to say "I'm going to have to figure out how to extend JWTPresetClaims" but you figured it out for me! I didn't realise there was a custom claims part. That package might work as a temporary measure.

I might've overshared in my previous post (and this one!), but it's a general frustration with documentation, language, and package design with programming in general. One of the reasons I picked Piccolo as an option is that ORMs like Ormar are (a) an abstraction of an abstraction (SQLAlchemy) and (b) have documentation and language design which is needlessly complex. The same could be said for security and other low-level programming details, depending on the language. Piccolo's ORM features seemed far simpler and easy to read in comparison.

I'm sorry to say it, but I find it hard to believe that someone will write the code for you, but I'll try [...] because someone has to do it and it has nothing to do with the ORM.

I wasn't expecting someone to code up a JWT solution for free (I'm happy living with my limitations and outsourcing), but it seems like Piccolo isn't "just" an ORM. It publicly exposes an authentication API and BaseUser, so I figured it was aiming for a lightweight but more robust solution to other ORMs (a bit like iceaxe has a wider ecosystem Mountaineer). We wouldn't be having this conversation if it were a similar design to Peewee, for example!

I think it's great Piccolo is focused on doing one thing really well (data management) but ... I raised the issue (and mentioned it in another post) because the auth feels like an advanced feature with only a bit of hand-holding and there's no external resources to walk me through it. I don't feel as confident in Python to hack away and make mistakes as the error messaging is cryptic compared to Elm Lang!

TL;DR I wouldn't let a junior or vibe coder without deep understanding of security measures loose on this kind of work. Luckily I have a mentor I can lean on, but not everyone is so lucky: routes and database stuff is easy enough to figure out, but I'm not (and shouldn't be) confident in my code when a user's security is at stake. The example you've given is great, but does it "tightly" integrate with Piccolo? Having a mature/stable/integrated way of setting up your app seems to be the goal of Piccolo and I'm not so sure it does (but BaseUser.login is brilliant). Authentication built-in also differentiates Piccolo from other ORM options, so I thought this kind of work would benefit the protect too.

If issues isn't the right place for these kind of philosophical questions, let me know where it's best to share ideas or chat. Sorry if I got my wires crossed about the goals of the Piccolo project, perhaps you could make it clearer on the homepage/docs what it's goals are not?

Again, thanks for the code. I wouldn't be spending all this time writing if I wasn't interested in Piccolo!

[sinisaos]

Appreciate the response and the example, I was going to say "I'm going to have to figure out how to extend JWTPresetClaims" but you figured it out for me! I didn't realise there was a custom claims part.

No worries, glad I can help.

I might've overshared in my previous post (and this one!), but it's a general frustration with documentation, language, and package design with programming in general. One of the reasons I picked Piccolo as an option is that ORMs like Ormar are (a) an abstraction of an abstraction (SQLAlchemy) and (b) have documentation and language design which is needlessly complex. The same could be said for security and other low-level programming details, depending on the language. Piccolo's ORM features seemed far simpler and easy to read in comparison.

I also think Piccolo is great (but it's not as popular as I would like) as a query builder and ORM, but the whole ecosystem (admin and API) is even better because it provides an easy way to build a web application faster (similar to Django but async from start). As far I know, almost every other ORM in the Python async world (except Tortoise ORM and iceaxe - at first glance it looks very similar to Piccolo) is more or less an abstraction over SQLAlchemy (which is the most popular Python ORM outside of Django) and then the question arises, why not use SQLAlchemy directly as a very mature (with lots of features), but complicated library.

I wasn't expecting someone to code up a JWT solution for free (I'm happy living with my limitations and outsourcing), but it seems like Piccolo isn't "just" an ORM. It publicly exposes an authentication API and BaseUser, so I figured it was aiming for a lightweight but more robust solution to other ORMs (a bit like iceaxe has a wider ecosystem Mountaineer). We wouldn't be having this conversation if it were a similar design to Peewee, for example!

Piccolo, like any library, has some limitations. I was just trying to be honest and say that if you need some custom solutions, you have to build them yourself because no library has a solution for everything. I don't know anything about Mountaineer, but I see that it doesn't have any built-in authentication.

The example you've given is great, but does it "tightly" integrate with Piccolo?

I must admit I don't understand this. In what sense is it "tight" integration? In the Piccolo ecosystem, then no. That example is a solution that can be integrated with any ORM or db driver. The only difference would be in the way the user's identity is verified on login. If you want "tight" integration, then you have to use Piccolo JWT which you said doesn't meet your needs.

Authentication built-in also differentiates Piccolo from other ORM options, so I thought this kind of work would benefit the protect too.

I am not a security expert, but I know that security is a big and complex topic and there are no easy solutions. Piccolo's best and most complete authentication method is session auth and is listed in the documentation.

If issues isn't the right place for these kind of philosophical questions, let me know where it's best to share ideas or chat. Sorry if I got my wires crossed about the goals of the Piccolo project, perhaps you could make it clearer on the homepage/docs what it's goals are not?

For that you have to wait for Piccolo author (I'm just a small contributor) because he knows best which direction the library is going.

[badlydrawnrob]

I also think Piccolo is great (but it's not as popular as I would like)

I've come to accept that (in the words of Iggy Pop) "not everything that's good will be big, and not everything that's big is any good". I can't find the original quote, but this song tells it! Same with languages (Elm), the arts, products (AirUp: that bottle is useless in my eyes but its marketing sells). The creator of Elm has some great talks on community building and success that might be helpful to the Piccolo team.

Yeah, I don't understand why everyone's building on top of SQLAlchemy rather than database engine libraries which I imagine is more efficient. SQLAlchemy seems like a bloated mess to me.

I was just trying to be honest and say that if you need some custom solutions, you have to build them yourself

Fair point. Perhaps I'll get in touch with the main author on authentication and see how he feels about things. In the meantime you've given me a good option.

Does it "tightly" integrate with Piccolo? [...] I must admit I don't understand this.

I mean well documented and stable that's part of the Piccolo ecosystem, with predictable changes. Yeah you're right, I guess there's always a compromise.

I know that security is a big and complex topic and there are no easy solutions

Tell me about it! I've been programming on and off for 10 years and I still don't feel comfortable with lower level stuff. I've come to the conclusion to stick to a narrower (lighter) segment of coding and leave the rest to others. I don't do it for my day job and it's easy to get rusty. Learning programming is boundless.

you have to wait for Piccolo author

Is Daniel the main contributor?

[sinisaos]

Yes, @dantownsend is author and maintainer of Piccolo.