Using multiple AWS profiles

[whiskey]

I've configured multiple AWS credential profiles on my system and usually select the right one via named profiles or environment variables such das export AWS_DEFAULT_PROFILE=foo.

However, my created User Model will be fetched from my default configuration.
Model definition:

class AWSUser(Model):
    class Meta:
        table_name = 'Users'
        region = 'eu-central-1'

    id = UnicodeAttribute(hash_key=True)
    display_name = UnicodeAttribute(attr_name='displayName')
    # ...

    def __str__(self):
        return self.display_name

How can I select the correct AWS profile?

[ma-rp]

Did you find a way to do this? Im facing this issue right now too.

[brandond]

pynamodb uses botocore under the hood, which should honor environment variables like AWS_DEFAULT_PROFILE. Have you tried something like:

AWS_DEFAULT_PROFILE=alt-profile python my-pynamodb-code.py

[alastairmccormack]

It would be better if a pre-configured boto session could be passed somewhere rather than relying on the user environment.

[brandond]

@use-sparingly yeah that would be nice. The model Meta has a connection_cls property, but this (counterintuitively) is used to look up the requests session class, NOT the botocore session. It might be nice to expose this, but as far as I can tell you'd have to plumb it all the way through model.MetaModel, connection.base.MetaTable, connection.table.TableConnection, and connection.base.Connection.

If you have an immediate need and don't want to set the environment variable, you could always monkeypatch connection.base.Connection.session (globally) or set self._get_connection().connection._session within your model's __init__ (for a specific model).

[bedge]

I ran into this too. This is contorted around some child classes, but here's the gist of it:

    SESSION = Session(profile='CrossDynamodbAccess')

    MyClass._connection = MyClass._get_connection()
    MyClass._connection.connection._client = SESSION.create_client(
        'dynamodb', region_name='us-west-2')

A cleaner option would be appreciated.

[kevgliss]

More control would also be preferable in the case where you have to assume role before using pynamodb, as it stands @bedge solution works okay for the time being.

[alexanderfanz]

@brandond can you explain your comment a little more and/or write an example code? I'm lost.

If you have an immediate need and don't want to set the environment variable, you could always monkeypatch connection.base.Connection.session (globally) or set self._get_connection().connection._session within your model's __init__ (for a specific model).

[brandond]

@alexanderfanz since _session is now thread-local, I don't think it's safe to set within the model any longer. Monkeypatching is probably your best option. Something like this - either in the module that holds your model definitions as shown in this example, or somewhere else that's called once during app startup - should do the trick.

from pynamodb.models import Model
from pynamodb.attributes import UnicodeAttribute, NumberAttribute


class MyModel(Model):
    class Meta:
        table_name = 'mytable'

    group = UnicodeAttribute(hash_key=True)
    id = NumberAttribute(range_key=True)
    name = UnicodeAttribute()


def monkeypatch_connection(profile=None):
    from botocore.session import Session
    from pynamodb.connection.base import Connection

    @property
    def session(self):
        if getattr(self._local, 'session', None) is None:
            self._local.session = Session(profile=profile)
        return self._local.session

    Connection.session = session


monkeypatch_connection(profile='my-aws-profile')

[mikegrima]

In general, we would love a good way of performing a role assumption and caching credentials. I would be happy to submit a PR for this if I knew where to look.

Profiles aren't great for programmatic and dynamic credential needs.

[twkiiim]

Any updates for this issue?

[ikonst]

@garrettheel, any thoughts on that?

I would rather have an API where models aren't dealing with auth or host. For that we'd need a public API for setting it globally. A context manager which overrides things like that in thread local-storage might be a good idea too. What do you think?

[garrettheel]

I'm not sure that I fully understand the use case here, so I'd want to gather a little more information. We intentionally expose as little of the boto/botocore interface as possible so that it's possible to optimize/change this under the hood without breaking users.

Is is that we want to configure a specific AWS_DEFAULT_PROFILE in code, rather than as an environment variable? Is there a desire to apply different profiles to different model classes/instances?