Assigning same index definition to multiple models

[joarleymoraes]

I have a use case for which it would be great if I could reuse an index across different models. But it seems it's not possible. Here's how to reproduce:

https://gist.github.com/joarleymoraes/9c8b2b2375d03522b276cc5b35373a5e

The error is:

pynamodb.exceptions.VerboseClientError: An error occurred (ResourceNotFoundException) on request (247af178-87b3-470a-8cd7-d88e2654948a) on table (Model2) when calling the DescribeTable operation: Cannot do operations on a non-existent table

During handling of the above exception, another exception occurred:
pynamodb.exceptions.TableDoesNotExist: Table does not exist: `Cannot do operations on a non-existent table`

It seems that, because Model2 is defined last, it overwrites the index previously assigned to Model1. So when a query issued to the index, it looks up the wrong model.

The issue is particularly hard to debug when you create the table Model2 (which is normally is the case), then the query will succeed, but return empty result set.

Not sure if this is by design. Currently, I have to duplicate index definition in order to get it working.

[chrisbrantley]

I am also running into this issue. Any updates?

[ikonst]

I'd classify the fact it's a silent error as a bug. As to whether we'd want to allow it, I don't have an opinion myself.

[chrisbrantley]

Coming back to this after 1.5+ years. Looks like this is still a problem. I did a bit of digging and it appears that when you add an instance of an Index class to a Model class the Model will MODIFY the Meta class on the Index class and set the model attribute to the Model class. The problem is that this one Meta class is shared across all instances of the Index class so whichever Model class does this last will win and leave the other Model classes in a broken state.

Of course you can fix this by duplicating the Index class but this is a hacky workaround for what, in my humble opinion, is a poor design.

One solution I could see would be to COPY the Meta class on instantiation and store in in self.meta (or something). Then the host Model class can modify THAT without effecting other instances. One issue with this is that all of the methods on Index are classmethods and don't have access to instance variables.

I can't understand the rationale for this design. Is there a benefit to forcing everything at the class level? If this is the case why do we even bother instantiating the Index at all... why not just reference the class directly?

[ikonst]

@chrisbrantley It is poor design and I'd change it exactly as you describe. Myself I found this design to be very surprising and confusing when I just started using PynamoDB.

[instil-richard]

I just discovered this today myself too. It's obvious that this was implemented without any understanding of how indexes are supposed to shared across item types. You can't even extend a base index class you have to duplicate the class everywhere you want it. For a larger app with potentially dozens and models and up to 20 indexes per dynamo table thats a lot of duplicated code.

[mjuopperi]

Seems to still be an issue. This was pretty time consuming to debug and I found this GitHub issue only after figuring the issue out by debugging...

[jonasmamona]

I've just spent quite some time trying to troubleshoot the wrong model being retrieved when querying, so this issue still exists. Solved it by duplicating the index declaration exactly as described by @chrisbrantley.