Better support for jailed unit testing

[ethanngoh]

When constructing a Model class, there are network calls made to AWS, for example: Model._get_meta_data.

It would be nice to have a 'unit testing' context that wouldn't make any network calls, and also have functionality to assert that certain actions were performed on the model context (i.e get, save) during an actual unit test.

[yedpodtrzitko]

Hi,
as a workaround I made a class mixin which you can use in combination with DynamoDBLocal to mock network calls: https://github.com/yedpodtrzitko/pynamock

[jmphilli]

@yhgoh88 -- if @yedpodtrzitko's stuff works for you let him know, seems like something we may want to merge in cc @garrettheel

[jneves]

@jmphilli I can confirm that @yedpodtrzitko's solution works with a minor fix: the teardown method needs to be renamed to tearDown.

[bradodarb]

I'm really liking this library, but it could be more test friendly.

@yedpodtrzitko 's solution is ok for integration tests but there doesn't seem to be much support for unit tests.. Unless I'm missing something the above solution seems like a bit of unnecessary redirection as we can just point the Model.Meta.host to an instance of Dynamo Local.

When I want to gain unit test coverage for a pynamodb model I patch the following on my test case:

class MetaTableMock():
    range_keyname = 'range_id'

@patch('pynamodb.connection.base.get_session', MagicMock())
@patch('pynamodb.connection.table.Connection', MagicMock())
@patch('pynamodb.models.Model._meta_table', MetaTableMock())
class TestPynamoModel(unittest.TestCase):

[joarleymoraes]

An alternative I have recently tested and worked great was the library moto (which mocks boto3 requests):

https://github.com/spulec/moto

Make sure to use mock_dynamodb2

[song-william]

@joarleymoraes do you mind posting an example of using moto with pynamodb? I am having trouble getting it to work. The docs do not show how to use moto with pynamodb.

[joarleymoraes]

Sure, here something to start with:

import unittest
from moto import mock_dynamodb2
from pynamodb.models import Model
from pynamodb.attributes import UnicodeAttribute

class SomeModel(Model):
    class Meta:
        table_name = 'SomeModel'
    id = UnicodeAttribute(hash_key=True)

@mock_dynamodb2
class MyTestCase(unittest.TestCase):

    def setUp(self):
        SomeModel.create_table(wait=True)
        super(MyTestCase, self).setUp()

    def tearDown(self):
        SomeModel.delete_table()
        super(MyTestCase, self).tearDown()

    def add_my_model(self):
        id = '1'
        obj = SomeModel(id)
        obj.save()
        get_obj = SomeModel.get(id)
        self.assertEqual(obj.id, get_obj.id)

[song-william]

If in the original table definition you assigned a host:

class SomeModel(Model):
    class Meta:
        table_name = 'SomeModel'
        host = ''http://localhost:8000"
    id = UnicodeAttribute(hash_key=True)

How did you reassign it at test time so that you did not get a requests.exceptions.ConnectionError: Connection refused error?

[joarleymoraes]

I don't have host assignment on my Meta class, so it's using whatever default pynamodb assigns.

[marcelloromani]

@joarleymoraes thanks so much for the code example

@song-william I also don't have any 'host' assignment in the Model classes under test, which means boto3 will build a "real" dynamodb endpoint. In fact, if you initialise the model with an invalid region you get an error.
Despite that, no actual connection to AWS is made. (I guess moto mocks the http level).