Skip to content

Returning objects to Python from Rust should use __del__ for cleanup rather than context manager #60

Open
@scooter-dangle

Description

2018-05-02 UPDATE: Per discussion below, issue is that Python FFI object example only shows the deterministic, with ... as ...: approach whereas the Ruby (and others?) example only provides the non-deterministic approach.


I've seen blogs advise against using __del__ for general cleanup in Python, but for de-allocation, it more closely maps to what we want than context managers. For instance, with the current code's use of context management, the only way to make a collection of ZipCodeDatabases would be to explicitly nest with ... as statements. As well, the current example complicates persisting a Rust object that is expensive to initialize but cheap to use, whereas __del__ supports this while still freeing the resources when the reference count drops to zero.

Proposed Python code:

class ZipCodeDatabase:
    def __init__(self):
        self.obj = lib.zip_code_database_new()

    def __del__(self):
        lib.zip_code_database_free(self.obj)

    def populate(self):
        lib.zip_code_database_populate(self.obj)

    def population_of(self, zip):
        return lib.zip_code_database_population_of(self.obj, zip.encode('utf-8'))

database = ZipCodeDatabase()
database.populate()
pop1 = database.population_of("90210")
pop2 = database.population_of("20500")
print(pop1 - pop2)

With that, we can also do this (which—granted—isn't super useful in this particular case):

databases = [ZipCodeDatabase() for _ in range(3)]

I can think of two drawbacks to using __del__ over context management:

  1. Memory cleanup in the presence of reference cycles in Python is less predictable.
  • This is already an issue in Python that developers will need to be aware of. The fact that some of the objects involved will be cleaned up by calling a Rust function doesn't exacerbate the problem, in my opinion.
  1. The Rust cleanup function might not run immediately upon the variable going out of scope
  • Since the cleanup function is to avoid a memory leak, I don't see how this is any different than any other allocated object in Python. Additionally, we know in the Rust world to not rely on drop being called to guarantee properties like safety, so I think __del__ maps better to the concept of the drop function it will be calling.

Aside from __del__ reducing the restrictions on using Rust objects within Python, it's also conceptually simpler. And to an audience that's already learning Rust, it should be easier to teach in conjunction with teaching Drop.

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions