KeyError: 'changed_data' in nested_admin.formsets.save_existing_objects

[scottgigante]

nested_admin.formsets.save_existing_objects seems to be accessing a field that sometimes does not exist. I'm not quite sure how to generate a MRE here, but this is the traceback from my app logs.

Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django/forms/models.py", line 1538, in to_python
value = self.queryset.get(**{key: value})
File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
raise self.model.DoesNotExist(
MyModel.DoesNotExist: MyModel matching query does not exist.

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/nested_admin/formsets.py", line 304, in save_existing_objects
pk_value = form.fields[pk_name].clean(raw_pk_value)
File "/usr/local/lib/python3.11/site-packages/django/forms/fields.py", line 198, in clean
value = self.to_python(value)
File "/usr/local/lib/python3.11/site-packages/django/forms/models.py", line 1540, in to_python
raise ValidationError(
django.core.exceptions.ValidationError: ['Select a valid choice. That choice is not one of the available choices.']

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 46, in _wrapper
return bound_method(*args, **kwargs)
File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 134, in _wrapper_view
response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 1747, in changeform_view
return self._changeform_view(request, object_id, form_url, extra_context)
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 1799, in _changeform_view
self.save_related(request, form, formsets, not add)
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 1255, in save_related
self.save_formset(request, form, formset, change=change)
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 1243, in save_formset
formset.save()
File "/usr/local/lib/python3.11/site-packages/nested_admin/formsets.py", line 154, in save
instance = self.get_saved_instance_for_form(form, commit, form_instances)
File "/usr/local/lib/python3.11/site-packages/nested_admin/formsets.py", line 237, in get_saved_instance_for_form
instances = self.save_existing_objects([form], commit)
File "/usr/local/lib/python3.11/site-packages/nested_admin/formsets.py", line 314, in save_existing_objects
form.__dict__["changed_data"].append(pk_name)
KeyError: 'changed_data'

[jordanvs]

@scottgigante did you ever figure this out?

[scottgigante]

Unfortunately I did not -- I haven't been able to reproduce it.

[Barsoomx]

For me it happened when i used readonly inlines and changed the non-inline object:

class ReadOnlyNestedTabularInline(NestedTabularInline):
    extra = 0

    def has_add_permission(self, request, obj=None):
        return False

    def has_delete_permission(self, request, obj=None):
        return False

    def has_change_permission(self, request, obj=None):
        return False

And a custom pk field in the inline model:

class Destination(TimeStampedModel):
    uuid = models.UUIDField(primary_key=True)

[jordanvs]

I managed to address my error. Looking up at the exception stack, the initial exception thrown was a Model not found error. I have some complex model inheritance, polymorphism, and soft deletion going on, so without diving into all that:

My issue was caused by one of the inline models of my ModelAdmin not being accessible/found by the methods that save because it was actually soft-deleted. It was present in the queryset that displayed records on the admin page (bug), but not in the queryset that was used to save objects (expected).

Hence the front end was sending the soft-deleted object id in its payload that could not be found for updating, resulting in exceptions that propagated down to this error.

[scottgigante]

Thanks @jordanvs for the follow-up! I also use soft deletion to filter querysets, so likely this is the same cause.

[ex5]

Stumbled upon the same stacktrace during the following scenario:

• admin displays an inline form of an existing one-to-one and, nested inside it, of one-to-many records;
• one-to-many records are deleted during a save() of the parent model.

[fdintino]

I've tried testing it with the following model and admin definitions and data, and am unable to trigger the KeyError. Is there something I'm missing?

models.py

from django.db import models
from django.db.models import ForeignKey, CASCADE, OneToOneField


class Country(models.Model):
    code = models.CharField(max_length=2, primary_key=True)
    name = models.CharField(max_length=64)

    def __str__(self):
        return self.name


class Person(models.Model):
    name = models.CharField(max_length=128)

    def __str__(self):
        return self.name


class Passport(models.Model):
    passport_id = models.CharField(max_length=9)
    country = models.ForeignKey(Country, on_delete=CASCADE)
    person = OneToOneField(Person, on_delete=CASCADE)

    def __str__(self):
        return f"Passport {self.passport_id} [{self.person}]"


class TravelHistory(models.Model):
    country = models.ForeignKey(Country, on_delete=CASCADE)
    date = models.DateField()
    passport = ForeignKey(Passport, on_delete=CASCADE)

    class Meta:
        ordering = ("-date", "country")

    def __str__(self):
        return f"{self.passport}: {self.country} ({self.date.isoformat()})"

admin.py

from django.contrib import admin
import nested_admin
from .models import Person, Passport, TravelHistory


class TravelHistoryInline(nested_admin.NestedStackedInline):
    model = TravelHistory
    extra = 0


class PassportInline(nested_admin.NestedStackedInline):
    model = Passport
    inlines = [TravelHistoryInline]
    extra = 0


@admin.register(Person)
class PersonAdmin(nested_admin.NestedModelAdmin):
    model = Person
    inlines = [PassportInline]

tests.py

from datetime import date
import time

from nested_admin.tests.base import BaseNestedAdminTestCase
from .models import Country, Person, Passport, TravelHistory


class TestKeyErrorChangedData(BaseNestedAdminTestCase):
    root_model = Person
    nested_models = (Passport, TravelHistory)

    def test_issue_245(self):
        countries = {}
        for code, name in (
            ("US", "United States"),
            ("DE", "Germany"),
            ("FR", "France"),
            ("BR", "Brazil"),
            ("PH", "Philippines"),
            ("ET", "Ethiopia"),
            ("CN", "China"),
        ):
            countries[code] = Country.objects.create(code=code, name=name)
        person = Person.objects.create(name="Jane Doe")
        passport = Passport.objects.create(
            passport_id="A1B2C3D4E", country=countries["US"], person=person
        )
        TravelHistory.objects.bulk_create(
            [
                TravelHistory(
                    passport=passport, country=countries["DE"], date=date(2010, 12, 16)
                ),
                TravelHistory(
                    passport=passport, country=countries["CN"], date=date(2013, 3, 22)
                ),
                TravelHistory(
                    passport=passport, country=countries["BR"], date=date(2016, 1, 25)
                ),
                TravelHistory(
                    passport=passport, country=countries["FR"], date=date(2010, 6, 19)
                ),
                TravelHistory(
                    passport=passport, country=countries["ET"], date=date(2018, 4, 10)
                ),
            ]
        )
        self.load_admin(person)
        time.sleep(1000)
        # Here, make a change to the parent, delete one or more inlines, and save

[ex5]

I forgot another important detail: the nested inlines must be read-only, my bad.

@fdintino it's tricky to reproduce in a test: a POST request with an entire admin form with management form and all the inlines needs to be constructed to emulate a change of the Person record done via admin.

Manually, it's reproducable if one navigates to the Person change page in the admin, and saves it. This assumes that there's some kind of logic in Person.save() method which causes TravelHistory records to be deleted.

In the example project above they are just deleted if Person name ends with -delete:

models.py

from django.db import models
from django.db.models import ForeignKey, CASCADE, OneToOneField


class Person(models.Model):
    name = models.CharField(max_length=128)

    def __str__(self):
        return self.name

    def save(self, *args, **kwargs):
        super().save(*args, **kwargs)
        # Imagine that there's some business logic here that leads
        # to some TravelHistory records being deleted:
        if self.name.endswith('-delete'):
            TravelHistory.objects.filter(passport=self.passport).delete()


class Passport(models.Model):
    passport_id = models.CharField(max_length=9)
    person = OneToOneField(Person, on_delete=CASCADE, related_name='passport')

    def __str__(self):
        return f"Passport {self.passport_id} [{self.person}]"


class TravelHistory(models.Model):
    date = models.DateField()
    passport = ForeignKey(Passport, on_delete=CASCADE, related_name='travel_history')

    class Meta:
        ordering = ("-date", )

    def __str__(self):
        return f"{self.passport}: ({self.date.isoformat()})"

admin.py

from django.contrib import admin
import nested_admin
from .models import Person, Passport, TravelHistory


class TravelHistoryInline(nested_admin.NestedStackedInline):
    model = TravelHistory
    extra = 0

    def has_change_permission(self, request, obj=None):
        """Make read-only as if these are never created via admin, but automatically by some code.

        In combination with `Person.save()` deleting TravelHistory records,
        this leads to

            KeyError at /admin/app/person/{pk}/change/

                'changed_data'
        """
        return False


class PassportInline(nested_admin.NestedStackedInline):
    model = Passport
    inlines = [TravelHistoryInline]
    extra = 0


@admin.register(Person)
class PersonAdmin(nested_admin.NestedModelAdmin):
    model = Person
    inlines = [PassportInline]

Here's the code of the small project for reproducing it:

https://github.com/ex5/nested_admin_keyerror