diff --git a/docs/manual/forms/export_import.rst b/docs/manual/forms/export_import.rst index 9c095f56ca..b235171c8f 100644 --- a/docs/manual/forms/export_import.rst +++ b/docs/manual/forms/export_import.rst @@ -62,23 +62,33 @@ Als het goed is, is het formulier inclusief alle stappen geïmporteerd. Het geïmporteerde formulier is standaard niet actief en dus niet direct voor de buitenwereld toegankelijk. -Bijzonderheden --------------- +Formulier importeer opties +-------------------------- + +Bij het importeren van een formulier heb je enkele opties over hoe het +ZIP-bestand wordt geïmporteerd en hoe het formulier wordt gemaakt: -Als een formulier wordt geïmporteerd en het bevat een stap waarvan het -*URL-deel* overeenkomt met een bestaande stap, dan controleert het import proces -of de te importeren stap **exact hetzelfde** is als de bestaande stap. Als dat -zo is, dan wordt de bestaande stap gebruikt voor het geïmporteerde formulier. Er -wordt dan geen nieuwe stap aangemaakt. +* **Formulierinstellingen**: Hiermee kunnen registratie backends, + betaalprovider, prefill en inlogmethode eenvoudig bij het importeren + weggelaten worden. Standaard worden alle instellingen mee geïmporteerd. +* **Aanvullende formulierinstellingen**: Hiermee kunnen aanvullende + instellingen uit het ZIP-bestand toegevoegd worden aan het formulier. Bij het + importeren wordt gekeken of de aanvullende objecten, zoals producten, al in + de Open Formulieren-omgeving bestaan. Zo ja, worden deze bestaande objecten + hergebruikt, zo niet worden ze aangemaakt. +* **Hergebruik formulierdefinities**: Hiermee kan aangegeven worden of al + bestaande herbruikbare formulierdefinities gebruikt moeten worden, of dat + elke formulierdefinitie opnieuw aangemaakt moet worden. +* **Stijl**: Welk stijl gebruik zal worden voor het formulier. +* **Categorie**: In welke categorie het formulier geplaatst zal worden. -Als de stap **niet exact hetzelfde** is, dan wordt er een nieuwe stap aangemaakt -en wordt het *URL-deel* van de stap gewijzigd. Bijvoorbeeld van -``persoonsgegevens`` naar ``persoonsgegevens-2``. U krijgt hier altijd een -melding van. +Bijzonderheden +-------------- -Als het *URL-deel* van een te importeren formulier zelf al bestaat, dan kan het -formulier niet worden geïmporteerd. U kunt dan het *URL-deel* van het -conflicterende formulier zelf aanpassen en het nogmaals proberen. +Als het *URL-deel* van een te importeren formulier zelf al bestaat, dan wordt +het *URL-deel* van het te importeren formulier uniek gemaakt door een reeks van +letters en cijfers er achter te plaatsen. Na het importeren kunt u het +*URL-deel* nakijken en eventueel aanpassen. Export en import van :ref:`logica ` regels die gebruik maken van het experimentele bevragen van registraties is niet geïmplementeerd. De diff --git a/docs/manual/workflows.rst b/docs/manual/workflows.rst index 40f100387f..d89a6ff010 100644 --- a/docs/manual/workflows.rst +++ b/docs/manual/workflows.rst @@ -11,11 +11,11 @@ u kunt gebruiken om de levensloop van een formulier te optimaliseren. Essentieel bij het beheer zijn 2 opties: * **Actief** geeft de globale beschikbaarheid aan. -* **Onderhoudsmodus** geeft aan dat een formulier tijdelijk niet beschikbaar +* **Onderhoudsmodus** geeft aan dat een formulier tijdelijk niet beschikbaar is. Zodra een formulier *niet actief* of in *onderhoudsmodus* staat, dan is het -formulier voor reguliere gebruikers direct niet meer te gebruiken en lopende +formulier voor reguliere gebruikers direct niet meer te gebruiken en lopende sessies kunnen niet worden afgemaakt. In het geval een formulier *niet actief* is, dan geldt dit ook voor beheerders. In *onderhoudsmodus* kunnen beheerders nog wel het formulier starten, invullen en verzenden. @@ -24,15 +24,15 @@ nog wel het formulier starten, invullen en verzenden. Nieuw formulier maken ===================== -Als u het formulier direct op de productieomgeving bouwt, wilt u niet dat +Als u het formulier direct op de productieomgeving bouwt, wilt u niet dat reguliere gebruikers dit formulier al kunnen benaderen of invullen. -Zet de optie **Onderhoudsmodus** aan voordat u de eerste maal op **Opslaan** -klikt. Het formulier is hierdoor enkel door te starten door beheerders. +Zet de optie **Onderhoudsmodus** aan voordat u de eerste maal op **Opslaan** +klikt. Het formulier kan hierdoor enkel door beheerders gestart worden. Doorloop altijd eerst zelf het formulier voordat u een formulier actief maakt. .. note:: - + Als u een nieuw formulier importeert staat de optie **Actief** automatisch uit, en het geïmporteerde formulier kan dus niet benaderd worden door reguliere gebruikers en beheerders. @@ -42,18 +42,18 @@ Bestaand formulier bijwerken in de beheerinterface ================================================== Als een formulier in gebruik is op de productieomgeving, wilt u het formulier -eventueel bijwerken. Gebruikers die op dat moment bezig zijn wilt u zo min +eventueel bijwerken. Gebruikers die op dat moment bezig zijn wilt u zo min mogelijk hinderen. -Zet de optie **Onderhoudsmodus** aan en klik op **Opslaan**. Wijzig vervolgens -het formulier waar nodig. U kunt de wijzigingen in het formulier als beheerder -nog wel bekijken, maar reguliere gebruikers kunnen het formulier niet +Zet de optie **Onderhoudsmodus** aan en klik op **Opslaan**. Wijzig vervolgens +het formulier waar nodig. U kunt de wijzigingen in het formulier als beheerder +nog wel bekijken, maar reguliere gebruikers kunnen het formulier niet gebruiken. .. note:: - Als de wijzigingen niet de structuur van het formulier wijzigen, zoals een - veld of stap toevoegen dan wel verwijderen, dan kunt u het formulier gewoon + Als de wijzigingen niet de structuur van het formulier wijzigen, zoals een + veld of stap toevoegen dan wel verwijderen, dan kunt u het formulier gewoon wijzigen en opslaan, zonder dit formulier eerst op inactief te zetten. Lopende sessies blijven gewoon bestaan en ondervinden geen hinder. @@ -62,7 +62,7 @@ gebruiken. Bestaande formulieren bijwerken middels import ============================================== -Als u een formulier heeft gemaakt op een andere omgeving (bijvoorbeeld een +Als u een formulier heeft gemaakt op een andere omgeving (bijvoorbeeld een test- of acceptatieomgeving) en deze wilt importeren om een bestaand formulier te vervangen, dan is dat niet direct mogelijk. @@ -74,24 +74,24 @@ U moet rekening houden met de volgende complicaties: * De URL van een formulier moet uniek zijn. Om deze reden kunnen 2 formulieren niet dezelfde URL hebben. Als de URL van een geïmporteerd formulier al in gebruik is, dan krijgt het geïmporteerde formulier een iets andere URL. -* De URL van formulierdefinities (stappen) moeten uniek zijn. Als er al een +* De URL van formulierdefinities (stappen) moeten uniek zijn. Als er al een formulierdefinitie bestaat met dezelfde URL, dan krijgt de formulierdefinitie van het geïmporteerde formulier een iets andere URL. -U kunt de volgende acties uitvoeren om toch een bestaand formulier te +U kunt de volgende acties uitvoeren om toch een bestaand formulier te vervangen middels een import: #. Open het bestaande formulier in de beheerinterface. -#. Zet de optie **Actief** uit en wijzig de naam zodat er *(oud)* achter komt +#. Zet de optie **Actief** uit en wijzig de naam zodat er *(oud)* achter komt te staan. #. Wijzig ook de URL zodat er *-oud* achter komt te staan. -#. Wijzig bij alle formulierstappen, de URL zodat ook hier overal *-oud* achter - komt te staan. Heeft u **Herbruikbare** formulierstappen? Wijzig hier de URL +#. Wijzig bij alle formulierstappen, de URL zodat ook hier overal *-oud* achter + komt te staan. Heeft u **Herbruikbare** formulierstappen? Wijzig hier de URL dan niet om wijzigingen aan andere formulieren te voorkomen. #. Sla het formulier op. Het formulier is nu niet meer actief. -#. Importeer het het vervangende formulier. +#. Importeer het vervangende formulier. #. Pas eventueel de URLs aan van de formulierstappen die herbruikbaar zijn om mooiere URLs te gebruiken. -#. Configureer het formulier verder met alle informatie die niet is meegekomen +#. Configureer het formulier verder met alle informatie die niet is meegekomen met het importeren. #. Zet het geïmporteerde formulier op **Actief** en sla het formulier op. diff --git a/src/openapi.yaml b/src/openapi.yaml index 29c03b633b..576ba2b72a 100644 --- a/src/openapi.yaml +++ b/src/openapi.yaml @@ -1476,7 +1476,10 @@ paths: - forms requestBody: content: - '*/*': + multipart/form-data: + schema: + $ref: '#/components/schemas/FormImport' + application/x-www-form-urlencoded: schema: $ref: '#/components/schemas/FormImport' required: true @@ -6860,6 +6863,18 @@ components: Note that every subsequent API call resets the expiry. required: true schemas: + AdditionalFormConfigurationEnum: + enum: + - product + - wmsTileLayers + - wmtsTileLayers + - yiviAttributeGroups + type: string + description: |- + * `product` - Product + * `wmsTileLayers` - WMS-tile layers + * `wmtsTileLayers` - Background tile layers + * `yiviAttributeGroups` - Yivi attribute groups AddressSearchResult: type: object properties: @@ -8078,6 +8093,18 @@ components: title: authentication backend options required: - backend + FormConfigurationEnum: + enum: + - registrationBackends + - prefill + - paymentBackend + - authBackends + type: string + description: |- + * `registrationBackends` - Registration backends + * `prefill` - Prefill + * `paymentBackend` - Payment backend + * `authBackends` - Authentication backends FormData: type: object properties: @@ -8404,6 +8431,32 @@ components: format: uri description: The file that contains the form, form definitions and form steps. + formConfiguration: + type: array + items: + $ref: '#/components/schemas/FormConfigurationEnum' + description: Which form configuration should be included in the export file + content. + reuseFormDefinitions: + type: boolean + title: Re-use form definitions + description: Whether to re-use existing form definitions or create new form + definitions for each form definition in the import file. (If no matching + reusable form definition is found, a new one will be created.) + additionalFormConfiguration: + type: array + items: + $ref: '#/components/schemas/AdditionalFormConfigurationEnum' + description: Which additional form configuration should be included in the + export file content. + theme: + type: string + format: uuid + description: Which theme should be used for the imported forms. + category: + type: string + format: uuid + description: Which category should be applied to the imported forms. required: - file FormImportResponse: diff --git a/src/openforms/conf/locale/nl/LC_MESSAGES/django.po b/src/openforms/conf/locale/nl/LC_MESSAGES/django.po index 4e25ddc640..064e238919 100644 --- a/src/openforms/conf/locale/nl/LC_MESSAGES/django.po +++ b/src/openforms/conf/locale/nl/LC_MESSAGES/django.po @@ -4,8 +4,8 @@ msgid "" msgstr "" "Project-Id-Version: Open Forms\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2026-08-02 13:12+0200\n" -"PO-Revision-Date: 2026-08-02 13:17+0200\n" +"POT-Creation-Date: 2026-08-02 13:29+0200\n" +"PO-Revision-Date: 2026-08-02 13:34+0200\n" "Last-Translator: \n" "Language-Team: Dutch \n" "Language: nl\n" @@ -87,7 +87,7 @@ msgstr "stafstatus" msgid "Designates whether the user can log into this admin site." msgstr "Bepaalt of de gebruiker zich op deze beheerwebsite kan aanmelden." -#: openforms/accounts/models.py:55 openforms/forms/models/form.py:312 +#: openforms/accounts/models.py:55 openforms/forms/models/form.py:318 msgid "active" msgstr "actief" @@ -115,7 +115,7 @@ msgstr "" "De voorkeurstaal voor de (beheer) gebruikersinterface. Wanneer geen waarde " "ingesteld is, worden de browserinstellingen gebruikt." -#: openforms/accounts/models.py:85 openforms/forms/models/form.py:854 +#: openforms/accounts/models.py:85 openforms/forms/models/form.py:928 #: openforms/forms/models/form_version.py:86 soap/models.py:53 msgid "user" msgstr "gebruiker" @@ -757,9 +757,9 @@ msgstr "Unieke sleutel van het product" #: openforms/config/models/map.py:70 openforms/config/models/theme.py:21 #: openforms/contrib/customer_interactions/models.py:14 #: openforms/contrib/objects_api/models.py:16 -#: openforms/dmn/api/serializers.py:76 openforms/forms/admin/form.py:271 +#: openforms/dmn/api/serializers.py:76 openforms/forms/admin/form.py:272 #: openforms/forms/admin/form_definition.py:69 -#: openforms/forms/models/category.py:13 openforms/forms/models/form.py:81 +#: openforms/forms/models/category.py:13 openforms/forms/models/form.py:87 #: openforms/forms/models/form_definition.py:40 #: openforms/forms/models/form_registration_backend.py:23 #: openforms/forms/models/form_variable.py:265 @@ -2100,8 +2100,8 @@ msgstr "" #: openforms/authentication/contrib/yivi_oidc/models.py:20 #: openforms/config/models/map.py:64 openforms/config/models/theme.py:26 #: openforms/forms/api/serializers/logic/action_serializers.py:214 -#: openforms/forms/models/category.py:10 openforms/forms/models/form.py:80 -#: openforms/forms/models/form.py:841 +#: openforms/forms/models/category.py:10 openforms/forms/models/form.py:86 +#: openforms/forms/models/form.py:915 #: openforms/forms/models/form_definition.py:39 #: openforms/forms/models/form_step.py:27 #: openforms/forms/models/form_version.py:66 @@ -3008,7 +3008,7 @@ msgstr "" msgid "Confirmation: {{ public_reference }}" msgstr "Bevestiging: {{ public_reference }}" -#: openforms/config/models/config.py:76 openforms/forms/models/form.py:171 +#: openforms/config/models/config.py:76 openforms/forms/models/form.py:177 msgid "submission confirmation template" msgstr "Bevestigingspagina tekst" @@ -3184,7 +3184,7 @@ msgid "" msgstr "" "Het label van de knop op de overzichtspagina om naar de vorige stap te gaan." -#: openforms/config/models/config.py:240 openforms/forms/models/form.py:272 +#: openforms/config/models/config.py:240 openforms/forms/models/form.py:278 msgid "change text" msgstr "Stap wijzigen-label" @@ -3199,7 +3199,7 @@ msgid "" msgstr "" "Het label de link op de overzichtspagina om een bepaalde stap te wijzigen" -#: openforms/config/models/config.py:249 openforms/forms/models/form.py:282 +#: openforms/config/models/config.py:249 openforms/forms/models/form.py:288 msgid "confirm text" msgstr "Formulier verzenden-label" @@ -3215,7 +3215,7 @@ msgid "" msgstr "" "Het label van de knop op de overzichtspagina om het formulier in te dienen" -#: openforms/config/models/config.py:258 openforms/forms/models/form.py:252 +#: openforms/config/models/config.py:258 openforms/forms/models/form.py:258 msgid "begin text" msgstr "Formulier starten-label" @@ -3435,11 +3435,11 @@ msgstr "" "tokens {year}, {public_reference} en {uid} zijn beschikbaar. Het sjabloon " "moet de placeholder {uid} bevatten." -#: openforms/config/models/config.py:434 openforms/forms/models/form.py:231 +#: openforms/config/models/config.py:434 openforms/forms/models/form.py:237 msgid "ask privacy consent" msgstr "vraag toestemming om gegevens te verwerken" -#: openforms/config/models/config.py:437 openforms/forms/models/form.py:236 +#: openforms/config/models/config.py:437 openforms/forms/models/form.py:242 msgid "" "If enabled, the user will have to agree to the privacy policy before " "submitting a form." @@ -3475,11 +3475,11 @@ msgstr "" "Ja, ik heb kennis genomen van het {% privacy_policy %} en geef uitdrukkelijk" " toestemming voor het verwerken van de door mij opgegeven gegevens." -#: openforms/config/models/config.py:467 openforms/forms/models/form.py:240 +#: openforms/config/models/config.py:467 openforms/forms/models/form.py:246 msgid "ask statement of truth" msgstr "vraag waarheidsverklaring" -#: openforms/config/models/config.py:470 openforms/forms/models/form.py:245 +#: openforms/config/models/config.py:470 openforms/forms/models/form.py:251 msgid "" "If enabled, the user will have to agree that they filled out the form " "truthfully before submitting it." @@ -3508,7 +3508,7 @@ msgstr "" "Ik verklaar dat ik deze aanvraag naar waarheid heb ingevuld en geen " "informatie heb verzwegen." -#: openforms/config/models/config.py:492 openforms/forms/models/form.py:336 +#: openforms/config/models/config.py:492 openforms/forms/models/form.py:342 msgid "successful submission removal limit" msgstr "bewaartermijn voor voltooide inzendingen." @@ -3516,7 +3516,7 @@ msgstr "bewaartermijn voor voltooide inzendingen." msgid "Amount of days successful submissions will remain before being removed" msgstr "Aantal dagen dat een voltooide inzending bewaard blijft." -#: openforms/config/models/config.py:500 openforms/forms/models/form.py:346 +#: openforms/config/models/config.py:500 openforms/forms/models/form.py:352 msgid "successful submissions removal method" msgstr "opschoonmethode voor voltooide inzendingen" @@ -3525,7 +3525,7 @@ msgid "How successful submissions will be removed after the limit" msgstr "" "Geeft aan hoe voltooide inzendingen worden opgeschoond na de bewaartermijn." -#: openforms/config/models/config.py:507 openforms/forms/models/form.py:356 +#: openforms/config/models/config.py:507 openforms/forms/models/form.py:362 msgid "incomplete submission removal limit" msgstr "bewaartermijn voor sessies" @@ -3533,7 +3533,7 @@ msgstr "bewaartermijn voor sessies" msgid "Amount of days incomplete submissions will remain before being removed" msgstr "Aantal dagen dat een sessie bewaard blijft." -#: openforms/config/models/config.py:515 openforms/forms/models/form.py:366 +#: openforms/config/models/config.py:515 openforms/forms/models/form.py:372 msgid "incomplete submissions removal method" msgstr "opschoonmethode voor sessies." @@ -3541,8 +3541,8 @@ msgstr "opschoonmethode voor sessies." msgid "How incomplete submissions will be removed after the limit" msgstr "Geeft aan hoe sessies worden opgeschoond na de bewaartermijn." -#: openforms/config/models/config.py:522 openforms/forms/models/form.py:376 -#: openforms/forms/models/form.py:386 +#: openforms/config/models/config.py:522 openforms/forms/models/form.py:382 +#: openforms/forms/models/form.py:392 msgid "errored submission removal limit" msgstr "bewaartermijn voor niet voltooide inzendingen" @@ -3562,7 +3562,7 @@ msgstr "" "Geeft aan hoe niet voltooide inzendingen (door fouten in de afhandeling) " "worden opgeschoond na de bewaartermijn." -#: openforms/config/models/config.py:537 openforms/forms/models/form.py:396 +#: openforms/config/models/config.py:537 openforms/forms/models/form.py:402 msgid "all submissions removal limit" msgstr "bewaartermijn van inzendingen" @@ -3733,7 +3733,6 @@ msgstr "" "unieke volgorde van het alfabet." #: openforms/config/models/config.py:673 -#| msgid "export content" msgid "help callout page content" msgstr "" @@ -3989,7 +3988,7 @@ msgstr "stijlen" #: openforms/config/templates/admin/config/overview.html:8 #: openforms/config/templates/admin/config/theme/preview.html:13 #: openforms/forms/templates/admin/forms/form/export.html:13 -#: openforms/forms/templates/admin/forms/form/import_form.html:13 +#: openforms/forms/templates/admin/forms/form/import_form.html:14 #: openforms/forms/templates/admin/forms/form/migrate-payment-backend.html:18 #: openforms/forms/templates/admin/forms/formsubmissionstatistics/export_form.html:19 msgid "Home" @@ -4008,7 +4007,7 @@ msgid "Status" msgstr "Status" #: openforms/config/templates/admin/config/overview.html:25 -#: openforms/forms/admin/category.py:37 openforms/forms/admin/form.py:264 +#: openforms/forms/admin/category.py:37 openforms/forms/admin/form.py:265 #: openforms/forms/api/serializers/logic/form_logic.py:132 #: openforms/submissions/api/serializers.py:244 msgid "Actions" @@ -5421,7 +5420,7 @@ msgstr "" " voor meer informatie." #: openforms/emails/models.py:97 openforms/forms/admin/form_logic.py:28 -#: openforms/forms/models/form.py:450 +#: openforms/forms/models/form.py:456 #: openforms/forms/models/form_variable.py:249 #: openforms/forms/models/form_version.py:68 msgid "form" @@ -6587,95 +6586,95 @@ msgstr "aantal formulieren" msgid "Show forms" msgstr "Toon formulieren" -#: openforms/forms/admin/form.py:54 +#: openforms/forms/admin/form.py:55 msgid "has reached submission limit" msgstr "status inzendingslimiet" -#: openforms/forms/admin/form.py:59 +#: openforms/forms/admin/form.py:60 msgid "Available for submission" msgstr "Accepteert nieuwe inzendingen" -#: openforms/forms/admin/form.py:60 +#: openforms/forms/admin/form.py:61 msgid "Unavailable for submission" msgstr "Limiet bereikt" -#: openforms/forms/admin/form.py:78 +#: openforms/forms/admin/form.py:79 msgid "is deleted" msgstr "is verwijderd" -#: openforms/forms/admin/form.py:106 +#: openforms/forms/admin/form.py:107 msgid "Available forms" msgstr "Beschikbare formulieren" -#: openforms/forms/admin/form.py:113 +#: openforms/forms/admin/form.py:114 msgid "Deleted forms" msgstr "Verwijderde formulieren" -#: openforms/forms/admin/form.py:268 +#: openforms/forms/admin/form.py:269 msgid "Show form" msgstr "Toon formulier" -#: openforms/forms/admin/form.py:275 +#: openforms/forms/admin/form.py:276 msgid "Live" msgstr "Live" -#: openforms/forms/admin/form.py:299 +#: openforms/forms/admin/form.py:300 #, python-brace-format msgid "{} {} was successfully copied" msgstr "{} {} is met succes gekopieerd" -#: openforms/forms/admin/form.py:334 +#: openforms/forms/admin/form.py:335 #, python-brace-format msgid "{} {} was successfully exported" msgstr "{} {} is met succes geëxporteerd" -#: openforms/forms/admin/form.py:357 +#: openforms/forms/admin/form.py:358 #: openforms/forms/admin/form_definition.py:82 #, python-format msgid "Copy selected %(verbose_name_plural)s" msgstr "Kopieer geselecteerde %(verbose_name_plural)s" -#: openforms/forms/admin/form.py:365 +#: openforms/forms/admin/form.py:366 #, python-brace-format msgid "Copied {count} {verbose_name} object." msgid_plural "Copied {count} {verbose_name} objects." msgstr[0] "{count} {verbose_name} object gekopieerd" msgstr[1] "{count} {verbose_name}objecten gekopieerd" -#: openforms/forms/admin/form.py:375 +#: openforms/forms/admin/form.py:376 #, python-format msgid "Set selected %(verbose_name_plural)s to maintenance mode" msgstr "" "Schakel de onderhoudsmodus in voor geselecteerde %(verbose_name_plural)s" -#: openforms/forms/admin/form.py:382 +#: openforms/forms/admin/form.py:383 #, python-brace-format msgid "Set {count} {verbose_name} object to maintenance mode" msgid_plural "Set {count} {verbose_name} objects to maintenance mode" msgstr[0] "Onderhoudsmodus aangezet voor {count} {verbose_name} object " msgstr[1] "Onderhoudsmodus aangezet voor {count} {verbose_name} objecten" -#: openforms/forms/admin/form.py:391 +#: openforms/forms/admin/form.py:392 #, python-format msgid "Remove %(verbose_name_plural)s from maintenance mode" msgstr "" "Schakel de onderhoudsmodus uit voor geselecteerde %(verbose_name_plural)s." -#: openforms/forms/admin/form.py:397 +#: openforms/forms/admin/form.py:398 #, python-brace-format msgid "Removed {count} {verbose_name} object from maintenance mode" msgid_plural "Removed {count} {verbose_name} objects from maintenance mode" msgstr[0] "Onderhoudsmodus uitgezet voor {count} {verbose_name} object" msgstr[1] "Onderhoudsmodus uitgezet voor {count} {verbose_name} objecten" -#: openforms/forms/admin/form.py:446 +#: openforms/forms/admin/form.py:447 #: openforms/forms/templates/admin/forms/form/export.html:15 #: openforms/forms/templates/admin/forms/form/export.html:19 #: openforms/forms/templates/admin/forms/form/export.html:22 msgid "Export forms" msgstr "Formulieren exporteren" -#: openforms/forms/admin/form.py:452 +#: openforms/forms/admin/form.py:453 msgid "" "Please configure your email address in your admin profile before requesting " "a bulk export" @@ -6707,36 +6706,40 @@ msgstr "ingezonden tussen" msgid "Red" msgstr "Rood" -#: openforms/forms/admin/tasks.py:78 +#: openforms/forms/admin/tasks.py:80 msgid "Forms export ready" msgstr "Export formulieren gereed" -#: openforms/forms/admin/views.py:41 +#: openforms/forms/admin/views.py:43 msgid "Anonymize form configuration" msgstr "Formulierinstellingen anonimiseren" -#: openforms/forms/admin/views.py:45 +#: openforms/forms/admin/views.py:47 #: openforms/forms/api/serializers/form.py:701 msgid "" "Whether sensative form configuration should be anonymized during exporting." msgstr "Anonimiseer gevoelige formulierinstellingen bij het exporteren." -#: openforms/forms/admin/views.py:49 +#: openforms/forms/admin/views.py:51 openforms/forms/forms/form.py:19 msgid "Form configuration" msgstr "Formulierinstellingen" -#: openforms/forms/admin/views.py:60 +#: openforms/forms/admin/views.py:62 #: openforms/forms/api/serializers/form.py:708 +#: openforms/forms/api/serializers/form.py:727 +#: openforms/forms/forms/form.py:30 msgid "" "Which form configuration should be included in the export file content." msgstr "Welke formulierinstellingen meegenomen worden bij het exporteren." -#: openforms/forms/admin/views.py:64 +#: openforms/forms/admin/views.py:66 openforms/forms/forms/form.py:34 msgid "Additional form configuration" msgstr "Aanvullende formulierinstellingen" -#: openforms/forms/admin/views.py:69 +#: openforms/forms/admin/views.py:71 #: openforms/forms/api/serializers/form.py:715 +#: openforms/forms/api/serializers/form.py:745 +#: openforms/forms/forms/form.py:39 msgid "" "Which additional form configuration should be included in the export file " "content." @@ -6744,24 +6747,24 @@ msgstr "" "Welke aanvullende formulierinstellingen meegenomen worden bij het " "exporteren." -#: openforms/forms/admin/views.py:75 +#: openforms/forms/admin/views.py:77 msgid "Invalid form uuids." msgstr "Ongeldige formulier-IDs." -#: openforms/forms/admin/views.py:88 +#: openforms/forms/admin/views.py:90 msgid "Success! You will receive an email when your export is ready." msgstr "Gelukt! U ontvangt een e-mail zodra de export gereed is." -#: openforms/forms/admin/views.py:143 +#: openforms/forms/admin/views.py:160 #, python-brace-format msgid "Something went wrong while importing form: {}" msgstr "Er is iets foutgegaan bij het importeren van formulier: {}" -#: openforms/forms/admin/views.py:156 +#: openforms/forms/admin/views.py:173 msgid "Form successfully imported!" msgstr "Formulier is succesvol geïmporteerd" -#: openforms/forms/admin/views.py:158 +#: openforms/forms/admin/views.py:175 msgid "" "The bulk import is being processed! The imported forms will soon be " "available." @@ -6802,12 +6805,10 @@ msgid "Help callout page display" msgstr "" #: openforms/forms/api/serializers/form.py:147 -#| msgid "The display label of the catalogue" msgid "When to display the help callout page." msgstr "" #: openforms/forms/api/serializers/form.py:152 -#| msgid "export content" msgid "Help callout page content" msgstr "" @@ -6869,7 +6870,7 @@ msgid "URL to the product in the Open Forms API" msgstr "URL van het product in de Open Formulieren API" #: openforms/forms/api/serializers/form.py:234 -#: openforms/forms/models/form.py:134 +#: openforms/forms/models/form.py:140 msgid "payment backend options" msgstr "betaalprovider backend opties" @@ -6909,33 +6910,61 @@ msgstr "" "Het bestand dat het formulier, de formulierdefinities en de formulierstappen" " bevat." -#: openforms/forms/api/serializers/form.py:727 +#: openforms/forms/api/serializers/form.py:732 +#: openforms/forms/forms/form.py:44 +#| msgid "form definition" +msgid "Re-use form definitions" +msgstr "Hergebruik formulierdefinities" + +#: openforms/forms/api/serializers/form.py:736 +#: openforms/forms/forms/form.py:48 +msgid "" +"Whether to re-use existing form definitions or create new form definitions " +"for each form definition in the import file. (If no matching reusable form " +"definition is found, a new one will be created.)" +msgstr "" +"Of bestaande formulierdefinities worden hergebruikt of dat er voor elke " +"formulierdefinitie in het importbestand een nieuwe formulierdefinitie worden" +" aangemaakt. (Als er geen overeenkomende herbruikbare formulierdefinitie " +"wordt gevonden, wordt er een nieuwe aangemaakt.)" + +#: openforms/forms/api/serializers/form.py:751 +#| msgid "The uuid of the imported form." +msgid "Which theme should be used for the imported forms." +msgstr "Welk stijl gebruikt zal worden voor de geïmporteerde formulieren." + +#: openforms/forms/api/serializers/form.py:755 +#| msgid "The uuid of the imported form." +msgid "Which category should be applied to the imported forms." +msgstr "Welke categorie toegepast zal worden op de geïmporteerde formulieren." + +#: openforms/forms/api/serializers/form.py:760 msgid "The uuid of the imported form." msgstr "ID van het geïmporteerde formulier." -#: openforms/forms/api/serializers/form.py:732 +#: openforms/forms/api/serializers/form.py:765 msgid "Registration backend key" msgstr "Registratiebackend-sleutel" -#: openforms/forms/api/serializers/form.py:734 +#: openforms/forms/api/serializers/form.py:767 msgid "The registration backend key for which to generate the schema." msgstr "" "De registratiebackend-sleutel waarvoor het schema moet worden gegenereerd." -#: openforms/forms/api/serializers/form.py:739 +#: openforms/forms/api/serializers/form.py:772 msgid "Registration backend identifier" msgstr "Registratiebackend-identificatie" -#: openforms/forms/api/serializers/form.py:744 +#: openforms/forms/api/serializers/form.py:777 msgid "Registration backend options" msgstr "Registratiebackend-instellingen" -#: openforms/forms/api/serializers/form.py:756 +#: openforms/forms/api/serializers/form.py:789 #, python-brace-format msgid "Backend with key '{key}' does not exist for form '{form}'" msgstr "Backend met sleutel '{key}' bestaat niet voor formulier '{form}'" -#: openforms/forms/api/serializers/form.py:773 +#: openforms/forms/api/serializers/form.py:806 #, python-brace-format msgid "Backend with id '{backend}' does not allow JSON schema generation" msgstr "" @@ -7632,9 +7661,9 @@ msgstr "Het UUID van de formulierversie" msgid "List form versions" msgstr "Formulierversies weergeven" -#: openforms/forms/api/viewsets.py:785 -#: openforms/forms/templates/admin/forms/form/import_form.html:15 -#: openforms/forms/templates/admin/forms/form/import_form.html:20 +#: openforms/forms/api/viewsets.py:788 +#: openforms/forms/templates/admin/forms/form/import_form.html:16 +#: openforms/forms/templates/admin/forms/form/import_form.html:21 msgid "Import form" msgstr "Formulier importeren" @@ -7727,12 +7756,10 @@ msgid "Single step" msgstr "Enkele stap" #: openforms/forms/constants.py:71 -#| msgid "Go to the payment page" msgid "Before the start page" msgstr "" #: openforms/forms/constants.py:72 -#| msgid "Go to the payment page" msgid "After the start page" msgstr "" @@ -7740,14 +7767,34 @@ msgstr "" msgid "Never" msgstr "" -#: openforms/forms/forms/form.py:7 +#: openforms/forms/forms/form.py:14 msgid "file" msgstr "bestand" -#: openforms/forms/forms/form.py:9 +#: openforms/forms/forms/form.py:16 msgid "Upload your exported ZIP-file." msgstr "Upload het geëxporteerde ZIP-bestand." +#: openforms/forms/forms/form.py:54 +#| msgid "Themes" +msgid "Theme" +msgstr "Stijl" + +#: openforms/forms/forms/form.py:56 +#| msgid "The uuid of the imported form." +msgid "Which theme should be used for the imported form." +msgstr "Welk stijl gebruikt zal worden voor het geïmporteerde formulier." + +#: openforms/forms/forms/form.py:60 +#| msgid "category" +msgid "Category" +msgstr "Categorie" + +#: openforms/forms/forms/form.py:62 +#| msgid "The uuid of the imported form." +msgid "Which category should be applied to the imported form." +msgstr "Welke categorie toegepast zal worden op het geïmporteerde formulier." + #: openforms/forms/forms/form_statistics.py:38 msgid "Successfully registered" msgstr "Successvol geregistreerd" @@ -7789,7 +7836,7 @@ msgstr "Exporteer inzendingen die op of voor deze datum ingestuurd zijn." #: openforms/forms/forms/form_statistics.py:72 #: openforms/forms/templates/admin/forms/form/export.html:14 -#: openforms/forms/templates/admin/forms/form/import_form.html:14 +#: openforms/forms/templates/admin/forms/form/import_form.html:15 #: openforms/forms/templates/admin/forms/form/migrate-payment-backend.html:19 msgid "Forms" msgstr "Formulieren" @@ -7886,21 +7933,21 @@ msgstr "Eenvoudige naam" msgid "categories" msgstr "categorieën" -#: openforms/forms/models/form.py:83 +#: openforms/forms/models/form.py:89 #: openforms/forms/models/form_definition.py:42 msgid "internal name" msgstr "interne naam" -#: openforms/forms/models/form.py:86 +#: openforms/forms/models/form.py:92 #: openforms/forms/models/form_definition.py:45 msgid "internal name for management purposes" msgstr "interne naam voor beheerdoeleinden " -#: openforms/forms/models/form.py:89 +#: openforms/forms/models/form.py:95 msgid "internal remarks" msgstr "interne opmerkingen" -#: openforms/forms/models/form.py:91 +#: openforms/forms/models/form.py:97 msgid "" "Remarks or intentions about the form. Can also be used to save notes for " "later use or for another admin user." @@ -7908,17 +7955,17 @@ msgstr "" "Opmerkingen of intenties over het formulier. Kan ook gebruikt worden voor " "het maken van notities voor later gebruik of voor een andere beheerder." -#: openforms/forms/models/form.py:98 +#: openforms/forms/models/form.py:104 #: openforms/forms/models/form_definition.py:47 #: openforms/forms/models/form_step.py:33 msgid "slug" msgstr "URL-deel" -#: openforms/forms/models/form.py:101 +#: openforms/forms/models/form.py:107 msgid "form type" msgstr "formuliertype" -#: openforms/forms/models/form.py:105 +#: openforms/forms/models/form.py:111 msgid "" "The type of the form. The choices are regular, appointment or a single step " "form. Depending on the choice a different form design is required/rendered." @@ -7927,11 +7974,11 @@ msgstr "" "standaard, afspraak of een formulier met één stap. Afhankelijk van de keuze " "zijn andere formulierinstellingen beschikbaar/verplicht." -#: openforms/forms/models/form.py:123 +#: openforms/forms/models/form.py:129 msgid "form theme" msgstr "formulierstijl" -#: openforms/forms/models/form.py:125 +#: openforms/forms/models/form.py:131 msgid "" "Apply a specific appearance configuration to the form. If left blank, then " "the globally configured default is applied." @@ -7939,33 +7986,33 @@ msgstr "" "Pas een specifieke stijl toe op het formulier. Indien geen optie gekozen is," " dan wordt de globale instelling toegepast." -#: openforms/forms/models/form.py:129 +#: openforms/forms/models/form.py:135 msgid "translation enabled" msgstr "meertaligheid ingeschakeld" -#: openforms/forms/models/form.py:132 openforms/forms/models/form.py:534 +#: openforms/forms/models/form.py:138 openforms/forms/models/form.py:540 msgid "payment backend" msgstr "betaalprovider backend" -#: openforms/forms/models/form.py:139 +#: openforms/forms/models/form.py:145 msgid "price variable key" msgstr "sleutel prijsvariabele" -#: openforms/forms/models/form.py:142 +#: openforms/forms/models/form.py:148 msgid "Key of the variable that contains the calculated submission price." msgstr "" "Sleutel van de variabele die de (berekende) kostprijs van de inzending " "bevat." -#: openforms/forms/models/form.py:149 +#: openforms/forms/models/form.py:155 msgid "automatic login" msgstr "automatisch inloggen" -#: openforms/forms/models/form.py:154 +#: openforms/forms/models/form.py:160 msgid "maximum allowed submissions" msgstr "maximum aantal inzendingen" -#: openforms/forms/models/form.py:158 +#: openforms/forms/models/form.py:164 msgid "" "Maximum number of allowed submissions per form. Leave this empty if no limit" " is needed." @@ -7973,11 +8020,11 @@ msgstr "" "Het maximum aantal inzendingen die toegestaan zijn voor het formulier. Laat " "dit veld leeg als er geen beperking is." -#: openforms/forms/models/form.py:162 +#: openforms/forms/models/form.py:168 msgid "submissions counter" msgstr "aantal inzendingen" -#: openforms/forms/models/form.py:165 +#: openforms/forms/models/form.py:171 msgid "" "Counter to track how many submissions have been completed for the specific " "form. This works in combination with the maximum allowed submissions per " @@ -7987,7 +8034,7 @@ msgstr "" "Deze werkt samen met de inzendingslimiet en kan gereset worden via de " "frontend." -#: openforms/forms/models/form.py:173 +#: openforms/forms/models/form.py:179 msgid "" "The content of the submission confirmation page. It can contain variables " "that will be templated from the submitted form data. If not specified, the " @@ -7997,12 +8044,12 @@ msgstr "" "de ingediende formuliergegevens worden weergegeven. Indien niet opgegeven " "dan wordt de globale bevestingspagina gebruikt." -#: openforms/forms/models/form.py:182 +#: openforms/forms/models/form.py:188 #: openforms/submissions/api/serializers.py:157 msgid "submission allowed" msgstr "inzenden toegestaan" -#: openforms/forms/models/form.py:186 +#: openforms/forms/models/form.py:192 msgid "" "Whether the user is allowed to submit this form or not, and whether the " "overview page should be shown if they are not." @@ -8010,31 +8057,31 @@ msgstr "" "Geeft aan of de gebruiker het formulier kan verzenden en of de " "overzichtspagina getoond wordt als het formulier niet verzonden kan worden." -#: openforms/forms/models/form.py:192 +#: openforms/forms/models/form.py:198 msgid "suspension allowed" msgstr "tussentijds opslaan" -#: openforms/forms/models/form.py:194 +#: openforms/forms/models/form.py:200 msgid "Whether the user is allowed to suspend this form or not." msgstr "" "Geeft aan of de gebruiker het formulier tussentijds kan opslaan/pauzeren of " "niet." -#: openforms/forms/models/form.py:197 +#: openforms/forms/models/form.py:203 msgid "show progress indicator" msgstr "toon formulierstappen" -#: openforms/forms/models/form.py:200 +#: openforms/forms/models/form.py:206 msgid "Whether the step progression should be displayed in the UI or not." msgstr "" "Indien aangevinkt dan worden de stappen van het formulier aan de gebruiker " "getoond." -#: openforms/forms/models/form.py:204 +#: openforms/forms/models/form.py:210 msgid "show summary of the progress" msgstr "toon voortgangsoverzicht" -#: openforms/forms/models/form.py:207 +#: openforms/forms/models/form.py:213 msgid "" "Whether to display the short progress summary, indicating the current step " "number and total amount of steps." @@ -8043,30 +8090,30 @@ msgstr "" " de huidige stap en het totaal aantal stappen, typisch onder de " "formuliertitel." -#: openforms/forms/models/form.py:212 +#: openforms/forms/models/form.py:218 msgid "display main website link" msgstr "toon hoofdsite link" -#: openforms/forms/models/form.py:215 +#: openforms/forms/models/form.py:221 msgid "" "Display the link to the main website on the submission confirmation page." msgstr "Toon de link naar de hoofdsite op de bevestigingspagina na inzenden." -#: openforms/forms/models/form.py:219 +#: openforms/forms/models/form.py:225 msgid "include confirmation page content in PDF" msgstr "voeg de \"bevestigingspaginatekst\" toe in de bevestigings-PDF" -#: openforms/forms/models/form.py:221 +#: openforms/forms/models/form.py:227 msgid "Display the instruction from the confirmation page in the PDF." msgstr "" "Vink aan om de inhoud van het \"bevestigingspaginatekst\"-veld toe te voegen" " aan de PDF met gegevens ter bevestiging." -#: openforms/forms/models/form.py:224 +#: openforms/forms/models/form.py:230 msgid "send confirmation email" msgstr "verstuur bevestigingsmail" -#: openforms/forms/models/form.py:226 +#: openforms/forms/models/form.py:232 msgid "" "Whether a confirmation email should be sent to the end user filling in the " "form." @@ -8074,7 +8121,7 @@ msgstr "" "Geef aan of er een bevestigingsmail naar de formulierinzender verstuurd " "dient te worden." -#: openforms/forms/models/form.py:256 +#: openforms/forms/models/form.py:262 msgid "" "The text that will be displayed at the start of the form to indicate the " "user can begin to fill in the form. Leave blank to get value from global " @@ -8083,11 +8130,11 @@ msgstr "" "Het label van de knop om het formulier te starten. Laat leeg om de waarde " "van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:262 +#: openforms/forms/models/form.py:268 msgid "previous text" msgstr "Vorige stap-label" -#: openforms/forms/models/form.py:266 +#: openforms/forms/models/form.py:272 msgid "" "The text that will be displayed in the overview page to go to the previous " "step. Leave blank to get value from global configuration." @@ -8095,7 +8142,7 @@ msgstr "" "Het label van de knop op de overzichtspagina om naar de vorige stap te gaan." " Laat leeg om de waarde van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:276 +#: openforms/forms/models/form.py:282 msgid "" "The text that will be displayed in the overview page to change a certain " "step. Leave blank to get value from global configuration." @@ -8103,7 +8150,7 @@ msgstr "" "Het label de link op de overzichtspagina om een bepaalde stap te wijzigen. " "Laat leeg om de waarde van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:286 +#: openforms/forms/models/form.py:292 msgid "" "The text that will be displayed in the overview page to confirm the form is " "filled in correctly. Leave blank to get value from global configuration." @@ -8111,11 +8158,11 @@ msgstr "" "Het label van de knop op de overzichtspagina om het formulier in te dienen. " "Laat leeg om de waarde van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:294 +#: openforms/forms/models/form.py:300 msgid "introduction page" msgstr "introductiepagina" -#: openforms/forms/models/form.py:296 +#: openforms/forms/models/form.py:302 msgid "" "Content for the introduction page that leads to the start page of the form. " "Leave blank to disable the introduction page." @@ -8123,11 +8170,11 @@ msgstr "" "Inhoud voor de introductiepagina die naar de formulierstartpagina leidt. " "Laat deze leeg om de introductiepagina uit te schakelen." -#: openforms/forms/models/form.py:304 +#: openforms/forms/models/form.py:310 msgid "explanation template" msgstr "Toelichtingssjabloon" -#: openforms/forms/models/form.py:306 +#: openforms/forms/models/form.py:312 msgid "" "Content that will be shown on the start page of the form, below the title " "and above the log in text." @@ -8135,31 +8182,31 @@ msgstr "" "Inhoud die op de formulierstartpagina wordt getoond, onder de titel en boven" " de startknop(pen)." -#: openforms/forms/models/form.py:314 +#: openforms/forms/models/form.py:320 msgid "maintenance mode" msgstr "onderhoudsmodus" -#: openforms/forms/models/form.py:317 +#: openforms/forms/models/form.py:323 msgid "Users will not be able to start the form if it is in maintenance mode." msgstr "Gebruikers kunnen een formulier in onderhoudsmodus niet starten." -#: openforms/forms/models/form.py:322 +#: openforms/forms/models/form.py:328 msgid "activate on" msgstr "Activeren op" -#: openforms/forms/models/form.py:325 +#: openforms/forms/models/form.py:331 msgid "Date and time on which the form should be activated." msgstr "Datum en tijdstip waarop het formulier geactiveerd moet worden." -#: openforms/forms/models/form.py:328 +#: openforms/forms/models/form.py:334 msgid "deactivate on" msgstr "Deactiveren op" -#: openforms/forms/models/form.py:331 +#: openforms/forms/models/form.py:337 msgid "Date and time on which the form should be deactivated." msgstr "Datum en tijdstip waarop het formulier gedeactiveerd moet worden." -#: openforms/forms/models/form.py:341 +#: openforms/forms/models/form.py:347 msgid "" "Amount of days successful submissions of this form will remain before being " "removed. Leave blank to use value in General Configuration." @@ -8167,7 +8214,7 @@ msgstr "" "Aantal dagen dat een voltooide inzending bewaard blijft. Laat leeg om de " "waarde van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:351 +#: openforms/forms/models/form.py:357 msgid "" "How successful submissions of this form will be removed after the limit. " "Leave blank to use value in General Configuration." @@ -8175,7 +8222,7 @@ msgstr "" "Geeft aan hoe voltooide inzendingen worden opgeschoond na de bewaartermijn. " "Laat leeg om de waarde van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:361 +#: openforms/forms/models/form.py:367 msgid "" "Amount of days incomplete submissions of this form will remain before being " "removed. Leave blank to use value in General Configuration." @@ -8183,7 +8230,7 @@ msgstr "" "Aantal dagen dat een sessie bewaard blijft. Laat leeg om de waarde van de " "algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:371 +#: openforms/forms/models/form.py:377 msgid "" "How incomplete submissions of this form will be removed after the limit. " "Leave blank to use value in General Configuration." @@ -8191,7 +8238,7 @@ msgstr "" "Geeft aan hoe sessies worden opgeschoond na de bewaartermijn. Laat leeg om " "de waarde van de algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:381 +#: openforms/forms/models/form.py:387 msgid "" "Amount of days errored submissions of this form will remain before being " "removed. Leave blank to use value in General Configuration." @@ -8200,7 +8247,7 @@ msgstr "" "afhandeling) bewaard blijft. Laat leeg om de waarde van de algemene " "configuratie te gebruiken." -#: openforms/forms/models/form.py:391 +#: openforms/forms/models/form.py:397 msgid "" "How errored submissions of this form will be removed after the limit. Leave " "blank to use value in General Configuration." @@ -8209,7 +8256,7 @@ msgstr "" "worden opgeschoond na de bewaartermijn. Laat leeg om de waarde van de " "algemene configuratie te gebruiken." -#: openforms/forms/models/form.py:401 +#: openforms/forms/models/form.py:407 msgid "" "Amount of days when all submissions of this form will be permanently " "deleted. Leave blank to use value in General Configuration." @@ -8218,21 +8265,21 @@ msgstr "" "verwijderd wordt. Laat leeg om de waarde van de algemene configuratie te " "gebruiken." -#: openforms/forms/models/form.py:408 +#: openforms/forms/models/form.py:414 msgid "help callout page display" msgstr "" -#: openforms/forms/models/form.py:412 +#: openforms/forms/models/form.py:418 msgid "" "When to display the help callout page. Note that this field is disabled when" " the help callout page content in the global configuration is not specified." msgstr "" -#: openforms/forms/models/form.py:421 +#: openforms/forms/models/form.py:427 msgid "enable new logic rule evaluation" msgstr "gebruik nieuwe logica-evaluatie" -#: openforms/forms/models/form.py:424 +#: openforms/forms/models/form.py:430 msgid "" "Enabling this will analyze logic rules and re-order them according to their " "dependency on other logic rules (happens when the form is saved). Each rule " @@ -8246,76 +8293,76 @@ msgstr "" "Daarnaast zullen ook andere snelheidsverbeteringen toegepast worden waar " "mogelijk." -#: openforms/forms/models/form.py:451 +#: openforms/forms/models/form.py:457 msgid "forms" msgstr "formulieren" -#: openforms/forms/models/form.py:455 +#: openforms/forms/models/form.py:461 #, python-brace-format msgid "{name} (deleted)" msgstr "{name} (verwijderd)" -#: openforms/forms/models/form.py:515 openforms/forms/models/form.py:531 -#: openforms/forms/models/form.py:544 +#: openforms/forms/models/form.py:521 openforms/forms/models/form.py:537 +#: openforms/forms/models/form.py:550 #, python-brace-format msgid "{backend} (invalid)" msgstr "{backend} (ongeldig)" -#: openforms/forms/models/form.py:522 +#: openforms/forms/models/form.py:528 msgid "registrations" msgstr "Registraties" -#: openforms/forms/models/form.py:549 +#: openforms/forms/models/form.py:555 msgid "logins" msgstr "inloggen" -#: openforms/forms/models/form.py:594 openforms/forms/models/form.py:609 +#: openforms/forms/models/form.py:600 openforms/forms/models/form.py:615 #: openforms/forms/models/form_definition.py:106 #: openforms/forms/models/form_definition.py:124 #, python-brace-format msgid "{name} (copy)" msgstr "{name} (kopie)" -#: openforms/forms/models/form.py:598 +#: openforms/forms/models/form.py:604 #: openforms/forms/models/form_definition.py:110 #, python-brace-format msgid "{slug}-copy" msgstr "{slug}-kopie" -#: openforms/forms/models/form.py:721 +#: openforms/forms/models/form.py:795 #, python-brace-format msgid "Restored form version {version} (from {created})." msgstr "Formulierversie {version} hersteld (van {created})." -#: openforms/forms/models/form.py:843 +#: openforms/forms/models/form.py:917 msgid "export content" msgstr "Exportinhoud" -#: openforms/forms/models/form.py:845 +#: openforms/forms/models/form.py:919 msgid "Zip file containing all the exported forms." msgstr "Het ZIP-bestand met daarin alle geëxporteerde formulieren." -#: openforms/forms/models/form.py:848 +#: openforms/forms/models/form.py:922 msgid "date time requested" msgstr "Moment van aanvraag" -#: openforms/forms/models/form.py:849 +#: openforms/forms/models/form.py:923 msgid "The date and time on which the bulk export was requested." msgstr "De datum en tijdstip waarop de bulk export aangevraagd is." -#: openforms/forms/models/form.py:855 +#: openforms/forms/models/form.py:929 msgid "The user that requested the download." msgstr "De gebruiker die de download heeft aangevraagd." -#: openforms/forms/models/form.py:862 +#: openforms/forms/models/form.py:936 msgid "forms export" msgstr "formulierenexport" -#: openforms/forms/models/form.py:863 +#: openforms/forms/models/form.py:937 msgid "forms exports" msgstr "Formulierexports" -#: openforms/forms/models/form.py:866 +#: openforms/forms/models/form.py:940 #, python-format msgid "Bulk export requested by %(username)s on %(datetime)s" msgstr "Bulk-export aangevraagd door %(username)s op %(datetime)s" @@ -8772,16 +8819,26 @@ msgstr "" " e-mail met een link naar het ZIP-bestand dat alle geëxporteerde formulieren" " bevat.

" -#: openforms/forms/templates/admin/forms/form/export.html:72 +#: openforms/forms/templates/admin/forms/form/export.html:81 #: openforms/forms/templates/admin/forms/formsubmissionstatistics/export_form.html:62 msgid "Export" msgstr "Exporteren" -#: openforms/forms/templates/admin/forms/form/import_form.html:9 +#: openforms/forms/templates/admin/forms/form/import_form.html:10 msgid "Import forms" msgstr "Importeer formulieren" -#: openforms/forms/templates/admin/forms/form/import_form.html:33 +#: openforms/forms/templates/admin/forms/form/import_form.html:27 +msgid "" +"

All hyperlinks in the email templates, whose (sub)domains are not " +"provided in the global configuration allowlist, will be removed during " +"import.

" +msgstr "" +"

Alle hyperlinks in de e-mailsjablonen waarvan de (sub)domeinen niet zijn " +"opgenomen in de toelatingslijst van de algemene configuratie, worden tijdens" +" het importeren verwijderd.

" + +#: openforms/forms/templates/admin/forms/form/import_form.html:65 msgid "Import" msgstr "Importeren" diff --git a/src/openforms/forms/admin/tasks.py b/src/openforms/forms/admin/tasks.py index bb41d8c0a9..8ca58b1c8b 100644 --- a/src/openforms/forms/admin/tasks.py +++ b/src/openforms/forms/admin/tasks.py @@ -18,16 +18,18 @@ from openforms.celery import app from openforms.emails.utils import send_mail_html from openforms.forms.import_export.export_form import export_form +from openforms.forms.import_export.import_form import import_form from openforms.forms.import_export.typing import ( FormExportOptions, FormExportOptionsData, + FormImportOptions, + FormImportOptionsData, ) from openforms.logging import audit_logger from openforms.utils.urls import build_absolute_uri from ..models import Form from ..models.form import FormsExport -from ..utils import import_form logger = structlog.stdlib.get_logger(__name__) @@ -83,7 +85,9 @@ def process_forms_export( @app.task(ignore_result=True) -def process_forms_import(import_file: str, user_id: int) -> None: +def process_forms_import( + import_file: str, user_id: int, import_options: FormImportOptionsData +) -> None: failed_files: list[tuple[str, object]] = [] # This deletes the temp dir once the context manager is exited with tempfile.TemporaryDirectory() as temp_dir: @@ -92,7 +96,8 @@ def process_forms_import(import_file: str, user_id: int) -> None: try: # This normalises the path before extracting the files (to avoid writing outside the temp_dir) import_form( - zip_file.extract(member=zipped_form_file, path=temp_dir) + zip_file.extract(member=zipped_form_file, path=temp_dir), + import_options=FormImportOptions(**import_options), ) except ValidationError as exc: filename = Path(zipped_form_file.filename).name diff --git a/src/openforms/forms/admin/views.py b/src/openforms/forms/admin/views.py index d1fda317d9..03eeb5240b 100644 --- a/src/openforms/forms/admin/views.py +++ b/src/openforms/forms/admin/views.py @@ -22,16 +22,18 @@ from privates.storages import private_media_storage from rest_framework.exceptions import ValidationError +from openforms.forms.import_export.import_form import import_form from openforms.forms.import_export.typing import ( AdditionalFormConfigurationOptions, FormConfigurationOptions, + FormImportOptions, + FormImportOptionsData, ) from openforms.logging import audit_logger from ..forms import ExportStatisticsForm from ..forms.form import FormImportForm from ..models import Form, FormsExport, FormSubmissionStatistics -from ..utils import import_form from .tasks import process_forms_export, process_forms_import @@ -134,9 +136,24 @@ def form_valid(self, form): import_file = form.cleaned_data["file"] is_bulk_import = self.get_is_bulk_import(import_file) + import_options = { + field_name: form.cleaned_data[field_name] + for field_name in ( + "form_configuration", + "reuse_form_definitions", + "additional_form_configuration", + "theme", + "category", + ) + if field_name in form.cleaned_data + } + if not is_bulk_import: try: - import_form(import_file) + import_form( + import_file, + import_options=FormImportOptions(**import_options), + ) except ValidationError as exc: messages.error( self.request, @@ -144,7 +161,7 @@ def form_valid(self, form): ) return super().form_invalid(form) else: - self._bulk_import_forms(import_file) + self._bulk_import_forms(import_file, import_options) return super().form_valid(form) @@ -158,11 +175,11 @@ def get_success_message(self, cleaned_data): "The bulk import is being processed! The imported forms will soon be available." ) - def _bulk_import_forms(self, import_file): + def _bulk_import_forms(self, import_file, import_options: FormImportOptionsData): name = f"imports/import_forms_{uuid4()}.zip" filename = private_media_storage.save(name, import_file) - process_forms_import.delay(filename, self.request.user.id) + process_forms_import.delay(filename, self.request.user.id, import_options) @method_decorator(staff_member_required, name="dispatch") diff --git a/src/openforms/forms/api/serializers/form.py b/src/openforms/forms/api/serializers/form.py index 3e71bea31a..1aba22628a 100644 --- a/src/openforms/forms/api/serializers/form.py +++ b/src/openforms/forms/api/serializers/form.py @@ -543,61 +543,8 @@ def to_internal_value(self, attrs): self.convert_objects_api_group(attrs) return super().to_internal_value(attrs) - def _handle_import(self, attrs) -> None: - # we're not importing, nothing to do - if not self.context.get("is_import", False) or not hasattr( - self, "initial_data" - ): - return - - if ( - "authentication_backends" not in self.initial_data - and "authentication_backend_options" not in self.initial_data - ): - return - - # Make sure `auth_backends` exists - attrs["auth_backends"] = attrs.get("auth_backends", []) - auth_backends_map = {} - - # Pre-fill the map with the `auth_backends` values - for auth_backend in attrs["auth_backends"]: - auth_backends_map[auth_backend["backend"]] = auth_backend - - # Collect all the backends that should be transformed to `auth_backends` - if "authentication_backends" in self.initial_data: - for plugin in self.initial_data["authentication_backends"]: - # Add plugin if it's not already in the map - if plugin not in auth_backends_map: - auth_backends_map[plugin] = { - "backend": plugin, - "options": None, - } - - if "authentication_backend_options" in self.initial_data: - for plugin, options in self.initial_data[ - "authentication_backend_options" - ].items(): - if plugin not in auth_backends_map: - auth_backends_map[plugin] = { - "backend": plugin, - "options": options, - } - continue - - if auth_backends_map[plugin]["options"] is None: - auth_backends_map[plugin]["options"] = options - - validated_auth_backends = [] - for config in auth_backends_map.values(): - validated_auth_backends.append( - FormAuthenticationBackendSerializer().validate(config) - ) - attrs["auth_backends"] = validated_auth_backends - def validate(self, attrs): super().validate(attrs) - self._handle_import(attrs) self.validate_backend_options( attrs, "payment_backend", "payment_backend_options", payment_register @@ -766,6 +713,39 @@ class FormImportSerializer(serializers.Serializer): file = serializers.FileField( help_text=_("The file that contains the form, form definitions and form steps.") ) + form_configuration = serializers.MultipleChoiceField( + choices=FormConfigurationOptions.choices, + help_text=_( + "Which form configuration should be included in the export file content." + ), + required=False, + ) + reuse_form_definitions = serializers.BooleanField( + label=_("Re-use form definitions"), + required=False, + initial=True, + help_text=_( + "Whether to re-use existing form definitions or create new form definitions " + "for each form definition in the import file. (If no matching reusable form " + "definition is found, a new one will be created.)" + ), + ) + additional_form_configuration = serializers.MultipleChoiceField( + required=False, + choices=AdditionalFormConfigurationOptions.choices, + help_text=_( + "Which additional form configuration should be included in the export file " + "content." + ), + ) + theme = serializers.UUIDField( + required=False, + help_text=_("Which theme should be used for the imported forms."), + ) + category = serializers.UUIDField( + required=False, + help_text=_("Which category should be applied to the imported forms."), + ) class FormImportResponseSerializer(serializers.Serializer): diff --git a/src/openforms/forms/api/serializers/logic/form_logic.py b/src/openforms/forms/api/serializers/logic/form_logic.py index daf7e817df..d637742c86 100644 --- a/src/openforms/forms/api/serializers/logic/form_logic.py +++ b/src/openforms/forms/api/serializers/logic/form_logic.py @@ -12,9 +12,6 @@ from openforms.api.serializers import ListWithChildSerializer from ....constants import FormTypeChoices -from ....disable_next_import_conversion import ( - add_form_step_uuid_to_disable_next_actions, -) from ....logic_analysis import ( CyclesDetected, analyze_rules, @@ -194,17 +191,3 @@ def __init__(self, *args, **kwargs): related_field = self.Meta.model._meta.get_field("form") self.fields["form"].help_text = related_field.help_text self.fields["form"].label = related_field.verbose_name - - def _handle_import(self, attrs) -> None: - if not self.context.get("is_import", False): - return - - add_form_step_uuid_to_disable_next_actions( - attrs, self.context["form_variables"].variables, self.context["form_steps"] - ) - - def run_validation(self, data) -> None: - # Override `run_validation` instead of `validate`, because it runs before - # `LogicComponentActionSerializer.validate`. - self._handle_import(data) - return super().run_validation(data) diff --git a/src/openforms/forms/api/viewsets.py b/src/openforms/forms/api/viewsets.py index 7ce38f4b11..b028a73da9 100644 --- a/src/openforms/forms/api/viewsets.py +++ b/src/openforms/forms/api/viewsets.py @@ -22,7 +22,8 @@ from openforms.api.pagination import PageNumberPagination from openforms.api.serializers import ExceptionSerializer, ValidationErrorSerializer from openforms.forms.import_export.export_form import export_form -from openforms.forms.import_export.typing import FormExportOptions +from openforms.forms.import_export.import_form import import_form +from openforms.forms.import_export.typing import FormExportOptions, FormImportOptions from openforms.translations.utils import set_language_cookie from openforms.utils.patches.rest_framework_nested.viewsets import NestedViewSetMixin from openforms.utils.urls import is_admin_request, reverse_plus @@ -36,7 +37,6 @@ FormStep, FormVersion, ) -from ..utils import import_form from .datastructures import FormVariableWrapper from .documentation import get_admin_fields_markdown from .filters import FormDefinitionFilter, FormVariableFilter @@ -777,7 +777,10 @@ def restore(self, request, *args, **kwargs): class FormsImportAPIView(views.APIView): serializer_class = FormImportSerializer - parser_classes = (parsers.FileUploadParser,) + parser_classes = ( + parsers.MultiPartParser, + parsers.FormParser, + ) authentication_classes = [TokenAuthentication] permission_classes = [FormAPIPermissions] @@ -794,7 +797,22 @@ def post(self, request, *args, **kwargs): serializer = self.serializer_class(data=request.data) serializer.is_valid(raise_exception=True) - form_instance = import_form(serializer.validated_data["file"]) + form_instance = import_form( + serializer.validated_data["file"], + import_options=FormImportOptions( + **{ + field_name: serializer.validated_data[field_name] + for field_name in ( + "form_configuration", + "reuse_form_definitions", + "additional_form_configuration", + "theme", + "category", + ) + if field_name in serializer.validated_data + } + ), + ) assert form_instance response_serializer = FormImportResponseSerializer(instance=form_instance) diff --git a/src/openforms/forms/forms/form.py b/src/openforms/forms/forms/form.py index 998640f641..7507bafaad 100644 --- a/src/openforms/forms/forms/form.py +++ b/src/openforms/forms/forms/form.py @@ -1,6 +1,13 @@ from django import forms from django.utils.translation import gettext_lazy as _ +from openforms.config.models import Theme +from openforms.forms.import_export.typing import ( + AdditionalFormConfigurationOptions, + FormConfigurationOptions, +) +from openforms.forms.models import Category + class FormImportForm(forms.Form): file = forms.FileField( @@ -8,3 +15,50 @@ class FormImportForm(forms.Form): required=True, help_text=_("Upload your exported ZIP-file."), ) + form_configuration = forms.MultipleChoiceField( + label=_("Form configuration"), + required=False, + widget=forms.CheckboxSelectMultiple, + initial=[ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + choices=FormConfigurationOptions.choices, + help_text=_( + "Which form configuration should be included in the export file content." + ), + ) + additional_form_configuration = forms.MultipleChoiceField( + label=_("Additional form configuration"), + required=False, + widget=forms.CheckboxSelectMultiple, + choices=AdditionalFormConfigurationOptions.choices, + help_text=_( + "Which additional form configuration should be included in the export file " + "content." + ), + ) + reuse_form_definitions = forms.BooleanField( + label=_("Re-use form definitions"), + required=False, + initial=True, + help_text=_( + "Whether to re-use existing form definitions or create new form definitions " + "for each form definition in the import file. (If no matching reusable form " + "definition is found, a new one will be created.)" + ), + ) + theme = forms.ModelChoiceField( + label=_("Theme"), + required=False, + help_text=_("Which theme should be used for the imported form."), + queryset=Theme.objects.all(), + ) + category = forms.ModelChoiceField( + label=_("Category"), + required=False, + help_text=_("Which category should be applied to the imported form."), + queryset=Category.objects.all(), + ) diff --git a/src/openforms/forms/import_export/import_form.py b/src/openforms/forms/import_export/import_form.py new file mode 100644 index 0000000000..c9f98fa238 --- /dev/null +++ b/src/openforms/forms/import_export/import_form.py @@ -0,0 +1,424 @@ +import json +import zipfile +from collections.abc import Collection +from typing import Required, TypedDict + +from django.conf import settings +from django.db import transaction +from django.utils.translation import override + +from rest_framework.exceptions import ValidationError +from rest_framework.request import Request +from rest_framework.test import APIRequestFactory + +from openforms.forms.api.datastructures import FormVariableWrapper +from openforms.forms.models import ( + Form, + FormDefinition, +) +from openforms.registrations.contrib.objects_api.constants import ( + PLUGIN_IDENTIFIER as OBJECTS_API_PLUGIN_IDENTIFIER, +) +from openforms.registrations.contrib.stuf_zds.plugin import ( + PLUGIN_IDENTIFIER as STUF_ZDS_PLUGIN_IDENTIFIER, +) +from openforms.registrations.contrib.zgw_apis.plugin import ( + PLUGIN_IDENTIFIER as ZGW_APIS_PLUGIN_IDENTIFIER, +) + +from .matchers.form_definition import FormDefinitionMatcher +from .serializers import ( + FormDefinitionImportSerializer, + FormImportSerializer, + FormLogicImportSerializer, + FormStepImportSerializer, + FormVariableImportSerializer, +) +from .typing import FormImportOptions +from .utils import import_additional_form_configuration_data + +EXPECTED_RESOURCES = ( + "forms", + "formDefinitions", + "formSteps", + "formVariables", + "formLogic", + "product", + "wmsTileLayers", + "wmtsTileLayers", + "yiviAttributeGroups", +) + + +def _get_mock_request(): + factory = APIRequestFactory() + first_allowed_host = ( + settings.ALLOWED_HOSTS[0] if settings.ALLOWED_HOSTS else "testserver" + ) + server_name = first_allowed_host if first_allowed_host != "*" else "testserver" + request = factory.get("/", SERVER_NAME=server_name) + request.is_mock_request = True # pyright: ignore[reportAttributeAccessIssue] + return request + + +@transaction.atomic +def import_form( + import_file, import_options: FormImportOptions, existing_form_instance=None +) -> Form | None: + import_data = {} + with zipfile.ZipFile(import_file, "r") as zip_file: + for resource in EXPECTED_RESOURCES: + if f"{resource}.json" in zip_file.namelist(): + import_data[resource] = zip_file.read(f"{resource}.json").decode() + + return import_form_data(import_data, import_options, existing_form_instance) + + +@transaction.atomic +@override(language=settings.LANGUAGE_CODE) +def import_form_data( + import_data: dict, + import_options: FormImportOptions, + existing_form_instance: Form | None = None, +) -> Form | None: + uuid_mapping = {} + form_definition_matcher = FormDefinitionMatcher() + + request = _get_mock_request() + created_form: Form | None = None + + # Import additional data + import_additional_form_configuration_data( + resources=import_data, + import_options=import_options, + uuid_mapping=uuid_mapping, + restore_previous_form_version=existing_form_instance is not None, + ) + + if (forms_data := import_data.get("forms")) is not None: + created_form = _import_form_resource( + data=forms_data, + uuid_mapping=uuid_mapping, + request=request, + import_options=import_options, + existing_form_instance=existing_form_instance, + ) + + if (form_definitions_data := import_data.get("formDefinitions")) is not None: + form_definitions = _import_form_definition_resources( + data=form_definitions_data, + uuid_mapping=uuid_mapping, + form=created_form, + request=request, + import_options=import_options, + matcher=form_definition_matcher, + ) + move_file_registration_options(created_form, form_definitions) + + if (form_steps_data := import_data.get("formSteps")) is not None: + _import_form_step_resources( + data=form_steps_data, + uuid_mapping=uuid_mapping, + form=created_form, + request=request, + import_options=import_options, + ) + + if (form_variables_data := import_data.get("formVariables")) is not None: + _import_form_variable_resources( + data=form_variables_data, + uuid_mapping=uuid_mapping, + form=created_form, + request=request, + import_options=import_options, + ) + + if (form_logic_data := import_data.get("formLogic")) is not None: + _import_form_logic_resources( + data=form_logic_data, + uuid_mapping=uuid_mapping, + form=created_form, + request=request, + import_options=import_options, + ) + + return created_form + + +def _import_form_resource( + data: dict, + uuid_mapping: dict[str, str], + request: Request, + import_options: FormImportOptions, + existing_form_instance: Form | None, +) -> Form | None: + imported_form = None + + for old, new in uuid_mapping.items(): + data = data.replace(old, new) + + for entry in json.loads(data): + old_uuid = entry.get("uuid") + + deserialized = FormImportSerializer( + data=entry, + context={ + "request": request, + "is_import": True, + "import_options": import_options, + }, + instance=existing_form_instance, + ) + + try: + deserialized.is_valid(raise_exception=True) + imported_form = deserialized.save() + + if hasattr(deserialized.instance, "uuid") and "uuid" in entry: + uuid_mapping[old_uuid] = str(deserialized.instance.uuid) + except ValidationError as e: + raise e + + return imported_form + + +def _import_form_definition_resources( + data: dict, + uuid_mapping: dict[str, str], + form: Form, + request: Request, + import_options: FormImportOptions, + matcher: FormDefinitionMatcher, +) -> list[FormDefinition]: + form_definitions: list[FormDefinition] = [] + + for old, new in uuid_mapping.items(): + data = data.replace(old, new) + + for entry in json.loads(data): + old_uuid = entry.get("uuid") + + instance: FormDefinition | None = None + if import_options.reuse_form_definitions: + instance = matcher.find(entry.get("configuration")) + + deserialized = FormDefinitionImportSerializer( + data=entry, + context={ + "request": request, + "form": form, + "is_import": True, + "import_options": import_options, + }, + instance=instance, + ) + + try: + deserialized.is_valid(raise_exception=True) + form_definitions.append(deserialized.save()) + + if hasattr(deserialized.instance, "uuid") and "uuid" in entry: + uuid_mapping[old_uuid] = str(deserialized.instance.uuid) + except ValidationError as e: + raise e + + return form_definitions + + +def _import_form_step_resources( + data: dict, + uuid_mapping: dict[str, str], + form: Form, + request: Request, + import_options: FormImportOptions, +): + for old, new in uuid_mapping.items(): + data = data.replace(old, new) + + for entry in json.loads(data): + old_uuid = entry.get("uuid") + + deserialized = FormStepImportSerializer( + data=entry, + context={ + "request": request, + "form": form, + "is_import": True, + "import_options": import_options, + }, + ) + + try: + deserialized.is_valid(raise_exception=True) + deserialized.save() + + if hasattr(deserialized.instance, "uuid") and "uuid" in entry: + uuid_mapping[old_uuid] = str(deserialized.instance.uuid) + except ValidationError as e: + raise e + + +def _import_form_variable_resources( + data: dict, + uuid_mapping: dict[str, str], + form: Form, + request: Request, + import_options: FormImportOptions, +): + for old, new in uuid_mapping.items(): + data = data.replace(old, new) + + for entry in json.loads(data): + old_uuid = entry.get("uuid") + + deserialized = FormVariableImportSerializer( + data=entry, + context={ + "request": request, + "form": form, + "is_import": True, + "import_options": import_options, + "forms": ({str(form.uuid): form}), + "form_definitions": { + str(fd.uuid): fd + for fd in FormDefinition.objects.filter(formstep__form=form) + }, + }, + ) + + try: + deserialized.is_valid(raise_exception=True) + deserialized.save() + + if hasattr(deserialized.instance, "uuid") and "uuid" in entry: + uuid_mapping[old_uuid] = str(deserialized.instance.uuid) + except ValidationError as e: + raise e + + +def _import_form_logic_resources( + data: dict, + uuid_mapping: dict[str, str], + form: Form, + request: Request, + import_options: FormImportOptions, +): + for old, new in uuid_mapping.items(): + data = data.replace(old, new) + + for entry in json.loads(data): + old_uuid = entry.get("uuid") + + deserialized = FormLogicImportSerializer( + data=entry, + context={ + "request": request, + "form": form, + "is_import": True, + "import_options": import_options, + "forms": ({str(form.uuid): form}), + "form_definitions": { + str(fd.uuid): fd + for fd in FormDefinition.objects.filter(formstep__form=form) + }, + "form_variables": FormVariableWrapper(form), + "form_steps": { + form_step.uuid: form_step + for form_step in form.formstep_set.all().order_by("order") + }, + }, + ) + + try: + deserialized.is_valid(raise_exception=True) + deserialized.save() + + if hasattr(deserialized.instance, "uuid") and "uuid" in entry: + uuid_mapping[old_uuid] = str(deserialized.instance.uuid) + except ValidationError as e: + raise e + + +class FileComponentOptions(TypedDict, total=False): + key: Required[str] + document_type_description: str + organization_rsin: str + confidentiality_level: str + title: str + + +# Original commit 2d1ef3cbaecd42350470864a1dbb9a134868732c +def move_file_registration_options( + form: Form, form_definitions: Collection[FormDefinition] +): + relevant_backends = [ + backend + for backend in form.registration_backends.all() + if backend.backend + in ( + OBJECTS_API_PLUGIN_IDENTIFIER, + STUF_ZDS_PLUGIN_IDENTIFIER, + ZGW_APIS_PLUGIN_IDENTIFIER, + ) + ] + if not relevant_backends: + return + + # collect all file components, including the ones inside edit grids + file_component_options: dict[str, FileComponentOptions] = {} + for fd in form_definitions: + for component in fd.configuration_wrapper: + if component["type"] != "file": + continue + if not (registration := component.get("registration")): + continue + opts: FileComponentOptions = {"key": component["key"]} + + # NOTE: we ignore the catalogue information - the backend-level catalogue + # option is used and this is validate at the serializer level + document_type_description = (registration.get("documentType") or {}).get( + "description" + ) + organization_rsin = registration.get("bronorganisatie") + confidentiality_level = registration.get("docVertrouwelijkheidaanduiding") + title = registration.get("titel") + + if document_type_description: + opts["document_type_description"] = document_type_description + if organization_rsin: + opts["organization_rsin"] = organization_rsin + if confidentiality_level: + opts["confidentiality_level"] = confidentiality_level + if title: + opts["title"] = title + + if len(opts.keys()) != 1: + file_component_options[component["key"]] = opts + + if not file_component_options: + return + + files = list(file_component_options.values()) + + def _file_for_stuf_zds(opts: FileComponentOptions): + if title := opts.get("title"): + return {"key": opts["key"], "title": title} + return None + + files_for_stuf_zds = [o for opts in files if (o := _file_for_stuf_zds(opts))] + + for backend in relevant_backends: + options = backend.options + if "files" in options: + continue + + plugin_id = backend.backend + if plugin_id in (OBJECTS_API_PLUGIN_IDENTIFIER, ZGW_APIS_PLUGIN_IDENTIFIER): + options["files"] = files + elif plugin_id == STUF_ZDS_PLUGIN_IDENTIFIER: + options["files"] = files_for_stuf_zds + else: # pragma: no cover + raise ValueError(f"Unknown registration plugin '{plugin_id}'.") + + # Persist the changes made to the registration backend + backend.save() diff --git a/src/openforms/forms/import_export/matchers/__init__.py b/src/openforms/forms/import_export/matchers/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/src/openforms/forms/import_export/matchers/form_definition.py b/src/openforms/forms/import_export/matchers/form_definition.py new file mode 100644 index 0000000000..8ab071614a --- /dev/null +++ b/src/openforms/forms/import_export/matchers/form_definition.py @@ -0,0 +1,57 @@ +import hashlib +import json + +from openforms.forms.models import FormDefinition +from openforms.typing import JSONObject, JSONValue + + +def normalize_configuration(value: JSONValue) -> JSONValue: + """ + Normalize the form configuration by removing the "id" field from all components. + """ + if isinstance(value, dict): + return { + key: normalize_configuration(child) + for key, child in value.items() + if key != "id" + } + + if isinstance(value, list): + return [normalize_configuration(child) for child in value] + + return value + + +def configuration_fingerprint(configuration: JSONObject) -> str: + """ + Turn form definition configuration into a fingerprint. + """ + normalized = normalize_configuration(configuration) + + serialized = json.dumps( + normalized, + sort_keys=True, + separators=(",", ":"), + ) + + return hashlib.sha256(serialized.encode("utf-8")).hexdigest() + + +class FormDefinitionMatcher: + def __init__(self): + self._instances_by_configuration = self._build_lookup() + + def _build_lookup(self) -> dict[str, list[FormDefinition]]: + result: dict[str, list[FormDefinition]] = {} + + for instance in (fd for fd in FormDefinition.objects.all() if fd.is_reusable): + key = configuration_fingerprint(instance.configuration) + result.setdefault(key, []).append(instance) + + return result + + def find(self, configuration: JSONObject) -> FormDefinition | None: + expected_fingerprint = configuration_fingerprint(configuration) + matches = self._instances_by_configuration.get(expected_fingerprint, None) + + return matches[0] if matches else None diff --git a/src/openforms/forms/import_export/resources/base.py b/src/openforms/forms/import_export/resources/base.py index 310176aedb..c7afaf2739 100644 --- a/src/openforms/forms/import_export/resources/base.py +++ b/src/openforms/forms/import_export/resources/base.py @@ -1,10 +1,105 @@ from import_export.resources import ModelResource from openforms.forms.models import Form +from openforms.typing import JSONObject class BaseResource(ModelResource): + """ + The base resource for all import/export resources. + + Attributes: + deep_comparison_fields: + The fields to use for deep comparison when searching for an existing + instance. An instance will have to match all these fields to be considered + an exact match. + identifier_field: + The field that uniquely identifies the instance. This field will be assigned + a new identifier value when the identifier in the import-data is already in + use, and we want to create a new instance. + force_deep_compare: + Whether to force a deep comparison when searching for an existing instance. + If False, the default behavior is to use the import_id_fields to do a first + lookup, and if that fails, fall back to a deep comparison. + """ + + deep_comparison_fields = () + identifier_field: str + force_deep_compare = False + + def __init__(self, force_deep_compare: bool = False, *args, **kwargs): + self.force_deep_compare = force_deep_compare + super().__init__(*args, **kwargs) + def export_for_form(self, form: Form): raise NotImplementedError( f"{self.__class__.__name__} must implement export_for_form()" ) + + def generate_identifier(self, row: JSONObject): + """ + A method to generate a unique identifier used when creating a new instance. + """ + raise NotImplementedError( + f"{self.__class__.__name__} must implement generate_identifier()" + ) + + def get_or_init_instance(self, instance_loader, row) -> tuple[JSONObject, bool]: + """ + Search for an existing AttributeGroup with matching attributes. + + The default get_or_init_instance uses the import_id_fields. This does a lookup + using the resource unique identifier. + + If the identifier lookup doesn't find an existing instance, we do a deep + comparison and return the first match. + + Finally, if no match is found, we create a new instance. + + The return value is a tuple of (instance, is_new). + """ + if not self.force_deep_compare: + instance = super().get_instance(instance_loader, row) + + # Return the existing instance, found using the import_id_fields. + if instance is not None: + row["_matched_existing_instance"] = True + return instance, False + + # Collect the parameters for the deep comparison. + params = {} + for key in self.deep_comparison_fields: + params[key] = self.fields[key].clean(row) + + # Perform the deep comparison and return the first match. + if ( + params + and (new_found := self.get_queryset().filter(**params).first()) is not None + ): + row["_matched_existing_instance"] = True + return new_found, False + + # No match found, return a new instance. + new_instance = super().init_instance(row) + new_instance._generated_new_identifier = True + return new_instance, True + + def import_field(self, field, instance, row, is_m2m=False, **kwargs) -> None: + if field.attribute == self.identifier_field and getattr( + instance, "_generated_new_identifier", False + ): + # Set the identifier field to a new value, generated by the subclass. + setattr(instance, field.attribute, self.generate_identifier(row)) + return + + # Import the field value normally. + super().import_field(field, instance, row, is_m2m=is_m2m, **kwargs) + + def skip_row(self, instance, original, row, import_validation_errors=None): + """ + When importing an existing instance, we don't update it. + """ + if row.get("_matched_existing_instance"): + return True + + return super().skip_row(instance, original, row, import_validation_errors) diff --git a/src/openforms/forms/import_export/resources/product.py b/src/openforms/forms/import_export/resources/product.py index 09dfe1fffc..36339be93f 100644 --- a/src/openforms/forms/import_export/resources/product.py +++ b/src/openforms/forms/import_export/resources/product.py @@ -1,3 +1,5 @@ +from uuid import uuid4 + from openforms.forms.models import Form from openforms.products.models import Product @@ -5,9 +7,18 @@ class ProductResource(BaseResource): + deep_comparison_fields = ("name", "price", "information") + identifier_field = "uuid" + class Meta: model = Product + import_id_fields = ("uuid",) fields = ("uuid", "name", "price", "information") + store_instance = True + store_row_values = True def export_for_form(self, form: Form): return self.export(queryset=[form.product] if form.product is not None else []) + + def generate_identifier(self, row): + return uuid4() diff --git a/src/openforms/forms/import_export/resources/wms_tile_layer.py b/src/openforms/forms/import_export/resources/wms_tile_layer.py index cb122dd9e7..74ddcde3f6 100644 --- a/src/openforms/forms/import_export/resources/wms_tile_layer.py +++ b/src/openforms/forms/import_export/resources/wms_tile_layer.py @@ -1,3 +1,5 @@ +from uuid import uuid4 + from openforms.config.models import MapWMSTileLayer from openforms.forms.models import Form @@ -5,9 +7,15 @@ class WMSTileLayerResource(BaseResource): + deep_comparison_fields = ("name", "url") + identifier_field = "uuid" + class Meta: model = MapWMSTileLayer + import_id_fields = ("uuid",) fields = ("uuid", "name", "url") + store_instance = True + store_row_values = True def export_for_form(self, form: Form): wms_tile_layers = [] @@ -26,3 +34,6 @@ def export_for_form(self, form: Form): return self.export( queryset=MapWMSTileLayer.objects.filter(uuid__in=list(set(wms_tile_layers))) ) + + def generate_identifier(self, row): + return uuid4() diff --git a/src/openforms/forms/import_export/resources/wmts_tile_layer.py b/src/openforms/forms/import_export/resources/wmts_tile_layer.py index 71cbc2c7aa..d183e26656 100644 --- a/src/openforms/forms/import_export/resources/wmts_tile_layer.py +++ b/src/openforms/forms/import_export/resources/wmts_tile_layer.py @@ -1,3 +1,5 @@ +import re + from openforms.config.models import MapTileLayer from openforms.forms.models import Form @@ -5,9 +7,15 @@ class WMTSTileLayerResource(BaseResource): + deep_comparison_fields = ("label", "url") + identifier_field = "identifier" + class Meta: model = MapTileLayer + import_id_fields = ("identifier",) fields = ("identifier", "label", "url") + store_instance = True + store_row_values = True def export_for_form(self, form: Form): wmts_tile_layers = [] @@ -24,3 +32,21 @@ def export_for_form(self, form: Form): identifier__in=list(set(wmts_tile_layers)) ) ) + + def generate_identifier(self, row): + current_identifier = row["identifier"] + + # In case the identifier already has a numeric suffix, use that as the starting + # point. + match = re.match(r"^(.*?)(\d+)?$", current_identifier) + identifier_base = match.group(1) + identifier_index = int(match.group(2)) if match.group(2) is not None else 0 + + new_identifier = current_identifier + existing_identifiers = MapTileLayer.objects.values_list("identifier", flat=True) + + while new_identifier in existing_identifiers: + identifier_index += 1 + new_identifier = "".join((identifier_base, str(identifier_index))) + + return new_identifier diff --git a/src/openforms/forms/import_export/resources/yivi_attribute_group.py b/src/openforms/forms/import_export/resources/yivi_attribute_group.py index db2eb28558..45086f3d93 100644 --- a/src/openforms/forms/import_export/resources/yivi_attribute_group.py +++ b/src/openforms/forms/import_export/resources/yivi_attribute_group.py @@ -1,3 +1,5 @@ +from uuid import uuid4 + from openforms.authentication.contrib.yivi_oidc.models import AttributeGroup from openforms.forms.models import Form @@ -5,9 +7,15 @@ class YiviAttributeGroupResource(BaseResource): + deep_comparison_fields = ("name", "description", "attributes") + identifier_field = "uuid" + class Meta: model = AttributeGroup + import_id_fields = ("uuid",) fields = ("uuid", "name", "description", "attributes") + store_instance = True + store_row_values = True def export_for_form(self, form: Form): yivi_auth_backend = form.auth_backends.all().filter(backend="yivi_oidc").first() @@ -22,3 +30,6 @@ def export_for_form(self, form: Form): uuid__in=yivi_auth_backend.options["additional_attributes_groups"] ) ) + + def generate_identifier(self, row): + return uuid4() diff --git a/src/openforms/forms/import_export/serializers/__init__.py b/src/openforms/forms/import_export/serializers/__init__.py index 663b367518..c5d903cf98 100644 --- a/src/openforms/forms/import_export/serializers/__init__.py +++ b/src/openforms/forms/import_export/serializers/__init__.py @@ -1,13 +1,21 @@ -from .form import FormExportSerializer -from .form_definition import FormDefinitionExportSerializer -from .form_logic import FormLogicExportSerializer -from .form_step import FormStepExportSerializer -from .form_variable import FormVariableExportSerializer +from .form import FormExportSerializer, FormImportSerializer +from .form_definition import ( + FormDefinitionExportSerializer, + FormDefinitionImportSerializer, +) +from .form_logic import FormLogicExportSerializer, FormLogicImportSerializer +from .form_step import FormStepExportSerializer, FormStepImportSerializer +from .form_variable import FormVariableExportSerializer, FormVariableImportSerializer __all__ = [ "FormExportSerializer", + "FormImportSerializer", "FormDefinitionExportSerializer", + "FormDefinitionImportSerializer", "FormLogicExportSerializer", + "FormLogicImportSerializer", "FormStepExportSerializer", + "FormStepImportSerializer", "FormVariableExportSerializer", + "FormVariableImportSerializer", ] diff --git a/src/openforms/forms/import_export/serializers/base.py b/src/openforms/forms/import_export/serializers/base.py index 2088e2497a..e899547633 100644 --- a/src/openforms/forms/import_export/serializers/base.py +++ b/src/openforms/forms/import_export/serializers/base.py @@ -1,5 +1,6 @@ from collections.abc import Sequence from typing import ClassVar +from uuid import uuid4 from rest_framework import serializers @@ -7,6 +8,7 @@ AdditionalFormConfigurationCleanup, FormConfigurationCleanup, FormExportOptions, + FormImportOptions, ) from openforms.typing import JSONObject @@ -85,3 +87,62 @@ def remove_excluded_additional_form_configuration( def get_export_options(self) -> FormExportOptions | None: return self.context.get("export_options", None) + + +class BaseImportSerializer(serializers.Serializer): + excluded_form_configuration_removal: list[FormConfigurationCleanup] = () + excluded_additional_form_configuration_removal: list[ + AdditionalFormConfigurationCleanup + ] = () + + def to_internal_value(self, instance): + value = instance.copy() + + # When importing an existing instance, we should not overwrite the uuid + if not self.instance: + value = self.set_new_uuid(value) + + value = self.apply_backwards_compatibility(value) + + value = self.remove_excluded_form_configuration(value) + value = self.remove_excluded_additional_form_configuration(value) + + return super().to_internal_value(value) + + def set_new_uuid(self, value: JSONObject) -> JSONObject: + value["uuid"] = uuid4() + return value + + def apply_backwards_compatibility(self, value: JSONObject) -> JSONObject: + return value + + def remove_excluded_form_configuration(self, value: JSONObject) -> JSONObject: + selected_options = ( + set(import_options.form_configuration) + if (import_options := self.get_import_options()) is not None + else [] + ) + + for config in self.excluded_form_configuration_removal: + if config.option not in selected_options: + config.cleanup(value) + + return value + + def remove_excluded_additional_form_configuration( + self, value: JSONObject + ) -> JSONObject: + selected_options = ( + set(import_options.additional_form_configuration) + if (import_options := self.get_import_options()) is not None + else [] + ) + + for config in self.excluded_additional_form_configuration_removal: + if config.option not in selected_options: + config.cleanup(value) + + return value + + def get_import_options(self) -> FormImportOptions | None: + return self.context.get("import_options", None) diff --git a/src/openforms/forms/import_export/serializers/form.py b/src/openforms/forms/import_export/serializers/form.py index 5cda8fd231..2d00964289 100644 --- a/src/openforms/forms/import_export/serializers/form.py +++ b/src/openforms/forms/import_export/serializers/form.py @@ -1,14 +1,26 @@ +import random +import string + +from django.urls import reverse + +from openforms.config.models import Theme +from openforms.emails.utils import sanitize_content from openforms.forms.api.serializers import FormSerializer -from openforms.forms.api.serializers.form import FormRegistrationBackendSerializer +from openforms.forms.api.serializers.form import ( + FormAuthenticationBackendSerializer, + FormRegistrationBackendSerializer, +) +from openforms.forms.constants import FormTypeChoices from openforms.forms.import_export.typing import ( AdditionalFormConfigurationCleanup, AdditionalFormConfigurationOptions, FormConfigurationCleanup, FormConfigurationOptions, ) +from openforms.forms.models import Category, Form from openforms.typing import JSONObject -from .base import BaseExportSerializer +from .base import BaseExportSerializer, BaseImportSerializer def clear_product(representation: JSONObject): @@ -152,3 +164,166 @@ def get_fields(self): if "payment_options" in fields: del fields["payment_options"] return fields + + +class FormImportSerializer(FormSerializer, BaseImportSerializer): + excluded_additional_form_configuration_removal = ( + AdditionalFormConfigurationCleanup( + option=AdditionalFormConfigurationOptions.product, + cleanup=clear_product, + ), + AdditionalFormConfigurationCleanup( + option=AdditionalFormConfigurationOptions.yivi_attribute_groups, + cleanup=clear_yivi_attribute_groups, + ), + ) + excluded_form_configuration_removal = ( + FormConfigurationCleanup( + option=FormConfigurationOptions.registration_backends, + cleanup=exclude_registration_backends, + ), + FormConfigurationCleanup( + option=FormConfigurationOptions.payment_backend, + cleanup=exclude_payment_backend, + ), + FormConfigurationCleanup( + option=FormConfigurationOptions.auth_backends, + cleanup=exclude_auth_backends, + ), + ) + + def to_internal_value(self, instance): + value = instance.copy() + + value = self.set_theme(value) + value = self.set_category(value) + + # We remove all unknown domains from the email templates + value = self.sanitize_email_templates(value) + + # When importing a form, it should be non-active by default + value["active"] = False + + # Make sure the slug is unique + if Form.objects.filter(slug=value.get("slug")).first() is not None: + value["slug"] = ( + f"{value['slug']}-{''.join(random.choices(string.hexdigits, k=6))}" + ) + + return super().to_internal_value(value) + + def set_theme(self, value: JSONObject) -> JSONObject: + if (import_options := self.get_import_options()) is not None and ( + theme := Theme.objects.filter(uuid=import_options.theme).first() + ): + theme_url = reverse("api:themes-detail", args=[theme.uuid]) + value["theme"] = theme_url + else: + value["theme"] = None + + return value + + def set_category(self, value: JSONObject) -> JSONObject: + if (import_options := self.get_import_options()) is not None and ( + category := Category.objects.filter(uuid=import_options.category).first() + ): + category_url = reverse("api:categories-detail", args=[category.uuid]) + value["category"] = category_url + else: + value["category"] = None + + return value + + def apply_backwards_compatibility(self, value: JSONObject) -> JSONObject: + # forms before v4.0 do not have the type field so in case we import an + # old appointment form we have to make sure that the form has the right + # type configured (by default is regular) + # Original commit d8b1d4ea9d31772f059a388347e8a4688be5d717 + if appointment_options := value.get("appointment_options"): + if appointment_options.get("is_appointment"): + value["type"] = FormTypeChoices.appointment + + # In v3.2 the authentication_backends field was replaced with auth_backends. This + # converter ensures that pre-v3.2 forms are converted correctly. See #5140 + # Original commit d08281dad2e426e2655d87f67cefec9b58c5c810 + if ( + "authentication_backends" in value + or "authentication_backend_options" in value + ): + # Make sure `auth_backends` exists + value["auth_backends"] = value.get("auth_backends", []) + auth_backends_map = {} + + # Pre-fill the map with the `auth_backends` values + for auth_backend in value["auth_backends"]: + auth_backends_map[auth_backend["backend"]] = auth_backend + + # Collect all the backends that should be transformed to `auth_backends` + if "authentication_backends" in value: + for plugin in value["authentication_backends"]: + # Add plugin if it's not already in the map + if plugin not in auth_backends_map: + auth_backends_map[plugin] = { + "backend": plugin, + "options": None, + } + + if "authentication_backend_options" in value: + for plugin, options in value["authentication_backend_options"].items(): + if plugin not in auth_backends_map: + auth_backends_map[plugin] = { + "backend": plugin, + "options": options, + } + continue + + if auth_backends_map[plugin]["options"] is None: + auth_backends_map[plugin]["options"] = options + + validated_auth_backends = [] + for config in auth_backends_map.values(): + validated_auth_backends.append( + FormAuthenticationBackendSerializer().validate(config) + ) + value["auth_backends"] = validated_auth_backends + + return value + + def sanitize_email_templates(self, value: JSONObject) -> JSONObject: + # Sanitize confirmation email templates + if value.get("confirmation_email_template", None) is not None: + email_template = value["confirmation_email_template"] + + if email_template.get("content") is not None: + email_template["content"] = sanitize_content(email_template["content"]) + + if email_template.get("cosign_content") is not None: + email_template["cosign_content"] = sanitize_content( + email_template["cosign_content"] + ) + + for translation in email_template.get("translations", {}).values(): + if translation.get("content") is not None: + translation["content"] = sanitize_content(translation["content"]) + + if translation.get("cosign_content") is not None: + translation["cosign_content"] = sanitize_content( + translation["cosign_content"] + ) + + # Sanitize email registration backend email templates + for registration in value.get("registration_backends", []): + if registration["backend"] == "email": + options = registration["options"] + + if options.get("email_content_template_html") is not None: + options["email_content_template_html"] = sanitize_content( + options["email_content_template_html"] + ) + + if options.get("email_content_template_text") is not None: + options["email_content_template_text"] = sanitize_content( + options["email_content_template_text"] + ) + + return value diff --git a/src/openforms/forms/import_export/serializers/form_definition.py b/src/openforms/forms/import_export/serializers/form_definition.py index a14240a421..536f6cda97 100644 --- a/src/openforms/forms/import_export/serializers/form_definition.py +++ b/src/openforms/forms/import_export/serializers/form_definition.py @@ -1,3 +1,6 @@ +import structlog + +from openforms.formio.migration_converters import CONVERTERS, DEFINITION_CONVERTERS from openforms.formio.utils import iter_components from openforms.forms.api.serializers import FormDefinitionSerializer from openforms.forms.import_export.typing import ( @@ -9,7 +12,9 @@ from openforms.prefill.constants import IdentifierRoles from openforms.typing import JSONObject -from .base import BaseExportSerializer +from .base import BaseExportSerializer, BaseImportSerializer + +logger = structlog.stdlib.get_logger(__name__) def clear_wms_tile_layers(representation: JSONObject): @@ -87,3 +92,53 @@ def remove_sensitive_content(self, instance, representation): component["defaultValue"] = "" return representation + + +class FormDefinitionImportSerializer(FormDefinitionSerializer, BaseImportSerializer): + excluded_additional_form_configuration_removal = ( + AdditionalFormConfigurationCleanup( + option=AdditionalFormConfigurationOptions.wms_tile_layers, + cleanup=clear_wms_tile_layers, + ), + AdditionalFormConfigurationCleanup( + option=AdditionalFormConfigurationOptions.wmts_tile_layers, + cleanup=clear_wmts_tile_layers, + ), + ) + excluded_form_configuration_removal = ( + FormConfigurationCleanup( + option=FormConfigurationOptions.prefill, + cleanup=remove_prefill_from_component_configuration, + ), + ) + + def to_internal_value(self, instance): + value = instance.copy() + + if configuration := value.get("configuration"): + self.apply_component_conversions(configuration) + self.apply_definition_conversions(configuration) + + return super().to_internal_value(value) + + def apply_component_conversions(self, configuration: JSONObject): + """ + Apply the known formio component conversions to the entire form definition. + """ + log = logger.bind(action="forms.apply_component_conversions") + for component in iter_components(configuration): + if not (component_type := component.get("type")): # pragma: no cover + continue + if not (converters := CONVERTERS.get(component_type)): + continue + for identifier, apply_converter in converters.items(): + log.debug( + "apply_converter", + component_type=component_type, + identifier=identifier, + ) + apply_converter(component) + + def apply_definition_conversions(self, configuration: JSONObject): + for converter in DEFINITION_CONVERTERS: + converter(configuration) diff --git a/src/openforms/forms/import_export/serializers/form_logic.py b/src/openforms/forms/import_export/serializers/form_logic.py index 6a20db6bbe..863059dbf4 100644 --- a/src/openforms/forms/import_export/serializers/form_logic.py +++ b/src/openforms/forms/import_export/serializers/form_logic.py @@ -1,6 +1,11 @@ from openforms.forms.api.serializers import FormLogicSerializer +from openforms.forms.constants import LogicActionTypes +from openforms.forms.disable_next_import_conversion import ( + add_form_step_uuid_to_disable_next_actions, +) +from openforms.typing import JSONObject -from .base import BaseExportSerializer +from .base import BaseExportSerializer, BaseImportSerializer class FormLogicExportSerializer(FormLogicSerializer, BaseExportSerializer): @@ -35,3 +40,45 @@ def remove_sensitive_content(self, instance, representation): action["action"]["value"] = "" return representation + + +class FormLogicImportSerializer(FormLogicSerializer, BaseImportSerializer): + def to_internal_value(self, instance): + value = instance.copy() + + if "order" not in value: + value["order"] = 0 + + if "service_fetch_configuration" in value: + # The transferring between systems case is very tricky better not import + # these, as we don't know where this came from. Services and ids may point to + # different things in different OF instances. Even when restoring a form + # version, we don't know if the service is the same as it was before. + del value["service_fetch_configuration"] + + self.clear_old_service_fetch_config(value) + + return super().to_internal_value(value) + + def clear_old_service_fetch_config(self, rule: dict) -> None: + for action in rule["actions"]: + if action["action"]["type"] != LogicActionTypes.fetch_from_service: + continue + + if "value" not in action["action"] or action["action"]["value"] == "": + continue + + # See comment in FormVariableImportSerializer `to_internal_value` where we + # check if the variable has a `service_fetch_configuration` attribute. + # We can't reliably relate the service fetch configured to an existing configuration. + # So we don't add any existing service fetch config to the variables + action["action"]["value"] = "" + + def apply_backwards_compatibility(self, value: JSONObject) -> JSONObject: + # Preparations for 4.0, ensuring legacy imports with disable-next logic actions + # still work. See #6254 + # Original commit 879751310d0bdf6a21bdd143b54f2a3d3f095023 + add_form_step_uuid_to_disable_next_actions( + value, self.context["form_variables"].variables, self.context["form_steps"] + ) + return value diff --git a/src/openforms/forms/import_export/serializers/form_step.py b/src/openforms/forms/import_export/serializers/form_step.py index 9229fc8570..d7934e3e2c 100644 --- a/src/openforms/forms/import_export/serializers/form_step.py +++ b/src/openforms/forms/import_export/serializers/form_step.py @@ -1,6 +1,7 @@ from openforms.forms.api.serializers import FormStepSerializer +from openforms.forms.models import FormVariable -from .base import BaseExportSerializer +from .base import BaseExportSerializer, BaseImportSerializer class FormStepExportSerializer(FormStepSerializer, BaseExportSerializer): @@ -22,3 +23,13 @@ class FormStepExportSerializer(FormStepSerializer, BaseExportSerializer): "next_text", "translations", ) + + +class FormStepImportSerializer(FormStepSerializer, BaseImportSerializer): + def save(self, *args, **kwargs): + super().save(*args, **kwargs) + + if (form := self.context.get("form")) is not None: + # Once the form steps have been created, we create the component + # FormVariables based on the form definition configurations. + FormVariable.objects.create_for_form(form) diff --git a/src/openforms/forms/import_export/serializers/form_variable.py b/src/openforms/forms/import_export/serializers/form_variable.py index a668d05263..a1755ed01f 100644 --- a/src/openforms/forms/import_export/serializers/form_variable.py +++ b/src/openforms/forms/import_export/serializers/form_variable.py @@ -6,7 +6,7 @@ from openforms.prefill.constants import IdentifierRoles from openforms.typing import JSONObject -from .base import BaseExportSerializer +from .base import BaseExportSerializer, BaseImportSerializer def remove_prefill_from_variable(representation: JSONObject): @@ -55,3 +55,24 @@ def remove_sensitive_content(self, instance, representation): return representation return representation + + +class FormVariableImportSerializer(FormVariableSerializer, BaseImportSerializer): + excluded_form_configuration_removal = ( + FormConfigurationCleanup( + option=FormConfigurationOptions.prefill, + cleanup=remove_prefill_from_variable, + ), + ) + + def to_internal_value(self, instance): + value = instance.copy() + + if "service_fetch_configuration" in value: + # The transferring between systems case is very tricky better not import + # these, as we don't know where this came from. Services and ids may point to + # different things in different OF instances. Even when restoring a form + # version, we don't know if the service is the same as it was before. + del value["service_fetch_configuration"] + + return super().to_internal_value(value) diff --git a/src/openforms/forms/import_export/typing.py b/src/openforms/forms/import_export/typing.py index cf20af6dc3..aac63f88a4 100644 --- a/src/openforms/forms/import_export/typing.py +++ b/src/openforms/forms/import_export/typing.py @@ -44,6 +44,32 @@ class FormExportOptions: ) +class FormImportOptionsData(TypedDict, total=False): + form_configuration: list[FormConfigurationOptions] + additional_form_configuration: list[AdditionalFormConfigurationOptions] + reuse_form_definitions: bool + theme: str | None + category: str | None + + +@dataclass(slots=True) +class FormImportOptions: + form_configuration: list[FormConfigurationOptions] = field( + default_factory=lambda: [ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ] + ) + additional_form_configuration: list[AdditionalFormConfigurationOptions] = field( + default_factory=list + ) + reuse_form_definitions: bool = True + theme: str | None = None + category: str | None = None + + @dataclass(frozen=True) class AdditionalFormConfigurationCleanup: option: AdditionalFormConfigurationOptions diff --git a/src/openforms/forms/import_export/utils.py b/src/openforms/forms/import_export/utils.py index a2a6f31f7c..c5ad0adaed 100644 --- a/src/openforms/forms/import_export/utils.py +++ b/src/openforms/forms/import_export/utils.py @@ -1,5 +1,8 @@ from dataclasses import dataclass +import tablib +from import_export.results import RowResult + from openforms.forms.models import Form from openforms.typing import JSONObject @@ -10,32 +13,36 @@ WMTSTileLayerResource, YiviAttributeGroupResource, ) -from .typing import AdditionalFormConfigurationOptions, FormExportOptions +from .typing import ( + AdditionalFormConfigurationOptions, + FormExportOptions, + FormImportOptions, +) @dataclass(frozen=True) -class ExportResourceConfig: +class ResourceConfig: resource: type[BaseResource] output_name: str ADDITIONAL_FORM_CONFIGURATION_RESOURCES: dict[ AdditionalFormConfigurationOptions, - ExportResourceConfig, + ResourceConfig, ] = { - AdditionalFormConfigurationOptions.product: ExportResourceConfig( + AdditionalFormConfigurationOptions.product: ResourceConfig( resource=ProductResource, output_name="product", ), - AdditionalFormConfigurationOptions.wms_tile_layers: ExportResourceConfig( + AdditionalFormConfigurationOptions.wms_tile_layers: ResourceConfig( resource=WMSTileLayerResource, output_name="wmsTileLayers", ), - AdditionalFormConfigurationOptions.wmts_tile_layers: ExportResourceConfig( + AdditionalFormConfigurationOptions.wmts_tile_layers: ResourceConfig( resource=WMTSTileLayerResource, output_name="wmtsTileLayers", ), - AdditionalFormConfigurationOptions.yivi_attribute_groups: ExportResourceConfig( + AdditionalFormConfigurationOptions.yivi_attribute_groups: ResourceConfig( resource=YiviAttributeGroupResource, output_name="yiviAttributeGroups", ), @@ -67,3 +74,45 @@ def get_additional_form_configuration_data( resources[config.output_name] = config.resource().export_for_form(form).json return resources + + +def import_additional_form_configuration_data( + resources: JSONObject, + import_options: FormImportOptions, + uuid_mapping=dict[str, str], + restore_previous_form_version: bool = False, +): + selected_options = set(import_options.additional_form_configuration) + unknown_options = selected_options - set(ADDITIONAL_FORM_CONFIGURATION_RESOURCES) + + if unknown_options: + raise ValueError( + f"Invalid additional form configuration option(s): {unknown_options}" + ) + + for option, config in ADDITIONAL_FORM_CONFIGURATION_RESOURCES.items(): + if option in selected_options and config.output_name in resources: + dataset = tablib.Dataset().load(resources[config.output_name], "json") + results = config.resource( + force_deep_compare=restore_previous_form_version + ).import_data(dataset) + + for row_result in results: + identifier_field = config.resource.identifier_field + old_identifier = row_result.row_values.get(identifier_field) + + new_identifier = None + match row_result.import_type: + case RowResult.IMPORT_TYPE_NEW: + new_identifier = getattr(row_result.instance, identifier_field) + + case RowResult.IMPORT_TYPE_SKIP: + new_identifier = getattr(row_result.original, identifier_field) + + case _: + raise ValueError( + f"Invalid import type: {row_result.import_type}" + ) + + if new_identifier is not None: + uuid_mapping[old_identifier] = str(new_identifier) diff --git a/src/openforms/forms/models/form.py b/src/openforms/forms/models/form.py index 9d9d97debd..11ca35e3b3 100644 --- a/src/openforms/forms/models/form.py +++ b/src/openforms/forms/models/form.py @@ -1,5 +1,6 @@ from __future__ import annotations +import json import uuid as _uuid from collections.abc import Iterator, Mapping from contextlib import suppress @@ -32,6 +33,11 @@ from openforms.data_removal.constants import RemovalMethods from openforms.formio.typing import Component from openforms.formio.validators import variable_key_validator +from openforms.forms.import_export.typing import ( + AdditionalFormConfigurationOptions, + FormConfigurationOptions, + FormImportOptions, +) from openforms.payments.fields import PaymentBackendChoiceField from openforms.payments.registry import register as payment_register from openforms.plugins.constants import UNIQUE_ID_MAX_LENGTH @@ -708,11 +714,40 @@ def iter_components(self, recursive=True) -> Iterator[Component]: for form_step in self.formstep_set.select_related("form_definition"): yield from form_step.iter_components(recursive=recursive) + @staticmethod + def _delete_current_form_configuration(current_form: Form): + from . import FormDefinition, FormLogic, FormStep, FormVariable + + form_steps = FormStep.objects.filter(form=current_form) + # delete single-use form definitions, they're orphan nodes when deleting the steps + fd_ids = list( + FormDefinition.objects.filter( + is_reusable=False, formstep__in=form_steps + ).values_list("id", flat=True) + ) + form_steps.delete() + FormDefinition.objects.filter(id__in=fd_ids).delete() + FormLogic.objects.filter(form=current_form).delete() + FormVariable.objects.filter(form=current_form).delete() + + @staticmethod + def _get_uuid_from_resource_url(url: str | None) -> UUID | None: + from urllib.parse import urlparse + + from django.urls import resolve + + if url is None: + return None + + result = resolve(urlparse(url).path) + return result.kwargs.get("uuid") + @transaction.atomic def restore_old_version( - self, form_version_uuid: str, user: User | None = None + self, form_version_uuid: UUID, user: User | None = None ) -> None: - from ..utils import import_form_data + from openforms.forms.import_export.import_form import import_form_data + from .form_version import FormVersion # we use the window function to find the record with its index in _all_ @@ -730,7 +765,46 @@ def restore_old_version( form_version = form_versions_mapping[form_version_uuid] old_version_data = form_version.export_blob - import_form_data(old_version_data, form_version.form) + # when restoring a previous version, delete the current form configuration; + # it will be replaced with the import data. + self._delete_current_form_configuration(form_version.form) + + forms = json.loads(old_version_data["forms"]) + assert len(forms) == 1, "expected exactly one form in the old version" + form_data = forms[0] + + theme_uuid = self._get_uuid_from_resource_url(form_data.get("theme")) + category_uuid = self._get_uuid_from_resource_url(form_data.get("category")) + + restored_form = import_form_data( + old_version_data, + FormImportOptions( + form_configuration=[ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + reuse_form_definitions=True, + theme=theme_uuid, + category=category_uuid, + ), + form_version.form, + ) + + # The FormImportSerializer sets the 'active' state to False by default. We should + # restore it to the state it had before. + if ( + previous_active := form_data.get("active") + ) is not None and previous_active is not restored_form.active: + restored_form.active = previous_active + restored_form.save() # now create a new FormVersion for this restore as well, tracking those nuances # in the description. diff --git a/src/openforms/forms/templates/admin/forms/form/export.html b/src/openforms/forms/templates/admin/forms/form/export.html index bbc8fda22d..fe6d3910bd 100644 --- a/src/openforms/forms/templates/admin/forms/form/export.html +++ b/src/openforms/forms/templates/admin/forms/form/export.html @@ -56,7 +56,16 @@

{% trans 'Export forms' %}

{{ field.errors }}
{{ field.label_tag }} - {{ field }} + + {% if field|length == 1 %} + {{ field }} + {% else %} +
    + {% for subfield in field %} +
  • {{ subfield }}
    • + {% endfor %} +
+ {% endif %}
{% if field.help_text %} diff --git a/src/openforms/forms/templates/admin/forms/form/import_form.html b/src/openforms/forms/templates/admin/forms/form/import_form.html index c258dfafa6..6f25564a10 100644 --- a/src/openforms/forms/templates/admin/forms/form/import_form.html +++ b/src/openforms/forms/templates/admin/forms/form/import_form.html @@ -2,6 +2,7 @@ {% load static i18n django_admin_index %} {% block extrastyle %}{{ block.super }} + {% endblock %} {% block nav-global %}{% include "django_admin_index/includes/app_list.html" %}{% endblock nav-global %} @@ -22,10 +23,41 @@

{% trans 'Import form' %}

+
+ {% blocktrans trimmed %} +

+ All hyperlinks in the email templates, whose (sub)domains + are not provided in the global configuration allowlist, + will be removed during import. +

+ {% endblocktrans %} +
+ {% csrf_token %} - {% for field in form %} - {{ field.label.capitalize }}: {{ field }}
- {{ field.help_text }}

+ + {% for field in form.visible_fields %} +
+ {{ field.errors }} +
+ {{ field.label_tag }} + + {% if field.field.widget.input_type == 'checkbox' and field|length > 1 %} +
    + {% for subfield in field %} +
  • {{ subfield }}
    • + {% endfor %} +
+ {% else %} + {{ field }} + {% endif %} +
+ + {% if field.help_text %} +
+
{{ field.help_text|safe }}
+
+ {% endif %} +
{% endfor %}
diff --git a/src/openforms/forms/tests/admin/test_tasks.py b/src/openforms/forms/tests/admin/test_tasks.py index af32d29944..57838d9933 100644 --- a/src/openforms/forms/tests/admin/test_tasks.py +++ b/src/openforms/forms/tests/admin/test_tasks.py @@ -12,13 +12,12 @@ from rest_framework.exceptions import ValidationError from openforms.accounts.tests.factories import SuperUserFactory +from openforms.forms.admin.tasks import process_forms_export, process_forms_import +from openforms.forms.models.form import Form, FormsExport +from openforms.forms.tests.factories import FormFactory from openforms.logging.models import TimelineLogProxy from openforms.utils.urls import build_absolute_uri -from ...admin.tasks import process_forms_export, process_forms_import -from ...models.form import Form, FormsExport -from ..factories import FormFactory - @temp_private_root() @override_settings(LANGUAGE_CODE="en") @@ -100,7 +99,7 @@ def _copy_file_to_imports_tempdir(self): def test_import_forms(self): imported_file_path = self._copy_file_to_imports_tempdir() assert private_media_storage.exists(imported_file_path) - process_forms_import(str(imported_file_path), self.user.id) + process_forms_import(str(imported_file_path), self.user.id, import_options={}) self.assertEqual(4, Form.objects.count()) @@ -113,7 +112,7 @@ def test_import_forms(self): ) def test_import_form_failure(self, m_import_form): imported_file_path = self._copy_file_to_imports_tempdir() - process_forms_import(str(imported_file_path), self.user.id) + process_forms_import(str(imported_file_path), self.user.id, import_options={}) self.assertEqual(2, Form.objects.count()) diff --git a/src/openforms/forms/tests/import_export/test_matchers.py b/src/openforms/forms/tests/import_export/test_matchers.py new file mode 100644 index 0000000000..cb09f164ea --- /dev/null +++ b/src/openforms/forms/tests/import_export/test_matchers.py @@ -0,0 +1,156 @@ +from django.test import TestCase + +from openforms.forms.import_export.matchers.form_definition import FormDefinitionMatcher +from openforms.forms.tests.factories import FormDefinitionFactory + + +class FormDefinitionMatcherTests(TestCase): + def test_find_similar_form_definition(self): + fd = FormDefinitionFactory.create( + uuid="0a1c2ac4-b5fb-429b-899d-2f4813d53bfa", + configuration={ + "components": [ + { + "id": "e0bdve", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + }, + ] + }, + is_reusable=True, + ) + + # Search for a form definition with a similar component, but different id + matcher = FormDefinitionMatcher() + found_instance = matcher.find( + { + "components": [ + { + "id": "123abc", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + }, + ] + } + ) + + self.assertIsNotNone(found_instance) + self.assertEqual(fd.pk, found_instance.pk) + + def test_find_similar_form_definition_with_nested_components(self): + fd = FormDefinitionFactory.create( + uuid="0a1c2ac4-b5fb-429b-899d-2f4813d53bfa", + configuration={ + "components": [ + { + "id": "c2sac1", + "type": "editgrid", + "key": "editgrid", + "label": "Editgrid", + "groupLabel": "item", + "components": [ + { + "id": "e0bdve", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + } + ], + }, + ] + }, + is_reusable=True, + ) + + # Search for a form definition with similar components, but different id's + matcher = FormDefinitionMatcher() + found_instance = matcher.find( + { + "components": [ + { + "id": "456def", + "type": "editgrid", + "key": "editgrid", + "label": "Editgrid", + "groupLabel": "item", + "components": [ + { + "id": "123abc", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + } + ], + }, + ] + } + ) + + self.assertIsNotNone(found_instance) + self.assertEqual(fd.pk, found_instance.pk) + + def test_return_none_when_no_similar_form_definition_is_found(self): + FormDefinitionFactory.create( + uuid="0a1c2ac4-b5fb-429b-899d-2f4813d53bfa", + configuration={ + "components": [ + { + "id": "e0bdve", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + } + ] + }, + is_reusable=True, + ) + + matcher = FormDefinitionMatcher() + found_instance = matcher.find( + { + "components": [ + { + "id": "123abc", + "key": "textfield", + "type": "textfield", + "label": "Textfield met een andere titel", + }, + ] + } + ) + + self.assertIsNone(found_instance) + + def test_return_none_when_similar_form_definition_is_not_reusable(self): + FormDefinitionFactory.create( + uuid="0a1c2ac4-b5fb-429b-899d-2f4813d53bfa", + configuration={ + "components": [ + { + "id": "e0bdve", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + } + ] + }, + is_reusable=False, + ) + + matcher = FormDefinitionMatcher() + found_instance = matcher.find( + { + "components": [ + { + "id": "123abc", + "key": "textfield", + "type": "textfield", + "label": "Textfield", + }, + ] + } + ) + + self.assertIsNone(found_instance) diff --git a/src/openforms/forms/tests/import_export/test_resources.py b/src/openforms/forms/tests/import_export/test_resources.py index 60296ae6e5..fed9d72b13 100644 --- a/src/openforms/forms/tests/import_export/test_resources.py +++ b/src/openforms/forms/tests/import_export/test_resources.py @@ -1,6 +1,10 @@ from django.test import TestCase +import tablib + +from openforms.authentication.contrib.yivi_oidc.models import AttributeGroup from openforms.authentication.tests.factories import AttributeGroupFactory +from openforms.config.models import MapTileLayer, MapWMSTileLayer from openforms.config.tests.factories import ( MapTileLayerFactory, MapWMSTileLayerFactory, @@ -12,10 +16,11 @@ YiviAttributeGroupResource, ) from openforms.forms.tests.factories import FormFactory +from openforms.products.models import Product from openforms.products.tests.factories import ProductFactory -class ProductResourceTests(TestCase): +class ProductResourceExportTests(TestCase): def test_export_for_form(self): product = ProductFactory.create() form = FormFactory.create(product=product) @@ -39,7 +44,108 @@ def test_export_for_form_without_product(self): self.assertEqual(len(dataset), 0) -class WMSTileLayerResourceTests(TestCase): +class ProductResourceImportTests(TestCase): + def test_import_product_with_same_identifier(self): + product = ProductFactory.create( + uuid="20523058-a69b-4d8e-b58d-b6a9291e3b66", + name="product", + price=10, + information="product information", + ) + + # Create a dataset with the same UUID, but with different product information + dataset = tablib.Dataset( + *[ + ( + "20523058-a69b-4d8e-b58d-b6a9291e3b66", + "product 2", + "15,00", + "different product information", + ) + ], + headers=["uuid", "name", "price", "information"], + ) + + results = ProductResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the product with the same UUID + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, product) + + # No new products have been created + self.assertEqual(Product.objects.count(), 1) + + def test_import_product_with_different_identifier_but_same_configuration(self): + product = ProductFactory.create( + uuid="20523058-a69b-4d8e-b58d-b6a9291e3b66", + name="product", + price=10, + information="product information", + ) + + # Create a dataset with a different UUID, but with the same product information + dataset = tablib.Dataset( + *[ + ( + "a29744bd-f30b-46ed-9677-30f55d767f14", + "product", + "10,00", + "product information", + ) + ], + headers=["uuid", "name", "price", "information"], + ) + + results = ProductResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the product with the same product information + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, product) + + # No new products have been created + self.assertEqual(Product.objects.count(), 1) + + def test_import_product_with_different_identifier_and_configuration(self): + product = ProductFactory.create( + uuid="20523058-a69b-4d8e-b58d-b6a9291e3b66", + name="product", + price=10, + information="product information", + ) + + # Create a dataset with a different UUID and product information + dataset = tablib.Dataset( + *[ + ( + "a29744bd-f30b-46ed-9677-30f55d767f14", + "product 2", + "15,00", + "different product information", + ) + ], + headers=["uuid", "name", "price", "information"], + ) + + results = ProductResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result does not represent any existing product + self.assertEqual(result.is_new(), True) + self.assertNotEqual(result.instance, product) + + # A new product has been created + self.assertEqual(Product.objects.count(), 2) + + +class WMSTileLayerResourceExportTests(TestCase): def test_export_for_form(self): wms_tile_layer1 = MapWMSTileLayerFactory.create() wms_tile_layer2 = MapWMSTileLayerFactory.create() @@ -150,7 +256,108 @@ def test_export_for_form_without_overlays(self): self.assertEqual(len(dataset), 0) -class WMTSTileLayerResourceTests(TestCase): +class WMSTileLayerResourceImportTests(TestCase): + def setUp(self): + # Make sure we start with a clean slate + MapWMSTileLayer.objects.all().delete() + + def test_import_wms_tile_layer_with_same_identifier(self): + tile_layer = MapWMSTileLayerFactory.create( + uuid="20523058-a69b-4d8e-b58d-b6a9291e3b66", + name="tile layer", + url="http://example.com", + ) + + # Create a dataset with the same UUID, but with different tile layer information + dataset = tablib.Dataset( + *[ + ( + "20523058-a69b-4d8e-b58d-b6a9291e3b66", + "different layer", + "http://different.com", + ) + ], + headers=["uuid", "name", "url"], + ) + + results = WMSTileLayerResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the tile layer with the same UUID + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, tile_layer) + + # No new tile layers have been created + self.assertEqual(MapWMSTileLayer.objects.count(), 1) + + def test_import_wms_tile_layer_with_different_identifier_but_same_configuration( + self, + ): + tile_layer = MapWMSTileLayerFactory.create( + uuid="20523058-a69b-4d8e-b58d-b6a9291e3b66", + name="tile layer", + url="http://example.com", + ) + + # Create a dataset with a different UUID, but with the same tile layer information + dataset = tablib.Dataset( + *[ + ( + "670596d5-8ce3-4b97-b4a4-bd1c05516c4a", + "tile layer", + "http://example.com", + ) + ], + headers=["uuid", "name", "url"], + ) + + results = WMSTileLayerResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the tile layer with the same information + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, tile_layer) + + # No new tile layers have been created + self.assertEqual(MapWMSTileLayer.objects.count(), 1) + + def test_import_wms_tile_layer_with_different_identifier_and_configuration(self): + tile_layer = MapWMSTileLayerFactory.create( + uuid="20523058-a69b-4d8e-b58d-b6a9291e3b66", + name="tile layer", + url="http://example.com", + ) + + # Create a dataset with a different UUID and tile layer information + dataset = tablib.Dataset( + *[ + ( + "670596d5-8ce3-4b97-b4a4-bd1c05516c4a", + "different layer", + "http://different.com", + ) + ], + headers=["uuid", "name", "url"], + ) + + results = WMSTileLayerResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result does not represent any existing tile layer + self.assertEqual(result.is_new(), True) + self.assertNotEqual(result.instance, tile_layer) + + # A new tile layer has been created + self.assertEqual(MapWMSTileLayer.objects.count(), 2) + + +class WMTSTileLayerExportResourceTests(TestCase): def test_export_for_form(self): wmts_tile_layer = MapTileLayerFactory.create() form = FormFactory.create( @@ -231,7 +438,108 @@ def test_export_for_form_without_tile_layer_identifier(self): self.assertEqual(len(dataset), 0) -class YiviAttributeGroupResourceTests(TestCase): +class WMSTTileLayerResourceImportTests(TestCase): + def setUp(self): + # Make sure we start with a clean slate + MapTileLayer.objects.all().delete() + + def test_import_wmts_tile_layer_with_same_identifier(self): + background_layer = MapTileLayerFactory.create( + identifier="background-layer", + label="background layer", + url="http://example.com", + ) + + # Create a dataset with the same identifier, but with different tile layer information + dataset = tablib.Dataset( + *[ + ( + "background-layer", + "different background layer", + "http://different.com", + ) + ], + headers=["identifier", "label", "url"], + ) + + results = WMTSTileLayerResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the tile layer with the same identifier + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, background_layer) + + # No new tile layers have been created + self.assertEqual(MapTileLayer.objects.count(), 1) + + def test_import_wmts_tile_layer_with_different_identifier_but_same_configuration( + self, + ): + background_layer = MapTileLayerFactory.create( + identifier="background-layer", + label="background layer", + url="http://example.com", + ) + + # Create a dataset with a different identifier, but with the same tile layer information + dataset = tablib.Dataset( + *[ + ( + "different-background-layer", + "background layer", + "http://example.com", + ) + ], + headers=["identifier", "label", "url"], + ) + + results = WMTSTileLayerResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the tile layer with the same information + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, background_layer) + + # No new tile layers have been created + self.assertEqual(MapTileLayer.objects.count(), 1) + + def test_import_wmts_tile_layer_with_different_identifier_and_configuration(self): + background_layer = MapTileLayerFactory.create( + identifier="background-layer", + label="background layer", + url="http://example.com", + ) + + # Create a dataset with a different identifier and tile layer information + dataset = tablib.Dataset( + *[ + ( + "different-background-layer", + "different background layer", + "http://different.com", + ) + ], + headers=["identifier", "label", "url"], + ) + + results = WMTSTileLayerResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result does not represent any existing tile layer + self.assertEqual(result.is_new(), True) + self.assertNotEqual(result.instance, background_layer) + + # A new tile layer has been created + self.assertEqual(MapTileLayer.objects.count(), 2) + + +class YiviAttributeGroupResourceExportTests(TestCase): def test_export_for_form(self): yivi_attribute_group = AttributeGroupFactory.create( attributes=["first_name", "last_name"] @@ -267,3 +575,112 @@ def test_export_for_form_without_yivi_auth_backend(self): dataset = YiviAttributeGroupResource().export_for_form(form).dict self.assertEqual(len(dataset), 0) + + +class YiviAttributeGroupResourceImportTests(TestCase): + def setUp(self): + # Make sure we start with a clean slate + AttributeGroup.objects.all().delete() + + def test_import_yivi_attribute_group_with_same_identifier(self): + attribute_group = AttributeGroupFactory.create( + uuid="670596d5-8ce3-4b97-b4a4-bd1c05516c4a", + name="base attributes", + description="some basic yivi attributes", + attributes=["firstname", "lastname"], + ) + + # Create a dataset with the same identifier, but with different attribute group information + dataset = tablib.Dataset( + *[ + ( + "670596d5-8ce3-4b97-b4a4-bd1c05516c4a", + "additional attributes", + "", + "DOB", + ) + ], + headers=["uuid", "name", "description", "attributes"], + ) + + results = YiviAttributeGroupResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the attribute group with the same identifier + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, attribute_group) + + # No new attribute groups have been created + self.assertEqual(AttributeGroup.objects.count(), 1) + + def test_import_yivi_attribute_group_with_different_identifier_but_same_configuration( + self, + ): + attribute_group = AttributeGroupFactory.create( + uuid="670596d5-8ce3-4b97-b4a4-bd1c05516c4a", + name="base attributes", + description="some basic yivi attributes", + attributes=["firstname", "lastname"], + ) + + # Create a dataset with a different identifier, but with the same attribute group information + dataset = tablib.Dataset( + *[ + ( + "79623448-fa11-4d86-91c5-0e2a5cd617ac", + "base attributes", + "some basic yivi attributes", + "firstname,lastname", + ) + ], + headers=["uuid", "name", "description", "attributes"], + ) + + results = YiviAttributeGroupResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should represent the attribute group with the same information + self.assertEqual(result.is_skip(), True) + self.assertEqual(result.instance, attribute_group) + + # No new attribute groups have been created + self.assertEqual(AttributeGroup.objects.count(), 1) + + def test_import_yivi_attribute_group_with_different_identifier_and_configuration( + self, + ): + attribute_group = AttributeGroupFactory.create( + uuid="670596d5-8ce3-4b97-b4a4-bd1c05516c4a", + name="base attributes", + description="some basic yivi attributes", + attributes=["firstname", "lastname"], + ) + + # Create a dataset with a different identifier and attribute group information + dataset = tablib.Dataset( + *[ + ( + "79623448-fa11-4d86-91c5-0e2a5cd617ac", + "additional attributes", + "", + "DOB", + ) + ], + headers=["uuid", "name", "description", "attributes"], + ) + + results = YiviAttributeGroupResource().import_data(dataset) + + self.assertEqual(len(results.rows), 1) + result = results.rows[0] + + # The result should not represent any existing attribute group + self.assertEqual(result.is_new(), True) + self.assertNotEqual(result.instance, attribute_group) + + # A new attribute group has been created + self.assertEqual(AttributeGroup.objects.count(), 2) diff --git a/src/openforms/forms/tests/test_api_import_export.py b/src/openforms/forms/tests/test_api_import_export.py index ba0721c8d2..073abbf12a 100644 --- a/src/openforms/forms/tests/test_api_import_export.py +++ b/src/openforms/forms/tests/test_api_import_export.py @@ -14,7 +14,9 @@ from openforms.accounts.tests.factories import TokenFactory, UserFactory from openforms.appointments.models import AppointmentsConfig from openforms.authentication.constants import AuthAttribute +from openforms.authentication.contrib.digid.constants import DIGID_DEFAULT_LOA from openforms.authentication.tests.factories import AttributeGroupFactory +from openforms.config.models import GlobalConfiguration, MapTileLayer, MapWMSTileLayer from openforms.config.tests.factories import ( MapTileLayerFactory, MapWMSTileLayerFactory, @@ -35,7 +37,7 @@ from ...emails.tests.factories import ConfirmationEmailTemplateFactory from ..constants import FormTypeChoices -from ..models import Form, FormDefinition, FormStep +from ..models import Form, FormDefinition, FormStep, FormVariable from .factories import ( CategoryFactory, FormDefinitionFactory, @@ -1083,7 +1085,10 @@ def test_form_import_form_slug_already_exists(self, _mock): url = reverse("api:forms-import") response = self.client.post( url, - {"file": f}, + { + "file": f, + "reuse_form_definitions": True, + }, format="multipart", HTTP_AUTHORIZATION=f"Token {self.token.key}", HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", @@ -1153,3 +1158,831 @@ def test_form_import_staff_required(self): ) self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN) + + def test_form_import_removes_all_unknown_links_from_email_templates(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + config = GlobalConfiguration.get_solo() + + # Start with google domain in allowlist, so we can create the initial form + config.email_template_netloc_allowlist = ["https://google.com", "allowed.com"] # pyright: ignore[reportAttributeAccessIssue] + config.save() + + form = FormFactory.create( + registration_backend="email", + registration_backend_options={ + "to_emails": ["some@email.com"], + "email_content_template_html": "

test https://google.com Google https://allowed.com test

", + "email_content_template_text": "test https://google.com https://allowed.com test", + }, + ) + ConfirmationEmailTemplateFactory( + form=form, + subject="Test", + content="

email content https://google.com Google https://allowed.com

{% appointment_information %}

{% payment_information %}

", + cosign_subject="Cosign test", + cosign_content="

cosign email content https://google.com Google https://allowed.com

{% payment_information %}

{% cosign_information %}

", + ) + + # Export the form with all the main form configuration + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, + format="json", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + data={ + # Keep sensitive data to keep the email registration config complete + "remove_sensitive_content": False, + "form_configuration": [ + FormConfigurationOptions.registration_backends, + ], + }, + ) + + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Remove google domain from allowlist + config.email_template_netloc_allowlist = ["allowed.com"] # pyright: ignore[reportAttributeAccessIssue] + config.save() + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "form_configuration": [ + FormConfigurationOptions.registration_backends, + ], + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + self.assertEqual(len(imported_form.registration_backends.all()), 1) + self.assertEqual( + imported_form.registration_backends.first().options, + { + "to_emails": ["some@email.com"], + "attach_files_to_email": None, + "email_content_template_html": "

test Google https://allowed.com test

", + "email_content_template_text": "test https://allowed.com test", + }, + ) + + # The confirmation and cosign email templates should not contain the google domain + self.assertEqual( + imported_form.confirmation_email_template.content, + "

email content Google https://allowed.com

{% appointment_information %}

{% payment_information %}

", + ) + self.assertEqual( + imported_form.confirmation_email_template.cosign_content, + "

cosign email content Google https://allowed.com

{% payment_information %}

{% cosign_information %}

", + ) + + def test_form_import_include_all_form_configuration(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + product = ProductFactory.create() + merchant = WorldlineMerchantFactory.create() + objects_api_group = ObjectsAPIGroupConfigFactory.create( + identifier="test-objects-api-group" + ) + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + authentication_backend="digid", + authentication_backend_options={ + "loa": DIGID_DEFAULT_LOA, + }, + payment_backend="worldline", + payment_backend_options={"merchant": merchant.pspid}, + registration_backend="email", + registration_backend_options={ + "to_emails": ["abc@xyz.com"], + }, + formstep__form_definition__configuration={ + "components": [ + { + "key": "textfield", + "type": "textfield", + "label": "Textfield", + "prefill": { + "plugin": "demo", + "attribute": "random_number", + "identifier_role": IdentifierRoles.authorizee, + }, + }, + ], + }, + ) + FormVariableFactory.create( + form=form, + key="variable_with_demo_prefill", + user_defined=True, + prefill_plugin="demo", + prefill_attribute="random_string", + prefill_identifier_role=IdentifierRoles.authorizee, + ) + FormVariableFactory.create( + form=form, + key="variable_with_objects_api_prefill", + user_defined=True, + prefill_plugin="objects_api", + prefill_options={ + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) + + # Export the form with all the main form configuration + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, + format="json", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + data={ + # Keep sensitive data to keep the email registration config complete + "remove_sensitive_content": False, + "form_configuration": [ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + }, + ) + + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "form_configuration": [ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + # Registration backend should be added + self.assertEqual(imported_form.registration_backends.count(), 1) + registration_backend = imported_form.registration_backends.first() + self.assertEqual(registration_backend.backend, "email") + self.assertEqual(registration_backend.options["to_emails"], ["abc@xyz.com"]) + + # The auth backend should be added + self.assertEqual(imported_form.auth_backends.count(), 1) + auth_backend = imported_form.auth_backends.first() + self.assertEqual(auth_backend.backend, "digid") + self.assertEqual(auth_backend.options, {"loa": DIGID_DEFAULT_LOA}) + + # Payment backend should be added + self.assertEqual(imported_form.payment_backend, "worldline") + self.assertEqual( + imported_form.payment_backend_options, + {"descriptor_template": "", "merchant": merchant.pspid, "variant": ""}, + ) + + imported_variables = FormVariable.objects.filter( + form=imported_form, + source=FormVariableSources.user_defined, + ) + self.assertEqual(len(imported_variables), 2) + self.assertEqual(imported_form.formstep_set.count(), 1) + imported_form_definition = imported_form.formstep_set.first().form_definition + imported_form_variable1 = imported_variables[0] + imported_form_variable2 = imported_variables[1] + + # Both variables should have their prefill data + self.assertEqual(imported_form_variable1.prefill_plugin, "demo") + self.assertEqual(imported_form_variable1.prefill_attribute, "random_string") + self.assertEqual( + imported_form_variable1.prefill_identifier_role, IdentifierRoles.authorizee + ) + + self.assertEqual(imported_form_variable2.prefill_plugin, "objects_api") + self.assertEqual( + imported_form_variable2.prefill_options, + { + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) + + # The component prefill data should be kept + self.assertEqual(len(imported_form_definition.configuration["components"]), 1) + component_definition = imported_form_definition.configuration["components"][0] + self.assertEqual(component_definition["prefill"]["plugin"], "demo") + self.assertEqual(component_definition["prefill"]["attribute"], "random_number") + self.assertEqual( + component_definition["prefill"]["identifier_role"], + IdentifierRoles.authorizee, + ) + + def test_form_import_exclude_all_form_configuration(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + product = ProductFactory.create() + merchant = WorldlineMerchantFactory.create() + objects_api_group = ObjectsAPIGroupConfigFactory.create( + identifier="test-objects-api-group" + ) + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + authentication_backend="digid", + authentication_backend_options={ + "loa": DIGID_DEFAULT_LOA, + }, + payment_backend="worldline", + payment_backend_options={"merchant": merchant.pspid}, + registration_backend="email", + registration_backend_options={ + "to_emails": ["abc@xyz.com"], + }, + formstep__form_definition__configuration={ + "components": [ + { + "key": "textfield", + "type": "textfield", + "label": "Textfield", + "prefill": { + "plugin": "demo", + "attribute": "random_number", + "identifier_role": IdentifierRoles.authorizee, + }, + }, + ], + }, + ) + FormVariableFactory.create( + form=form, + key="variable_with_demo_prefill", + user_defined=True, + prefill_plugin="demo", + prefill_attribute="random_string", + prefill_identifier_role=IdentifierRoles.authorizee, + ) + FormVariableFactory.create( + form=form, + key="variable_with_objects_api_prefill", + user_defined=True, + prefill_plugin="objects_api", + prefill_options={ + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) + + # Export the form with all the main form configuration + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, + format="json", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + data={ + # Keep sensitive data to keep the email registration config complete + "remove_sensitive_content": False, + "form_configuration": [ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + }, + ) + + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "form_configuration": [], + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + # Registration backend should not be added + self.assertEqual(imported_form.registration_backends.count(), 0) + + # Auth backend should not be added + self.assertEqual(imported_form.auth_backends.count(), 0) + + # Payment backend should not be added + self.assertEqual(imported_form.payment_backend, "") + self.assertEqual(imported_form.payment_backend_options, {}) + + imported_variables = FormVariable.objects.filter( + form=imported_form, + source=FormVariableSources.user_defined, + ) + self.assertEqual(len(imported_variables), 2) + self.assertEqual(imported_form.formstep_set.count(), 1) + imported_form_definition = imported_form.formstep_set.first().form_definition + variable1 = imported_variables[0] + variable2 = imported_variables[1] + + # Neither variable should have any prefill data + self.assertEqual(variable1.prefill_plugin, "") + self.assertEqual(variable1.prefill_attribute, "") + self.assertEqual(variable1.prefill_identifier_role, IdentifierRoles.main) + + self.assertEqual(variable2.prefill_plugin, "") + self.assertEqual(variable2.prefill_options, {}) + + # The component prefill data should be removed + self.assertEqual(len(imported_form_definition.configuration["components"]), 1) + component_definition = imported_form_definition.configuration["components"][0] + self.assertEqual(component_definition["prefill"]["plugin"], "") + self.assertEqual(component_definition["prefill"]["attribute"], "") + self.assertEqual( + component_definition["prefill"]["identifier_role"], IdentifierRoles.main + ) + + def test_form_import_include_all_additional_form_configuration(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + # Every OF instance has various default WMTS- and WMS- tile layers. + # For more accurate and easier testing, we should start with zero. + MapTileLayer.objects.all().delete() + MapWMSTileLayer.objects.all().delete() + + product = ProductFactory.create() + wmts_tile_layer = MapTileLayerFactory.create() + wms_tile_layer = MapWMSTileLayerFactory.create() + yivi_attribute_group = AttributeGroupFactory.create( + attributes=["first_name", "last_name"] + ) + + # Define form with all additional form configuration + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + authentication_backend="yivi_oidc", + authentication_backend__options={ + "authentication_options": [AuthAttribute.bsn], + "additional_attributes_groups": [ + yivi_attribute_group.uuid, + ], + }, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmts_tile_layer.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wms_tile_layer.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + }, + ], + } + ], + }, + ) + + # Export form + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, + format="json", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + data={ + "form_configuration": [ + FormConfigurationOptions.auth_backends, + ], + "additional_form_configuration": [ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + }, + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "form_configuration": [ + FormConfigurationOptions.auth_backends, + ], + "additional_form_configuration": [ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + # The correct product should be used + self.assertEqual(imported_form.product, product) + + # The correct auth backend + configuration should be used + self.assertEqual(imported_form.auth_backends.count(), 1) + auth_backend = imported_form.auth_backends.first() + self.assertEqual(auth_backend.backend, "yivi_oidc") + self.assertEqual( + auth_backend.options["additional_attributes_groups"], + [str(yivi_attribute_group.uuid)], + ) + + # Make sure the map tile layers are imported correctly + self.assertEqual(imported_form.formstep_set.count(), 1) + form_definition = imported_form.formstep_set.first().form_definition + + self.assertEqual(len(form_definition.configuration["components"]), 1) + component_definition = form_definition.configuration["components"][0] + + self.assertEqual( + component_definition["tileLayerIdentifier"], wmts_tile_layer.identifier + ) + self.assertEqual(len(component_definition["overlays"]), 1) + self.assertEqual( + component_definition["overlays"][0]["uuid"], str(wms_tile_layer.uuid) + ) + + def test_form_import_exclude_all_additional_form_configuration(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + # Every OF instance has various default WMTS- and WMS- tile layers. + # For more accurate and easier testing, we should start with zero. + MapTileLayer.objects.all().delete() + MapWMSTileLayer.objects.all().delete() + + product = ProductFactory.create() + wmts_tile_layer = MapTileLayerFactory.create() + wms_tile_layer = MapWMSTileLayerFactory.create() + yivi_attribute_group = AttributeGroupFactory.create( + attributes=["first_name", "last_name"] + ) + + # Define form with all additional form configuration + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + authentication_backend="yivi_oidc", + authentication_backend__options={ + "authentication_options": [AuthAttribute.bsn], + "additional_attributes_groups": [ + yivi_attribute_group.uuid, + ], + }, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmts_tile_layer.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wms_tile_layer.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + }, + ], + } + ], + }, + ) + + # Export form + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, + format="json", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + data={ + "form_configuration": [ + FormConfigurationOptions.auth_backends, + ], + "additional_form_configuration": [ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + }, + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "form_configuration": [ + FormConfigurationOptions.auth_backends, + ], + "additional_form_configuration": [], + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + # Product should not be imported + self.assertIsNone(imported_form.product) + + # The yivi attirbute groups should not be imported + self.assertEqual(imported_form.auth_backends.count(), 1) + auth_backend = imported_form.auth_backends.first() + self.assertEqual(auth_backend.backend, "yivi_oidc") + self.assertEqual(auth_backend.options["additional_attributes_groups"], []) + + # Make sure the map tile layers are imported correctly + self.assertEqual(imported_form.formstep_set.count(), 1) + form_definition = imported_form.formstep_set.first().form_definition + + self.assertEqual(len(form_definition.configuration["components"]), 1) + component_definition = form_definition.configuration["components"][0] + + self.assertEqual(component_definition["tileLayerIdentifier"], "") + self.assertEqual(len(component_definition["overlays"]), 1) + self.assertEqual(component_definition["overlays"][0]["uuid"], "") + + def test_form_import_with_theme_and_category(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + theme = ThemeFactory.create() + category = CategoryFactory.create() + form = FormFactory.create( + generate_minimal_setup=True, + theme=theme, + category=category, + ) + + # Export form + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, format="json", HTTP_AUTHORIZATION=f"Token {self.token.key}" + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "theme": theme.uuid, + "category": category.uuid, + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + self.assertEqual(imported_form.theme, theme) + self.assertEqual(imported_form.category, category) + + def test_form_import_without_theme_and_category(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + theme = ThemeFactory.create() + category = CategoryFactory.create() + form = FormFactory.create( + generate_minimal_setup=True, + theme=theme, + category=category, + ) + + # Export form + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, format="json", HTTP_AUTHORIZATION=f"Token {self.token.key}" + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + {"file": f}, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + self.assertIsNone(imported_form.theme) + self.assertIsNone(imported_form.category) + + def test_form_import_create_new_form_definitions(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + form = FormFactory.create( + generate_minimal_setup=True, + formstep__form_definition__configuration={ + "components": [ + {"label": "Textfield", "key": "textfield", "type": "textfield"} + ], + }, + formstep__form_definition__is_reusable=True, + ) + + # We start with one form definition + self.assertEqual(FormDefinition.objects.count(), 1) + + # Export form + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, format="json", HTTP_AUTHORIZATION=f"Token {self.token.key}" + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "reuse_form_definitions": False, + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + # A new form definition should have been created + self.assertEqual(FormDefinition.objects.count(), 2) + new_form_definition = FormDefinition.objects.last() + + self.assertEqual(len(new_form_definition.used_in), 1) + self.assertIn(imported_form, new_form_definition.used_in) + + def test_form_import_reuse_form_definitions(self): + self.user.user_permissions.add(Permission.objects.get(codename="change_form")) + self.user.is_staff = True + self.user.save() + + form = FormFactory.create( + generate_minimal_setup=True, + formstep__form_definition__configuration={ + "components": [ + {"label": "Textfield", "key": "textfield", "type": "textfield"} + ], + }, + formstep__form_definition__is_reusable=True, + ) + form_definition = form.formstep_set.get().form_definition + + # We start with one form definition + self.assertEqual(FormDefinition.objects.count(), 1) + + # Export form + url = reverse("api:form-export", args=(form.uuid,)) + response = self.client.post( + url, format="json", HTTP_AUTHORIZATION=f"Token {self.token.key}" + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + # Import form + f = SimpleUploadedFile( + "file.zip", response.content, content_type="application/zip" + ) + url = reverse("api:forms-import") + response = self.client.post( + url, + { + "file": f, + "reuse_form_definitions": True, + }, + format="multipart", + HTTP_AUTHORIZATION=f"Token {self.token.key}", + HTTP_CONTENT_DISPOSITION="attachment;filename=file.zip", + ) + + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + imported_form = Form.objects.last() + + # A new form definition should have been created + self.assertEqual(FormDefinition.objects.count(), 1) + self.assertEqual(Form.objects.count(), 2) + + self.assertEqual(len(form_definition.used_in), 2) + self.assertIn(imported_form, form_definition.used_in) diff --git a/src/openforms/forms/tests/test_import_export.py b/src/openforms/forms/tests/test_import_export.py index d693b67c20..1e49f385e3 100644 --- a/src/openforms/forms/tests/test_import_export.py +++ b/src/openforms/forms/tests/test_import_export.py @@ -5,19 +5,28 @@ from shutil import rmtree from textwrap import dedent from unittest.mock import patch -from uuid import UUID +from uuid import UUID, uuid4 from django.test import TestCase, override_settings, tag from django.utils import translation from digid_eherkenning.choices import AssuranceLevels, DigiDAssuranceLevels from freezegun import freeze_time +from privates.storages import private_media_storage +from privates.test import temp_private_root from rest_framework.exceptions import ValidationError +from openforms.accounts.tests.factories import SuperUserFactory from openforms.authentication.constants import AuthAttribute +from openforms.authentication.contrib.yivi_oidc.models import AttributeGroup from openforms.authentication.tests.factories import AttributeGroupFactory from openforms.config.constants import UploadFileType -from openforms.config.models import GlobalConfiguration +from openforms.config.models import ( + GlobalConfiguration, + MapTileLayer, + MapWMSTileLayer, + Theme, +) from openforms.config.tests.factories import ( MapTileLayerFactory, MapWMSTileLayerFactory, @@ -26,17 +35,21 @@ from openforms.contrib.objects_api.tests.factories import ObjectsAPIGroupConfigFactory from openforms.emails.models import ConfirmationEmailTemplate from openforms.emails.tests.factories import ConfirmationEmailTemplateFactory +from openforms.forms.admin.tasks import process_forms_export, process_forms_import from openforms.forms.import_export.constants import EXPORT_META_KEY from openforms.forms.import_export.export_form import export_form, form_to_json +from openforms.forms.import_export.import_form import import_form from openforms.forms.import_export.typing import ( AdditionalFormConfigurationOptions, FormConfigurationOptions, FormExportOptions, + FormImportOptions, ) from openforms.payments.contrib.worldline.tests.factories import ( WorldlineMerchantFactory, ) from openforms.prefill.constants import IdentifierRoles +from openforms.products.models import Product from openforms.products.tests.factories import ProductFactory from openforms.registrations.contrib.objects_api.config import ( ObjectsAPIOptionsSerializer, @@ -72,15 +85,16 @@ from ..disable_next_import_conversion import add_form_step_uuid_to_disable_next_actions from ..models import ( + Category, Form, FormAuthenticationBackend, FormDefinition, FormLogic, FormRegistrationBackend, + FormsExport, FormStep, FormVariable, ) -from ..utils import import_form from .factories import ( CategoryFactory, FormDefinitionFactory, @@ -1239,7 +1253,17 @@ def test_import(self): form.slug = "modified" form.save() - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ] + ), + ) forms = Form.objects.all() imported_form = forms.last() @@ -1317,7 +1341,9 @@ def test_import(self): @tag("gh-3379") def test_import_2_1_3_export_does_not_fail(self): - import_form(import_file=PATH / "data/smol.zip") + import_form( + import_file=PATH / "data/smol.zip", import_options=FormImportOptions() + ) self.assertTrue(Form.objects.filter(name="Smol").exists()) def test_import_no_backends(self): @@ -1338,7 +1364,7 @@ def test_import_no_backends(self): form.slug = "modified" form.save() - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) def test_import_form_slug_already_exists(self): product = ProductFactory.create() @@ -1362,7 +1388,10 @@ def test_import_form_slug_already_exists(self): form.pk, archive_name=self.filepath, export_options=FormExportOptions() ) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=True), + ) imported_form = Form.objects.last() imported_form_step = imported_form.formstep_set.get() @@ -1408,7 +1437,10 @@ def test_import_form_definition_uuid_already_exists_configuration_duplicate(self form.slug = "modified" form.save() - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=True), + ) forms = Form.objects.all() imported_form = forms.last() @@ -1501,7 +1533,10 @@ def test_import_form_definition_uuid_already_exists_configuration_different(self form_definition.configuration = {"foo": ["bar"]} form_definition.save() - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=True), + ) forms = Form.objects.all() imported_form = forms.last() @@ -1579,7 +1614,7 @@ def test_import_reusable_form_definition_uuid_already_exists(self): form.pk, archive_name=self.filepath, export_options=FormExportOptions() ) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) form_definitions = FormDefinition.objects.all() fd2 = form_definitions.last() @@ -1613,7 +1648,7 @@ def test_import_form_with_category(self): form.delete() category.delete() - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) form = Form.objects.get() self.assertIsNone(form.category) @@ -1700,7 +1735,7 @@ def test_roundtrip_a_translated_form(self): self.assertEqual(Form.objects.count(), 0) self.assertEqual(FormDefinition.objects.count(), 0) self.assertEqual(FormStep.objects.count(), 0) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.get() imported_form_step = imported_form.formstep_set.select_related().get() @@ -1910,7 +1945,7 @@ def test_import_form_with_disable_step_logic(self): for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) self.assertTrue(Form.objects.filter(slug="auth-plugins").exists()) @@ -1968,8 +2003,11 @@ def add_foo(component): ) converters = {"textfield": {"add_foo": add_foo}} - with patch("openforms.forms.utils.CONVERTERS", new=converters): - import_form(import_file=self.filepath) + with patch( + "openforms.forms.import_export.serializers.form_definition.CONVERTERS", + new=converters, + ): + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.exclude(pk=form.pk).get() fd = imported_form.formstep_set.get().form_definition @@ -1986,7 +2024,7 @@ def test_rountrip_form_with_theme_override(self): ) # run the import again - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.exclude(pk=form.pk).get() self.assertIsNone(imported_form.theme) @@ -2055,7 +2093,7 @@ def test_import_form_with_old_service_fetch_config(self): for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) rule = FormLogic.objects.get(form__slug="old-service-fetch-config", order=0) self.assertEqual(rule.actions[0]["action"]["value"], "") @@ -2215,7 +2253,7 @@ def test_import_form_with_old_simple_conditionals_with_numbers(self): for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) form_definition = FormDefinition.objects.get(slug="test-definition") fixed_components = form_definition.configuration["components"] @@ -2259,7 +2297,7 @@ def test_import_applies_converters_map_component_interactions(self): export_form( form.pk, archive_name=self.filepath, export_options=FormExportOptions() ) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.exclude(pk=form.pk).get() fd = imported_form.formstep_set.get().form_definition @@ -2312,7 +2350,7 @@ def test_import_form_with_old_authentication_backends( for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.get(slug="test-form") authentication_backends = imported_form.auth_backends.all() @@ -2352,7 +2390,7 @@ def test_import_form_with_old_authentication_backend_options( for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.get(slug="test-form") form_authentication_backend = FormAuthenticationBackend.objects.filter( @@ -2395,7 +2433,7 @@ def test_import_form_with_old_authentication_backends_and_authentication_backend for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form(import_file=self.filepath, import_options=FormImportOptions()) imported_form = Form.objects.get(slug="test-form") authentication_backends = imported_form.auth_backends.all() @@ -2459,7 +2497,12 @@ def test_import_form_with_old_and_new_authentication_backends_will_merge_both_to for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends] + ), + ) imported_form = Form.objects.get(slug="test-form") authentication_backends = imported_form.auth_backends.all() @@ -2531,7 +2574,12 @@ def test_import_form_with_auth_backends( for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends] + ), + ) imported_form = Form.objects.get(slug="test-form") authentication_backends = imported_form.auth_backends.all() @@ -2579,7 +2627,15 @@ def test_import_form_with_missing_backend_in_auth_backends( zip_file.writestr(f"{name}.json", json.dumps(data)) with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.yivi_attribute_groups + ], + ), + ) error_detail = exc.exception.detail["auth_backends"][0]["backend"][0] @@ -2610,7 +2666,12 @@ def test_import_form_with_unknown_backend_in_auth_backends( zip_file.writestr(f"{name}.json", json.dumps(data)) with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + ), + ) error_detail = exc.exception.detail["auth_backends"][0]["backend"][0] @@ -2652,7 +2713,15 @@ def test_import_form_with_yivi_auth_backend(self): for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.yivi_attribute_groups + ], + ), + ) imported_form = Form.objects.get(slug="test-form") authentication_backends = imported_form.auth_backends.all() @@ -2711,7 +2780,15 @@ def test_import_form_with_yivi_auth_backend_with_known_old_additional_attributes for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.yivi_attribute_groups + ], + ), + ) imported_form = Form.objects.get(slug="test-form") authentication_backends = imported_form.auth_backends.all() @@ -2761,7 +2838,15 @@ def test_import_form_with_yivi_auth_backend_with_unknown_additional_attributes_g zip_file.writestr(f"{name}.json", json.dumps(data)) with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.yivi_attribute_groups + ], + ), + ) error_detail = exc.exception.detail["auth_backends"][0]["options"][ "additional_attributes_groups" @@ -2812,7 +2897,12 @@ def test_import_export_in_stuf_zds(self): form_configuration=[FormConfigurationOptions.registration_backends], ), ) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends], + ), + ) updated_form = Form.objects.last() registration_backend = updated_form.registration_backends.get() @@ -2978,7 +3068,12 @@ def test_import_with_disable_next_actions(self): ) # Import form - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.prefill], + ), + ) imported_form = Form.objects.last() imported_steps = list(imported_form.formstep_set.all()) @@ -3059,21 +3154,27 @@ def test_import_with_disable_next_actions(self): rule.actions[0]["form_step_uuid"], str(imported_steps[1].uuid) ) + def test_import_unknown_domains_are_removed(self): + config = GlobalConfiguration.get_solo() -class ExportObjectsAPITests(TempdirMixin, TestCase): - @tag("gh-5384") - def test_export_form_with_objects_registration_backend(self): - objects_api_group = ObjectsAPIGroupConfigFactory.create( - identifier="test-objects-api-group" + # Start with google domain in allowlist, so we can create the initial form + config.email_template_netloc_allowlist = ["https://google.com", "allowed.com"] # pyright: ignore[reportAttributeAccessIssue] + config.save() + + form = FormFactory.create( + registration_backend="email", + registration_backend_options={ + "to_emails": ["some@email.com"], + "email_content_template_html": "

test https://google.com Google https://allowed.com test

", + "email_content_template_text": "test https://google.com https://allowed.com test", + }, ) - form = FormFactory.create() - FormRegistrationBackendFactory.create( + ConfirmationEmailTemplateFactory( form=form, - backend="objects_api", - key="test-objects-backend", - options={ - "objects_api_group": objects_api_group.identifier, - }, + subject="Test", + content="

email content https://google.com Google https://allowed.com

{% appointment_information %}

{% payment_information %}

", + cosign_subject="Cosign test", + cosign_content="

cosign email content https://google.com Google https://allowed.com

{% payment_information %}

{% cosign_information %}

", ) export_form( @@ -3085,128 +3186,1419 @@ def test_export_form_with_objects_registration_backend(self): ), ) - with zipfile.ZipFile(self.filepath, "r") as f: - self.assertEqual( - f.namelist(), - [ - "forms.json", - "formSteps.json", - "formDefinitions.json", - "formLogic.json", - "formVariables.json", - f"{EXPORT_META_KEY}.json", - ], - ) + # Delete the original form to make the slug available + form.delete() - forms = json.loads(f.read("forms.json")) - self.assertEqual(len(forms), 1) - self.assertEqual(len(forms[0]["registration_backends"]), 1) - self.assertEqual( - forms[0]["registration_backends"][0]["key"], "test-objects-backend" - ) - self.assertEqual( - forms[0]["registration_backends"][0]["options"]["objects_api_group"], - "test-objects-api-group", - ) + # Remove google domain from allowlist + config.email_template_netloc_allowlist = ["allowed.com"] # pyright: ignore[reportAttributeAccessIssue] + config.save() + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends], + ), + ) -class ImportObjectsAPITests(TempdirMixin, OFVCRMixin, TestCase): - """This test case requires the Objects & Objecttypes API and Open Zaak to be running. + # The email templates of the imported form should not contain the google domain + imported_form = Form.objects.last() - See the relevant Docker compose in the ``docker/`` folder. - """ + self.assertEqual(len(imported_form.registration_backends.all()), 1) + self.assertEqual( + imported_form.registration_backends.first().options, + { + "to_emails": ["some@email.com"], + "attach_files_to_email": None, + "email_content_template_html": "

test Google https://allowed.com test

", + "email_content_template_text": "test https://allowed.com test", + }, + ) - def test_import_form_with_objects_registration_backend_no_group(self): - resources = { - "forms": [ - { - "active": True, - "name": "Test Form 1", - "internal_name": "Test Form Internal 1", - "slug": "objects-api-no-group", - "uuid": "324cadce-a627-4e3f-b117-37ca232f16b2", - "registration_backends": [ - { - "key": "test-backend", - "name": "Test backend", - "backend": "objects_api", - "options": { - "version": 2, - "objecttype": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", - "objecttype_version": 1, - }, - } - ], - } - ] - } + # The confirmation and cosign email templates should not contain the google domain + self.assertEqual( + imported_form.confirmation_email_template.content, + "

email content Google https://allowed.com

{% appointment_information %}

{% payment_information %}

", + ) + self.assertEqual( + imported_form.confirmation_email_template.cosign_content, + "

cosign email content Google https://allowed.com

{% payment_information %}

{% cosign_information %}

", + ) - with zipfile.ZipFile(self.filepath, "w") as zip_file: - for name, data in resources.items(): - zip_file.writestr(f"{name}.json", json.dumps(data)) + def test_import_with_options_exclude_all_form_configuration(self): + merchant = WorldlineMerchantFactory.create() + objects_api_group = ObjectsAPIGroupConfigFactory.create( + identifier="test-objects-api-group" + ) - with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + form = FormFactory.create( + generate_minimal_setup=True, + authentication_backend="digid", + payment_backend="worldline", + payment_backend_options={"merchant": merchant.pspid}, + registration_backend="email", + registration_backend_options={ + "to_emails": ["abc@xyz.com"], + }, + formstep__form_definition__configuration={ + "components": [ + { + "key": "textfield", + "type": "textfield", + "label": "Textfield", + "prefill": { + "plugin": "demo", + "attribute": "random_number", + "identifier_role": "authorised_person", + }, + }, + ], + }, + ) + FormVariableFactory.create( + form=form, + key="variable_with_demo_prefill", + user_defined=True, + prefill_plugin="demo", + prefill_attribute="random_string", + prefill_identifier_role=IdentifierRoles.authorizee, + ) + FormVariableFactory.create( + form=form, + key="variable_with_objects_api_prefill", + user_defined=True, + prefill_plugin="objects_api", + prefill_options={ + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) - error_detail = exc.exception.detail["registration_backends"][0]["options"][ - "objects_api_group" - ][0] - self.assertEqual(error_detail.code, "required") + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions( + remove_sensitive_content=True, + form_configuration=[ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + ], + ), + ) + form.delete() - def test_import_form_with_objecttype_url_objects_api_registration_backend(self): - """Test forms with an Objects API registration backend where objecttype is specified as an URL - doesn't gets converted to a UUID and throws an Error. - """ + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[], + ), + ) - resources = { - "forms": [ - { - "active": True, - "name": "Test Form 1", - "internal_name": "Test Form Internal 1", - "slug": "old-objecttype-url", - "uuid": "324cadce-a627-4e3f-b117-37ca232f16b2", - "registration_backends": [ - { - "key": "test-backend", - "name": "Test backend", - "backend": "objects_api", - "options": { - "objects_api_group": ObjectsAPIGroupConfigFactory.create( - for_test_docker_compose=True - ).identifier, - "version": 2, - "objecttype": "http://localhost:8001/api/v2/objecttypes/8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", - "objecttype_version": 1, - }, - } - ], - } - ] - } + imported_form = Form.objects.last() - with zipfile.ZipFile(self.filepath, "w") as zip_file: - for name, data in resources.items(): - zip_file.writestr(f"{name}.json", json.dumps(data)) + # Expect registration, payment, and auth to be excluded + self.assertEqual(len(imported_form.auth_backends.all()), 0) + self.assertEqual(len(imported_form.registration_backends.all()), 0) + self.assertEqual(imported_form.payment_backend, "") + self.assertEqual(imported_form.payment_backend_options, {}) - with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + # The prefill config of the component should be removed + fd = imported_form.formstep_set.get().form_definition + self.assertEqual(len(fd.configuration["components"]), 1) + component = fd.configuration["components"][0] + self.assertEqual( + component["prefill"], + { + "plugin": "", + "attribute": "", + "identifier_role": IdentifierRoles.main, + }, + ) - error_detail = exc.exception.detail["registration_backends"][0]["options"][ - "objecttype" - ][0] - self.assertEqual(error_detail, "Must be a valid UUID.") - self.assertEqual(error_detail.code, "invalid") + # There should be two user-defined variables, both without prefill config + variables = FormVariable.objects.filter( + form=imported_form, source="user_defined" + ) + self.assertEqual(variables.count(), 2) + self.assertEqual(variables[0].prefill_plugin, "") + self.assertEqual(variables[0].prefill_attribute, "") + self.assertEqual(variables[0].prefill_identifier_role, IdentifierRoles.main) + self.assertEqual(variables[0].prefill_options, {}) - with self.assertRaises(FormRegistrationBackend.DoesNotExist): - FormRegistrationBackend.objects.get(key="test-backend") + self.assertEqual(variables[1].prefill_plugin, "") + self.assertEqual(variables[1].prefill_attribute, "") + self.assertEqual(variables[1].prefill_identifier_role, IdentifierRoles.main) + self.assertEqual(variables[1].prefill_options, {}) - def test_import_form_with_objecttype_uuid_objects_api_registration_backend(self): - """Test forms with an Objects API registration backend where objecttype is specified as an UUID - stays as is. - """ + def test_import_with_options_include_all_form_configuration(self): + merchant = WorldlineMerchantFactory.create() + objects_api_group = ObjectsAPIGroupConfigFactory.create( + identifier="test-objects-api-group" + ) - resources = { + form = FormFactory.create( + generate_minimal_setup=True, + authentication_backend="digid", + payment_backend="worldline", + payment_backend_options={"merchant": merchant.pspid}, + registration_backend="email", + registration_backend_options={ + "to_emails": ["abc@xyz.com"], + }, + formstep__form_definition__configuration={ + "components": [ + { + "key": "textfield", + "type": "textfield", + "label": "Textfield", + "prefill": { + "plugin": "demo", + "attribute": "random_number", + "identifier_role": IdentifierRoles.authorizee, + }, + }, + ], + }, + ) + FormVariableFactory.create( + form=form, + key="variable_with_demo_prefill", + user_defined=True, + prefill_plugin="demo", + prefill_attribute="random_string", + prefill_identifier_role=IdentifierRoles.authorizee, + ) + FormVariableFactory.create( + form=form, + key="variable_with_objects_api_prefill", + user_defined=True, + prefill_plugin="objects_api", + prefill_options={ + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions( + remove_sensitive_content=False, + form_configuration=[ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + ), + ) + form.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[ + FormConfigurationOptions.registration_backends, + FormConfigurationOptions.prefill, + FormConfigurationOptions.payment_backend, + FormConfigurationOptions.auth_backends, + ], + ), + ) + + imported_form = Form.objects.last() + + # Expect registration, payment, and auth to be the same as the exported form + self.assertEqual(len(imported_form.auth_backends.all()), 1) + self.assertEqual(imported_form.auth_backends.first().backend, "digid") + + self.assertEqual(len(imported_form.registration_backends.all()), 1) + registration_backend = imported_form.registration_backends.first() + self.assertEqual(registration_backend.backend, "email") + self.assertEqual( + registration_backend.options["to_emails"], + ["abc@xyz.com"], + ) + + self.assertEqual(imported_form.payment_backend, "worldline") + self.assertEqual( + imported_form.payment_backend_options["merchant"], merchant.pspid + ) + + # The prefill config of the component should be removed + fd = imported_form.formstep_set.get().form_definition + self.assertEqual(len(fd.configuration["components"]), 1) + component = fd.configuration["components"][0] + self.assertEqual( + component["prefill"], + { + "plugin": "demo", + "attribute": "random_number", + "identifier_role": IdentifierRoles.authorizee, + }, + ) + + # There should be two user-defined variables, both without prefill config + variables = FormVariable.objects.filter( + form=imported_form, source=FormVariableSources.user_defined + ) + self.assertEqual(variables.count(), 2) + variable_with_demo = variables.get(key="variable_with_demo_prefill") + variable_with_objects_api = variables.get( + key="variable_with_objects_api_prefill" + ) + + self.assertEqual(variable_with_demo.prefill_plugin, "demo") + self.assertEqual(variable_with_demo.prefill_attribute, "random_string") + self.assertEqual( + variable_with_demo.prefill_identifier_role, IdentifierRoles.authorizee + ) + + self.assertEqual(variable_with_objects_api.prefill_plugin, "objects_api") + self.assertEqual( + variable_with_objects_api.prefill_options, + { + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) + + def test_import_with_options_exclude_all_additional_form_configuration(self): + # Every OF instance has 5 default WMTS and 1 default WMS map tile layer. For more + # accurate and easier testing, we should start at zero. + MapTileLayer.objects.all().delete() + MapWMSTileLayer.objects.all().delete() + + product = ProductFactory.create() + theme = ThemeFactory.create(design_token_values={"key": "token"}) + category = CategoryFactory.create() + + wmtsMap1 = MapTileLayerFactory.create() + wmtsMap2 = MapTileLayerFactory.create() + wmsMap1 = MapWMSTileLayerFactory.create() + wmsMap2 = MapWMSTileLayerFactory.create() + wmsMap3 = MapWMSTileLayerFactory.create() + + yiviAttributeGroup1 = AttributeGroupFactory.create( + attributes=["first_name", "last_name"] + ) + yiviAttributeGroup2 = AttributeGroupFactory.create(attributes=["email_address"]) + + # Define form with all additional form configuration + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + theme=theme, + category=category, + authentication_backend="yivi_oidc", + authentication_backend__options={ + "authentication_options": [AuthAttribute.bsn], + "additional_attributes_groups": [ + # The uuids of the `personal` and `mail` attribute groups + yiviAttributeGroup1.uuid, + yiviAttributeGroup2.uuid, + ], + }, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map 1", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmtsMap1.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wmsMap1.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + }, + ], + }, + { + "label": "Map 2", + "key": "map2", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmtsMap2.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wmsMap2.uuid), + "label": "height", + "layers": ["EL.GridCoverage"], + }, + { + "url": "", + "type": "wms", + "uuid": str(wmsMap3.uuid), + "label": "LGN", + "layers": ["lgn-actueel"], + }, + ], + }, + ], + }, + ) + + # The export is made with all additional_form_configuration + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions( + additional_form_configuration=[ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + ), + ) + + # Delete all previous data + form.delete() + product.delete() + theme.delete() + category.delete() + wmtsMap1.delete() + wmtsMap2.delete() + wmsMap1.delete() + wmsMap2.delete() + wmsMap3.delete() + yiviAttributeGroup1.delete() + yiviAttributeGroup2.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[], + ), + ) + + imported_form = Form.objects.last() + + # The imported form has no product, theme or category + self.assertEqual(imported_form.product, None) + self.assertEqual(imported_form.category, None) + self.assertEqual(imported_form.theme, None) + + # Auth backend is imported without the additional attribute groups + self.assertEqual(len(imported_form.auth_backends.all()), 1) + auth_backend = imported_form.auth_backends.first() + self.assertEqual(auth_backend.backend, "yivi_oidc") + self.assertEqual( + auth_backend.options["authentication_options"], [AuthAttribute.bsn] + ) + self.assertEqual(auth_backend.options["additional_attributes_groups"], []) + + # Map components don't have configured background and overlay tile layers + fd = imported_form.formstep_set.get().form_definition + self.assertEqual(len(fd.configuration["components"]), 2) + map_component1 = fd.configuration["components"][0] + map_component2 = fd.configuration["components"][1] + + self.assertEqual(map_component1["tileLayerIdentifier"], "") + self.assertEqual( + map_component1["overlays"], + [ + { + "url": "", + "type": "wms", + "uuid": "", + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": [], + } + ], + ) + + self.assertEqual(map_component2["tileLayerIdentifier"], "") + self.assertEqual( + map_component2["overlays"], + [ + { + "url": "", + "type": "wms", + "uuid": "", + "label": "height", + "layers": [], + }, + { + "url": "", + "type": "wms", + "uuid": "", + "label": "LGN", + "layers": [], + }, + ], + ) + + # None of the additional form configurations have been imported/created + self.assertEqual(Product.objects.count(), 0) + self.assertEqual(Theme.objects.count(), 0) + self.assertEqual(Category.objects.count(), 0) + self.assertEqual(MapTileLayer.objects.count(), 0) + self.assertEqual(MapWMSTileLayer.objects.count(), 0) + self.assertEqual(AttributeGroup.objects.count(), 0) + + def test_import_with_options_include_all_additional_form_configuration_create_new( + self, + ): + # Every OF instance has a couple default WMTS- and WMS-tile layers. + # For more accurate and easier testing, we should start at zero. + MapTileLayer.objects.all().delete() + MapWMSTileLayer.objects.all().delete() + + product = ProductFactory.create() + theme = ThemeFactory.create(design_token_values={"key": "token"}) + category = CategoryFactory.create() + + wmtsMap1 = MapTileLayerFactory.create( + identifier="wmts-map-1", url="https://example.wmts.1.com", label="wmtsMap1" + ) + wmtsMap2 = MapTileLayerFactory.create( + identifier="wmts-map-2", url="https://example.wmts.2.com", label="wmtsMap2" + ) + wmsMap1 = MapWMSTileLayerFactory.create( + url="https://example.wms.1.com", name="wmsMap1" + ) + wmsMap2 = MapWMSTileLayerFactory.create( + url="https://example.wms.2.com", name="wmsMap2" + ) + wmsMap3 = MapWMSTileLayerFactory.create( + url="https://example.wms.3.com", name="wmsMap3" + ) + + yiviAttributeGroup1 = AttributeGroupFactory.create( + attributes=["first_name", "last_name"] + ) + yiviAttributeGroup2 = AttributeGroupFactory.create(attributes=["email_address"]) + + # Define form with all additional form configuration + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + theme=theme, + category=category, + authentication_backend="yivi_oidc", + authentication_backend__options={ + "authentication_options": [AuthAttribute.bsn], + "additional_attributes_groups": [ + # The uuids of the `personal` and `mail` attribute groups + yiviAttributeGroup1.uuid, + yiviAttributeGroup2.uuid, + ], + }, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map 1", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmtsMap1.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wmsMap1.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + }, + ], + }, + { + "label": "Map 2", + "key": "map2", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmtsMap2.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wmsMap2.uuid), + "label": "height", + "layers": ["EL.GridCoverage"], + }, + { + "url": "", + "type": "wms", + "uuid": str(wmsMap3.uuid), + "label": "LGN", + "layers": ["lgn-actueel"], + }, + ], + }, + ], + }, + ) + + # The export is made with all additional_form_configuration + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + ), + ) + + # Delete all previous data + form.delete() + product.delete() + theme.delete() + category.delete() + wmtsMap1.delete() + wmtsMap2.delete() + wmsMap1.delete() + wmsMap2.delete() + wmsMap3.delete() + yiviAttributeGroup1.delete() + yiviAttributeGroup2.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + ), + ) + + imported_form = Form.objects.last() + + # All of the additional form configurations have been imported and created, + # except for theme and category. + self.assertEqual(Product.objects.count(), 1) + self.assertEqual(Theme.objects.count(), 0) + self.assertEqual(Category.objects.count(), 0) + self.assertEqual(MapTileLayer.objects.count(), 2) + self.assertEqual(MapWMSTileLayer.objects.count(), 3) + self.assertEqual(AttributeGroup.objects.count(), 2) + imported_product = Product.objects.last() + imported_wmts_layers = MapTileLayer.objects.all() + imported_wms_layers = MapWMSTileLayer.objects.all() + attribute_groups = AttributeGroup.objects.all() + + # The imported form has product + self.assertEqual(str(imported_form.product.uuid), str(imported_product.uuid)) + # Theme and category are not set + self.assertIsNone(imported_form.category) + self.assertIsNone(imported_form.theme) + + # Auth backend is imported with the additional attribute groups + self.assertEqual(len(imported_form.auth_backends.all()), 1) + auth_backend = imported_form.auth_backends.first() + self.assertEqual(auth_backend.backend, "yivi_oidc") + self.assertEqual( + auth_backend.options["additional_attributes_groups"], + [ + str(attribute_groups[0].uuid), + str(attribute_groups[1].uuid), + ], + ) + + # Map components have configured background and overlay tile layers + fd = imported_form.formstep_set.get().form_definition + self.assertEqual(len(fd.configuration["components"]), 2) + map_component1 = fd.configuration["components"][0] + map_component2 = fd.configuration["components"][1] + + self.assertEqual( + map_component1["tileLayerIdentifier"], + str(imported_wmts_layers[0].identifier), + ) + self.assertEqual( + map_component1["overlays"], + [ + { + "url": "", + "type": "wms", + "uuid": str(imported_wms_layers[0].uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + } + ], + ) + + self.assertEqual( + map_component2["tileLayerIdentifier"], + str(imported_wmts_layers[1].identifier), + ) + self.assertEqual( + map_component2["overlays"], + [ + { + "url": "", + "type": "wms", + "uuid": str(imported_wms_layers[1].uuid), + "label": "height", + "layers": ["EL.GridCoverage"], + }, + { + "url": "", + "type": "wms", + "uuid": str(imported_wms_layers[2].uuid), + "label": "LGN", + "layers": ["lgn-actueel"], + }, + ], + ) + + # The configuration of the WMS and WMTS tile layers hasn't changed + wmts_layer_1 = imported_wmts_layers.get(identifier="wmts-map-1") + wmts_layer_2 = imported_wmts_layers.get(identifier="wmts-map-2") + self.assertEqual(wmts_layer_1.url, "https://example.wmts.1.com") + self.assertEqual(wmts_layer_1.label, "wmtsMap1") + self.assertEqual(wmts_layer_2.url, "https://example.wmts.2.com") + self.assertEqual(wmts_layer_2.label, "wmtsMap2") + + wms_layer_1 = imported_wms_layers.get(name="wmsMap1") + wms_layer_2 = imported_wms_layers.get(name="wmsMap2") + wms_layer_3 = imported_wms_layers.get(name="wmsMap3") + self.assertEqual(wms_layer_1.url, "https://example.wms.1.com") + self.assertEqual(wms_layer_2.url, "https://example.wms.2.com") + self.assertEqual(wms_layer_3.url, "https://example.wms.3.com") + + def test_import_with_options_include_all_additional_form_configuration_reuses_already_existing_objects( + self, + ): + # Every OF instance has 5 default WMTS and 1 default WMS map tile layer. For more + # accurate and easier testing, we should start at zero. + MapTileLayer.objects.all().delete() + MapWMSTileLayer.objects.all().delete() + + product = ProductFactory.create() + theme = ThemeFactory.create(design_token_values={"key": "token"}) + category = CategoryFactory.create() + + wmts_layer = MapTileLayerFactory.create() + wms_layer = MapWMSTileLayerFactory.create() + + yiviAttributeGroup = AttributeGroupFactory.create( + attributes=["first_name", "last_name"] + ) + + # Define form with all additional form configuration + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + theme=theme, + category=category, + authentication_backend="yivi_oidc", + authentication_backend__options={ + "authentication_options": [AuthAttribute.bsn], + "additional_attributes_groups": [yiviAttributeGroup.uuid], + }, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map 1", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmts_layer.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wms_layer.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + }, + ], + }, + ], + }, + ) + + # The export is made with all additional_form_configuration + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions( + additional_form_configuration=[ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + ), + ) + + # Only delete the form + form.delete() + # Change UUID's, as would be the case in a regular situation where a form is + # shared across different OF instances. + product.uuid = uuid4() + wms_layer.uuid = uuid4() + yiviAttributeGroup.uuid = uuid4() + product.save() + wms_layer.save() + yiviAttributeGroup.save() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.auth_backends], + additional_form_configuration=[ + AdditionalFormConfigurationOptions.product, + AdditionalFormConfigurationOptions.wms_tile_layers, + AdditionalFormConfigurationOptions.wmts_tile_layers, + AdditionalFormConfigurationOptions.yivi_attribute_groups, + ], + ), + ) + + imported_form = Form.objects.last() + + # No new data was created + self.assertEqual(Product.objects.count(), 1) + self.assertEqual(Theme.objects.count(), 1) + self.assertEqual(Category.objects.count(), 1) + self.assertEqual(MapTileLayer.objects.count(), 1) + self.assertEqual(MapWMSTileLayer.objects.count(), 1) + self.assertEqual(AttributeGroup.objects.count(), 1) + + # The imported form uses all existing objects + self.assertEqual(str(imported_form.product.uuid), str(product.uuid)) + # Theme and category should be removed + self.assertIsNone(imported_form.category) + self.assertIsNone(imported_form.theme) + + self.assertEqual(len(imported_form.auth_backends.all()), 1) + auth_backend = imported_form.auth_backends.first() + self.assertEqual(auth_backend.backend, "yivi_oidc") + self.assertEqual( + auth_backend.options["additional_attributes_groups"], + [str(yiviAttributeGroup.uuid)], + ) + + fd = imported_form.formstep_set.get().form_definition + self.assertEqual(len(fd.configuration["components"]), 1) + map_component1 = fd.configuration["components"][0] + + self.assertEqual( + map_component1["tileLayerIdentifier"], str(wmts_layer.identifier) + ) + self.assertEqual( + map_component1["overlays"], + [ + { + "url": "", + "type": "wms", + "uuid": str(wms_layer.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + } + ], + ) + + def test_import_with_options_reuse_form_definitions_with_duplicate_configurations( + self, + ): + # Expect existing form definitions to be re-used + form = FormFactory.create() + form_definition = FormDefinitionFactory.create( + is_reusable=True, + configuration={ + "components": [ + { + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + ], + }, + ) + FormStepFactory.create(form=form, form_definition=form_definition) + + # The export is made with all additional_form_configuration + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=True), + ) + + imported_form = Form.objects.last() + + # No new form definitions have been created + self.assertEqual(FormDefinition.objects.count(), 1) + + # Assert that the imported form FD is the same as the existing FD + imported_form_definition = imported_form.formstep_set.get().form_definition + self.assertEqual(imported_form_definition, form_definition) + + def test_import_with_options_reuse_form_definitions_with_different_configurations( + self, + ): + # Expect existing form definitions to be re-used + form = FormFactory.create() + form_definition = FormDefinitionFactory.create( + is_reusable=True, + configuration={ + "components": [ + { + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + ], + }, + ) + FormStepFactory.create(form=form, form_definition=form_definition) + + # The export is made with all additional_form_configuration + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Update form definition + form_definition.configuration = { + "components": [ + { + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + { + "label": "Textfield 2", + "key": "textfield2", + "type": "textfield", + }, + ], + } + form_definition.save() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=True), + ) + + imported_form = Form.objects.last() + + # A new form definition has been created, as the imported form FD differs from + # the existing FD + self.assertEqual(FormDefinition.objects.count(), 2) + + # Assert that the imported form FD is the same as the existing FD + imported_form_definition = imported_form.formstep_set.get().form_definition + self.assertNotEqual(imported_form_definition, form_definition) + # The imported FD has the same configuration as the original form had during + # exporting + self.assertEqual( + imported_form_definition.configuration, + { + "components": [ + { + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + ], + }, + ) + + def test_import_with_options_reuse_form_definitions_with_different_ids_in_configuration( + self, + ): + form = FormFactory.create() + form_definition = FormDefinitionFactory.create( + is_reusable=True, + configuration={ + "components": [ + { + "id": "c4e49f", + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + ], + }, + ) + FormStepFactory.create(form=form, form_definition=form_definition) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + form.delete() + + # Change the component id + form_definition.configuration = { + "components": [ + { + "id": "eb0d38", + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + ], + } + form_definition.save() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=True), + ) + + imported_form = Form.objects.last() + + # There should be no new form definitions + self.assertEqual(FormDefinition.objects.count(), 1) + + # Assert that the imported form FD is the same as the existing FD + imported_form_definition = imported_form.formstep_set.get().form_definition + self.assertEqual(imported_form_definition, form_definition) + + def test_import_with_options_create_all_new_form_definitions(self): + # Expect existing form definitions to be re-used + form = FormFactory.create() + form_definition = FormDefinitionFactory.create( + is_reusable=True, + configuration={ + "components": [ + { + "label": "Textfield", + "key": "textfield", + "type": "textfield", + }, + ], + }, + ) + FormStepFactory.create(form=form, form_definition=form_definition) + + # The export is made with all additional_form_configuration + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions(reuse_form_definitions=False), + ) + + imported_form = Form.objects.last() + + # New form definition has been created + self.assertEqual(FormDefinition.objects.count(), 2) + + # Assert that the imported form FD is similar to the existing FD + imported_form_definition = imported_form.formstep_set.get().form_definition + self.assertNotEqual(imported_form_definition, form_definition) + self.assertEqual( + imported_form_definition.configuration, form_definition.configuration + ) + + def test_import_with_theme_option(self): + theme = ThemeFactory.create() + theme2 = ThemeFactory.create() + form = FormFactory.create(theme=theme) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions(theme=str(theme2.uuid)), + ) + + imported_form = Form.objects.last() + self.assertEqual(imported_form.theme, theme2) + + def test_import_without_theme_option(self): + theme = ThemeFactory.create() + form = FormFactory.create(theme=theme) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Import form + import_form( + import_file=self.filepath, import_options=FormImportOptions(theme=None) + ) + + imported_form = Form.objects.last() + self.assertIsNone(imported_form.theme) + + def test_import_with_category_option(self): + category = CategoryFactory.create() + category2 = CategoryFactory.create() + form = FormFactory.create(category=category) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Import form + import_form( + import_file=self.filepath, + import_options=FormImportOptions(category=str(category2.uuid)), + ) + + imported_form = Form.objects.last() + self.assertEqual(imported_form.category, category2) + + def test_import_without_category_option(self): + category = CategoryFactory.create() + form = FormFactory.create(category=category) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions(), + ) + + # Only delete the form + form.delete() + + # Import form + import_form( + import_file=self.filepath, import_options=FormImportOptions(category=None) + ) + + imported_form = Form.objects.last() + self.assertIsNone(imported_form.category) + + +@temp_private_root(reset_storage=False) +class BulkImportExportTests(TempdirMixin, TestCase): + def test_bulk_import_with_same_reusable_form_definition(self): + """ + Expect that when multiple instances of the same reusable form definition are + imported at the same time with the import option `reuse_form_definitions=True`, + that only one is created which will be used by all imported forms. + """ + user = SuperUserFactory.create(email="test@email.nl") + + form1 = FormFactory.create() + form2 = FormFactory.create() + form_definition = FormDefinitionFactory.create( + is_reusable=True, + configuration={ + "components": [ + { + "label": "Textfield", + "key": "textfield", + "type": "textfield", + } + ] + }, + ) + FormStepFactory.create(form=form1, form_definition=form_definition) + FormStepFactory.create(form=form2, form_definition=form_definition) + + # Perform bulk export + process_forms_export( + forms_uuids=[form1.uuid, form2.uuid], + user_id=user.id, + export_options={}, + ) + form_export = FormsExport.objects.get() + + # Remove the original forms and form definition + form1.delete() + form2.delete() + form_definition.delete() + self.assertEqual(Form.objects.count(), 0) + self.assertEqual(FormDefinition.objects.count(), 0) + + # Perform bulk import + exported_zip_file = form_export.export_content + exported_zip_file.seek(0) + + name = "imports/tmp_import_file.zip" + filename = private_media_storage.save(name, exported_zip_file) + + process_forms_import( + str(filename), + user.id, + import_options={ + "reuse_form_definitions": True, + }, + ) + + # Check that the import file is cleaned up + self.assertFalse(private_media_storage.exists(filename)) + + # There should be 2 form and 1 form definition + self.assertEqual(2, Form.objects.count()) + self.assertEqual(1, FormDefinition.objects.count()) + # Both forms use the same form definition + imported_form1 = Form.objects.first() + imported_form2 = Form.objects.last() + imported_form_definition = FormDefinition.objects.first() + self.assertEqual( + imported_form1.formstep_set.get().form_definition, imported_form_definition + ) + self.assertEqual( + imported_form2.formstep_set.get().form_definition, imported_form_definition + ) + self.assertEqual( + imported_form2.formstep_set.get().form_definition, imported_form_definition + ) + + +class ExportObjectsAPITests(TempdirMixin, TestCase): + @tag("gh-5384") + def test_export_form_with_objects_registration_backend(self): + objects_api_group = ObjectsAPIGroupConfigFactory.create( + identifier="test-objects-api-group" + ) + form = FormFactory.create() + FormRegistrationBackendFactory.create( + form=form, + backend="objects_api", + key="test-objects-backend", + options={ + "objects_api_group": objects_api_group.identifier, + }, + ) + + export_form( + form.pk, + archive_name=self.filepath, + export_options=FormExportOptions( + remove_sensitive_content=False, + form_configuration=[FormConfigurationOptions.registration_backends], + ), + ) + + with zipfile.ZipFile(self.filepath, "r") as f: + self.assertEqual( + f.namelist(), + [ + "forms.json", + "formSteps.json", + "formDefinitions.json", + "formLogic.json", + "formVariables.json", + f"{EXPORT_META_KEY}.json", + ], + ) + + forms = json.loads(f.read("forms.json")) + self.assertEqual(len(forms), 1) + self.assertEqual(len(forms[0]["registration_backends"]), 1) + self.assertEqual( + forms[0]["registration_backends"][0]["key"], "test-objects-backend" + ) + self.assertEqual( + forms[0]["registration_backends"][0]["options"]["objects_api_group"], + "test-objects-api-group", + ) + + +class ImportObjectsAPITests(TempdirMixin, OFVCRMixin, TestCase): + """This test case requires the Objects & Objecttypes API and Open Zaak to be running. + + See the relevant Docker compose in the ``docker/`` folder. + """ + + def test_import_form_with_objects_registration_backend_no_group(self): + resources = { + "forms": [ + { + "active": True, + "name": "Test Form 1", + "internal_name": "Test Form Internal 1", + "slug": "objects-api-no-group", + "uuid": "324cadce-a627-4e3f-b117-37ca232f16b2", + "registration_backends": [ + { + "key": "test-backend", + "name": "Test backend", + "backend": "objects_api", + "options": { + "version": 2, + "objecttype": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 1, + }, + } + ], + } + ] + } + + with zipfile.ZipFile(self.filepath, "w") as zip_file: + for name, data in resources.items(): + zip_file.writestr(f"{name}.json", json.dumps(data)) + + with self.assertRaises(ValidationError) as exc: + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) + + error_detail = exc.exception.detail["registration_backends"][0]["options"][ + "objects_api_group" + ][0] + self.assertEqual(error_detail.code, "required") + + def test_import_form_with_objecttype_url_objects_api_registration_backend(self): + """Test forms with an Objects API registration backend where objecttype is specified as an URL + doesn't gets converted to a UUID and throws an Error. + """ + + resources = { + "forms": [ + { + "active": True, + "name": "Test Form 1", + "internal_name": "Test Form Internal 1", + "slug": "old-objecttype-url", + "uuid": "324cadce-a627-4e3f-b117-37ca232f16b2", + "registration_backends": [ + { + "key": "test-backend", + "name": "Test backend", + "backend": "objects_api", + "options": { + "objects_api_group": ObjectsAPIGroupConfigFactory.create( + for_test_docker_compose=True + ).identifier, + "version": 2, + "objecttype": "http://localhost:8001/api/v2/objecttypes/8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 1, + }, + } + ], + } + ] + } + + with zipfile.ZipFile(self.filepath, "w") as zip_file: + for name, data in resources.items(): + zip_file.writestr(f"{name}.json", json.dumps(data)) + + with self.assertRaises(ValidationError) as exc: + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) + + error_detail = exc.exception.detail["registration_backends"][0]["options"][ + "objecttype" + ][0] + self.assertEqual(error_detail, "Must be a valid UUID.") + self.assertEqual(error_detail.code, "invalid") + + with self.assertRaises(FormRegistrationBackend.DoesNotExist): + FormRegistrationBackend.objects.get(key="test-backend") + + def test_import_form_with_objecttype_uuid_objects_api_registration_backend(self): + """Test forms with an Objects API registration backend where objecttype is specified as an UUID + stays as is. + """ + + resources = { "forms": [ { "active": True, @@ -3237,7 +4629,12 @@ def test_import_form_with_objecttype_uuid_objects_api_registration_backend(self) for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend = FormRegistrationBackend.objects.get(key="test-backend") self.assertEqual( @@ -3278,7 +4675,12 @@ def test_import_form_with_objects_registration_backend_no_version_and_no_variabl for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend = FormRegistrationBackend.objects.get(key="test-backend") @@ -3335,7 +4737,12 @@ def test_import_form_with_objects_registration_backend_without_variables_mapping for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend_v1 = FormRegistrationBackend.objects.get( key="test-backend-v1" @@ -3390,7 +4797,12 @@ def test_import_form_with_objects_registration_backend_with_valid_variables_mapp for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend_valid_mapping = FormRegistrationBackend.objects.get( key="test-backend" @@ -3439,7 +4851,12 @@ def test_import_form_with_objects_registration_backend_with_invalid_variables_ma zip_file.writestr(f"{name}.json", json.dumps(data)) with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) error_detail = exc.exception.detail["registration_backends"][0]["options"][ "variables_mapping" @@ -3484,7 +4901,12 @@ def test_import_form_with_objects_api_group_pk(self): for name, data in resources.items(): zip_file.writestr(f"{name}.json", json.dumps(data)) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend = FormRegistrationBackend.objects.get(key="test-backend") self.assertEqual( @@ -3599,7 +5021,12 @@ def test_import_form_with_legacy_file_registration_options(self): ) form.delete() - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) backends: dict[str, FormRegistrationBackend] = { backend.key: backend for backend in FormRegistrationBackend.objects.all() @@ -3690,7 +5117,12 @@ def test_import_form_with_zgw_registration_backend_with_objects_api_group_apply_ }, ) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend = FormRegistrationBackend.objects.get(key="test-backend") self.assertEqual( @@ -3722,7 +5154,14 @@ def test_import_form_with_zgw_registration_backend_cant_determine_objects_api_gr ) with self.assertRaises(ValidationError) as exc: - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[ + FormConfigurationOptions.registration_backends + ] + ), + ) error_detail = exc.exception.detail["registration_backends"][0]["options"][ "objects_api_group" @@ -3751,7 +5190,12 @@ def test_import_form_with_zgw_registration_backend_cant_determine_objects_api_gr }, ) - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) registration_backend = FormRegistrationBackend.objects.get( key="test-backend" @@ -3852,7 +5296,12 @@ def test_import_form_with_legacy_file_registration_options(self): ) form.delete() - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) backends: dict[str, FormRegistrationBackend] = { backend.key: backend for backend in FormRegistrationBackend.objects.all() @@ -3986,7 +5435,12 @@ def test_import_form_with_legacy_file_registration_options(self): ) form.delete() - import_form(import_file=self.filepath) + import_form( + import_file=self.filepath, + import_options=FormImportOptions( + form_configuration=[FormConfigurationOptions.registration_backends] + ), + ) backends: dict[str, FormRegistrationBackend] = { backend.key: backend for backend in FormRegistrationBackend.objects.all() diff --git a/src/openforms/forms/tests/test_remove_key_from_dict.py b/src/openforms/forms/tests/test_remove_key_from_dict.py deleted file mode 100644 index 1490ffc545..0000000000 --- a/src/openforms/forms/tests/test_remove_key_from_dict.py +++ /dev/null @@ -1,30 +0,0 @@ -from django.test import TestCase - -from openforms.forms.utils import remove_key_from_dict - - -class RemoveKeyFromDictTests(TestCase): - def test_remove_key_from_dict_removes_expected_key(self): - dictionary = { - "remove_me": False, - "dont_remove_me": True, - "inner_dict": {"remove_me": False, "dont_remove_me": True}, - "a_list": [ - { - "remove_me": False, - "dont_remove_me": True, - "inner_dict": {"remove_me": False, "dont_remove_me": True}, - } - ], - } - - remove_key_from_dict(dictionary, "remove_me") - - expected_result = { - "dont_remove_me": True, - "inner_dict": {"dont_remove_me": True}, - "a_list": [ - {"dont_remove_me": True, "inner_dict": {"dont_remove_me": True}} - ], - } - self.assertEqual(dictionary, expected_result) diff --git a/src/openforms/forms/tests/test_restore_version.py b/src/openforms/forms/tests/test_restore_version.py index bed0e57ae0..755da2786b 100644 --- a/src/openforms/forms/tests/test_restore_version.py +++ b/src/openforms/forms/tests/test_restore_version.py @@ -8,10 +8,27 @@ from freezegun import freeze_time +from openforms.authentication.constants import AuthAttribute +from openforms.authentication.contrib.yivi_oidc.models import AttributeGroup +from openforms.authentication.tests.factories import AttributeGroupFactory +from openforms.config.models import MapTileLayer, MapWMSTileLayer, Theme +from openforms.config.tests.factories import ( + MapTileLayerFactory, + MapWMSTileLayerFactory, + ThemeFactory, +) +from openforms.contrib.objects_api.tests.factories import ObjectsAPIGroupConfigFactory +from openforms.forms.models import Category, FormDefinition, FormStep, FormVersion +from openforms.payments.contrib.worldline.tests.factories import ( + WorldlineMerchantFactory, +) +from openforms.prefill.constants import IdentifierRoles +from openforms.products.models import Product +from openforms.products.tests.factories import ProductFactory from openforms.variables.constants import FormVariableDataTypes, FormVariableSources -from ..models import FormDefinition, FormStep, FormVersion from .factories import ( + CategoryFactory, FormDefinitionFactory, FormFactory, FormStepFactory, @@ -325,6 +342,374 @@ def test_restore_form_with_reusable_form_definition(self): ) self.assertFalse(form_steps[1].form_definition.is_reusable) + def test_full_form_save_and_restore(self): + """ + Tests that all form configuration that receives dedicated attention from the + import/export is correctly saved and restored. + + Ensuring all configuration is kept, especially the product, theme, category, and + service fetch. (@TODO service fetch is not yet implemented) + """ + product = ProductFactory.create() + theme = ThemeFactory.create(design_token_values={"key": "token"}) + category = CategoryFactory.create() + + wmts_tile_layer = MapTileLayerFactory.create() + wms_tile_layer = MapWMSTileLayerFactory.create() + + yivi_attribute_group = AttributeGroupFactory.create( + attributes=["first_name", "last_name"] + ) + + merchant = WorldlineMerchantFactory.create() + objects_api_group = ObjectsAPIGroupConfigFactory.create( + identifier="test-objects-api-group" + ) + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + theme=theme, + category=category, + authentication_backend="yivi_oidc", + authentication_backend__options={ + "authentication_options": [AuthAttribute.bsn], + "additional_attributes_groups": [yivi_attribute_group.uuid], + }, + internal_remarks="Some internal remark that should be kept", + payment_backend="worldline", + payment_backend_options={"merchant": merchant.pspid}, + registration_backend="email", + registration_backend_options={"to_emails": ["abc@xyz.com"]}, + formstep__form_definition__configuration={ + "components": [ + { + "key": "textfield", + "type": "textfield", + "label": "Textfield", + "prefill": { + "plugin": "demo", + "attribute": "random_number", + "identifier_role": IdentifierRoles.authorizee, + }, + }, + { + "label": "Map", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmts_tile_layer.identifier, + "overlays": [ + { + "url": "", + "type": "wms", + "uuid": str(wms_tile_layer.uuid), + "label": "Basisregistratie Adressen en Gebouwen (BAG)", + "layers": ["pand", "verblijfsobject"], + }, + ], + }, + ], + }, + ) + FormVariableFactory.create( + form=form, + key="variable_with_demo_prefill", + user_defined=True, + prefill_plugin="demo", + prefill_attribute="random_string", + prefill_identifier_role=IdentifierRoles.authorizee, + ) + FormVariableFactory.create( + form=form, + key="variable_with_objects_api_prefill", + user_defined=True, + prefill_plugin="objects_api", + prefill_options={ + "objects_api_group": objects_api_group.identifier, + "objecttype_uuid": "8e46e0a5-b1b4-449b-b9e9-fa3cea655f48", + "objecttype_version": 3, + "variables_mapping": [ + {"variable_key": "lastName", "target_path": ["name", "last.name"]}, + {"variable_key": "age", "target_path": ["age"]}, + ], + "auth_attribute_path": ["bsn"], + }, + ) + + # Validate the initial setup + self.assertEqual(Product.objects.count(), 1) + self.assertEqual(Theme.objects.count(), 1) + self.assertEqual(Category.objects.count(), 1) + self.assertEqual(MapTileLayer.objects.count(), 6) + self.assertEqual(MapWMSTileLayer.objects.count(), 2) + self.assertEqual(AttributeGroup.objects.count(), 1) + + version = FormVersionFactory.create(form=form) + self.assertNotEqual(version.export_blob, {}) + + # Restore it + form.restore_old_version(version.uuid) + + # get all fresh DB records + form.refresh_from_db() + + # Validate that no new additional data was created + self.assertEqual(Product.objects.count(), 1) + self.assertEqual(MapTileLayer.objects.count(), 6) + self.assertEqual(MapWMSTileLayer.objects.count(), 2) + self.assertEqual(AttributeGroup.objects.count(), 1) + + # Validate product, theme and category + self.assertEqual(form.product.uuid, product.uuid) + self.assertEqual(form.theme.uuid, theme.uuid) + self.assertEqual(form.category.uuid, category.uuid) + + # Validate payment backend + self.assertEqual(form.payment_backend, "worldline") + self.assertEqual(form.payment_backend_options["merchant"], merchant.pspid) + + # Validate auth backend + self.assertEqual(form.auth_backends.count(), 1) + auth_backend = form.auth_backends.first() + self.assertEqual(auth_backend.backend, "yivi_oidc") + self.assertEqual( + auth_backend.options["authentication_options"], [AuthAttribute.bsn] + ) + self.assertEqual( + auth_backend.options["additional_attributes_groups"], + [str(yivi_attribute_group.uuid)], + ) + + # Validate registration backend and sensitive data was kept + self.assertEqual( + form.internal_remarks, + "Some internal remark that should be kept", + ) + self.assertEqual(form.registration_backends.count(), 1) + registration_backend = form.registration_backends.first() + self.assertEqual(registration_backend.backend, "email") + self.assertEqual(registration_backend.options["to_emails"], ["abc@xyz.com"]) + + # Validate map component tile layer configuration + fd = form.formstep_set.first().form_definition + map_component = fd.configuration["components"][1] + self.assertEqual(map_component["type"], "map") + self.assertEqual( + map_component["tileLayerIdentifier"], wmts_tile_layer.identifier + ) + self.assertEqual(len(map_component["overlays"]), 1) + self.assertEqual(map_component["overlays"][0]["uuid"], str(wms_tile_layer.uuid)) + + def test_form_restore_creating_new_resources_when_existing_have_been_altered(self): + product = ProductFactory.create(name="old product name", price=10) + form = FormFactory.create( + generate_minimal_setup=True, + product=product, + ) + + version = FormVersionFactory.create(form=form) + self.assertNotEqual(version.export_blob, {}) + + with self.subTest("Modify used product"): + product.name = "new product name" + product.price = 20 + product.save() + self.assertEqual(Product.objects.count(), 1) + + # Restore it + form.restore_old_version(version.uuid) + + # get all fresh DB records + form.refresh_from_db() + + # Because the product was updated after the version was made, a new product is + # created which represents the product state from before the version + self.assertEqual(Product.objects.count(), 2) + new_product = Product.objects.last() + self.assertEqual(form.product.uuid, new_product.uuid) + self.assertEqual(new_product.name, "old product name") + self.assertEqual(new_product.price, 10) + + def test_form_restore_creating_new_resources_when_existing_have_been_altered_with_slug_identifier( + self, + ): + # Make sure that we start without any wmts tile layers + MapTileLayer.objects.all().delete() + self.assertEqual(MapTileLayer.objects.count(), 0) + + wmts_tile_layer = MapTileLayerFactory.create( + identifier="test-wmts-identifier", + url="test-wmts-url.com", + label="test-wmts-label", + ) + + form = FormFactory.create( + generate_minimal_setup=True, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmts_tile_layer.identifier, + "overlays": [], + }, + ], + }, + ) + + version = FormVersionFactory.create(form=form) + self.assertNotEqual(version.export_blob, {}) + + with self.subTest("Modify used wmts tile layer"): + wmts_tile_layer.label = "new label" + wmts_tile_layer.url = "some-different-url.com" + wmts_tile_layer.save() + self.assertEqual(Product.objects.count(), 1) + + # Restore it + form.restore_old_version(version.uuid) + + # get all fresh DB records + form.refresh_from_db() + + # Because the wmts tile layer was updated after the version was made, a new wmts + # tile layer is created which represents the previous state. + self.assertEqual(MapTileLayer.objects.count(), 2) + new_wmts_tile_layer = MapTileLayer.objects.last() + map = form.formstep_set.first().form_definition.configuration["components"][0] + + self.assertEqual(map["tileLayerIdentifier"], new_wmts_tile_layer.identifier) + self.assertEqual(new_wmts_tile_layer.label, "test-wmts-label") + self.assertEqual(new_wmts_tile_layer.url, "test-wmts-url.com") + + def test_form_restore_using_resource_with_similar_data(self): + product1 = ProductFactory.create(name="product 1", price=1) + product2 = ProductFactory.create(name="product 2", price=2) + + form = FormFactory.create(generate_minimal_setup=True, product=product1) + + version = FormVersionFactory.create(form=form) + self.assertNotEqual(version.export_blob, {}) + + with self.subTest("Modify products"): + # Swap the product data + product1.name = "product 2" + product1.price = 2 + product1.save() + product2.name = "product 1" + product2.price = 1 + product2.save() + + # Restore the form + form.restore_old_version(version.uuid) + + # get all fresh DB records + form.refresh_from_db() + + # No new products were created + self.assertEqual(Product.objects.count(), 2) + # Product 2 is now used, as it has the same data as product 1 had when the form + # version was made. + self.assertEqual(form.product.uuid, product2.uuid) + + def test_form_restore_using_resource_with_similar_data_and_slug_identifier(self): + # Make sure that we start without any wmts tile layers + MapTileLayer.objects.all().delete() + self.assertEqual(MapTileLayer.objects.count(), 0) + + wmts_tile_layer1 = MapTileLayerFactory.create( + identifier="test-wmts-identifier", + url="test-wmts-url.com", + label="test-wmts-label", + ) + wmts_tile_layer2 = MapTileLayerFactory.create( + identifier="test-wmts-identifier2", + url="test-wmts-url2.com", + label="test-wmts-label2", + ) + + form = FormFactory.create( + generate_minimal_setup=True, + formstep__form_definition__configuration={ + "components": [ + { + "label": "Map", + "key": "map", + "type": "map", + "useConfigDefaultMapSettings": False, + "interactions": { + "marker": True, + "polygon": False, + "polyline": False, + }, + "tileLayerIdentifier": wmts_tile_layer1.identifier, + "overlays": [], + }, + ], + }, + ) + + version = FormVersionFactory.create(form=form) + self.assertNotEqual(version.export_blob, {}) + + with self.subTest("Modify wmts tile layers"): + # Swap the wmts tile layer data + wmts_tile_layer1.label = "test-wmts-label2" + wmts_tile_layer1.url = "test-wmts-url2.com" + wmts_tile_layer1.save() + wmts_tile_layer2.label = "test-wmts-label" + wmts_tile_layer2.url = "test-wmts-url.com" + wmts_tile_layer2.save() + + # Restore it + form.restore_old_version(version.uuid) + + # get all fresh DB records + form.refresh_from_db() + + # No new wmts tile layers should have been created + self.assertEqual(MapTileLayer.objects.count(), 2) + # wmts_tile_layer2 is now used, as it has the same data as wmts_tile_layer1 had + # when the form version was made. + map = form.formstep_set.first().form_definition.configuration["components"][0] + + self.assertEqual(map["tileLayerIdentifier"], wmts_tile_layer2.identifier) + + def test_form_restore_active_state_is_correctly_kept(self): + for initial_active_state in (True, False): + with self.subTest(data=initial_active_state): + form = FormFactory.create( + generate_minimal_setup=True, active=initial_active_state + ) + + version = FormVersionFactory.create(form=form) + self.assertNotEqual(version.export_blob, {}) + + # Change active state after version was made + form.active = not initial_active_state + form.save() + + # Restore it + form.restore_old_version(version.uuid) + + # get all fresh DB records + form.refresh_from_db() + + # Assert active state has not been changed + self.assertEqual(form.active, initial_active_state) + FORM_STEP = [ { diff --git a/src/openforms/forms/utils.py b/src/openforms/forms/utils.py deleted file mode 100644 index 9dc99c07df..0000000000 --- a/src/openforms/forms/utils.py +++ /dev/null @@ -1,449 +0,0 @@ -import json -import random -import string -import zipfile -from collections.abc import Collection -from typing import Any, Required, TypedDict -from uuid import uuid4 - -from django.conf import settings -from django.core.serializers.json import DjangoJSONEncoder -from django.db import transaction -from django.utils.translation import override - -import structlog -from rest_framework.exceptions import ValidationError -from rest_framework.test import APIRequestFactory - -from openforms.formio.migration_converters import CONVERTERS, DEFINITION_CONVERTERS -from openforms.formio.utils import iter_components -from openforms.forms.constants import FormTypeChoices -from openforms.registrations.contrib.objects_api.constants import ( - PLUGIN_IDENTIFIER as OBJECTS_API_PLUGIN_IDENTIFIER, -) -from openforms.registrations.contrib.stuf_zds.plugin import ( - PLUGIN_IDENTIFIER as STUF_ZDS_PLUGIN_IDENTIFIER, -) -from openforms.registrations.contrib.zgw_apis.plugin import ( - PLUGIN_IDENTIFIER as ZGW_APIS_PLUGIN_IDENTIFIER, -) -from openforms.typing import JSONObject - -from .api.datastructures import FormVariableWrapper -from .api.serializers import ( - FormDefinitionSerializer, - FormLogicSerializer, - FormSerializer, - FormStepSerializer, - FormVariableSerializer, -) -from .constants import LogicActionTypes -from .models import Form, FormDefinition, FormLogic, FormStep, FormVariable - -logger = structlog.stdlib.get_logger(__name__) - - -IMPORT_ORDER = { - "formDefinitions": FormDefinition, - "forms": Form, - "formSteps": FormStep, - "formVariables": FormVariable, - "formLogic": FormLogic, -} - -SERIALIZERS = { - "formDefinitions": FormDefinitionSerializer, - "forms": FormSerializer, - "formSteps": FormStepSerializer, - "formLogic": FormLogicSerializer, - "formVariables": FormVariableSerializer, -} - - -def _get_mock_request(): - factory = APIRequestFactory() - first_allowed_host = ( - settings.ALLOWED_HOSTS[0] if settings.ALLOWED_HOSTS else "testserver" - ) - server_name = first_allowed_host if first_allowed_host != "*" else "testserver" - request = factory.get("/", SERVER_NAME=server_name) - request.is_mock_request = True # pyright: ignore[reportAttributeAccessIssue] - return request - - -def to_json(obj: Any): - return json.dumps(obj, cls=DjangoJSONEncoder) - - -@transaction.atomic -def import_form(import_file, existing_form_instance=None) -> Form | None: - import_data = {} - with zipfile.ZipFile(import_file, "r") as zip_file: - for resource in IMPORT_ORDER.keys(): - if f"{resource}.json" in zip_file.namelist(): - import_data[resource] = zip_file.read(f"{resource}.json").decode() - - return import_form_data(import_data, existing_form_instance) - - -def check_form_definition(uuid: str, attrs: dict[str, Any], for_existing_form: bool): - """ - Import a form definition with a given UUID. - - If the UUID is already present, check if the configuration is the same or not. If - it's the same, the existing record is updated, otherwise a new form definition is - created. - """ - existing = FormDefinition.objects.filter(uuid=uuid).first() - # no existing record -> let the import flow create one - if existing is None: - return None - - # if there is an existing form definition, but it's not being related to the same - # form, then we need to create a copy - if not for_existing_form and existing.used_in.exists() and not existing.is_reusable: - return None - - # Compare hashes to check if form fields configuration changed or not. If there are - # changes, the import data should be created as a new record. - existing_fd_hash = existing.get_hash() - imported_fd_hash = FormDefinition(configuration=attrs["configuration"]).get_hash() - if existing_fd_hash == imported_fd_hash: - return existing - return None - - -@transaction.atomic -@override(language=settings.LANGUAGE_CODE) -def import_form_data( - import_data: dict, - existing_form_instance: Form | None = None, -) -> Form | None: - uuid_mapping = {} - - request = _get_mock_request() - - created_form = None - - # when restoring a previous version, delete the current form configuration, - # it will be replaced with the import data. - if existing_form_instance: - form_steps = FormStep.objects.filter(form=existing_form_instance) - # delete single-use form definitions, they're orphan nodes when deleting the steps - fd_ids = list( - FormDefinition.objects.filter( - is_reusable=False, formstep__in=form_steps - ).values_list("id", flat=True) - ) - form_steps.delete() - FormDefinition.objects.filter(id__in=fd_ids).delete() - FormLogic.objects.filter(form=existing_form_instance).delete() - FormVariable.objects.filter(form=existing_form_instance).delete() - - _form_definitions = [] - - for resource in IMPORT_ORDER.keys(): - if resource not in import_data: - continue - - data = import_data[resource] - for old, new in uuid_mapping.items(): - data = data.replace(old, new) - - try: - serializer = SERIALIZERS[resource] - except KeyError: - raise ValidationError(f"Unknown resource {resource}") - - for entry in json.loads(data): - if old_uuid := entry.get("uuid"): - entry["uuid"] = str(uuid4()) - - if resource == "forms": - # we can only extract a category UUID from the URL here, but that requires - # an exact match and we currently don't provide import/export functionality - # for categories. Relying on ID/Name is not much better than guesswork either, - # so we always import forms with NO category at all to prevent import errors. - # See #1774 for one such example of an error. - entry["category"] = None - # theme overrides cannot be imported, since the theme records/FKs have to - # exist in the target environment. Importing/exporting themes is also not - # possible at this time, so we reset the theme and admins need to update - # the imported form. - entry["theme"] = None - - # forms before v4.0 do not have the type field so in case we import an - # old appointment form we have to make sure that the form has the right - # type configured (by default is regular) - if appointment_options := entry.get("appointment_options"): - if appointment_options.get("is_appointment"): - entry["type"] = FormTypeChoices.appointment - - # check for file components in the form definitions and move - # registration options to the backend registration options - move_file_registration_options(entry, _form_definitions) - - if resource == "forms" and not existing_form_instance: - entry["active"] = False - - serializer_kwargs = { - "data": entry, - "context": { - "request": request, - "form": created_form, - "is_import": True, - }, - } - - if resource == "formDefinitions": - existing_form_definition_instance = check_form_definition( - old_uuid, - entry, - for_existing_form=existing_form_instance is not None, - ) - if existing_form_definition_instance: - # The form definition that is being imported is identical to - # the existing form definition with the same UUID, use - # existing instead of creating new definition. This may be - # both single and multiple use (is_reusable=True) form - # definitions, depending on whether it's for an existing form or not. - # Note that the mapping will include the same UUID here often, - # which is okay for find-and-replace. - serializer_kwargs["instance"] = existing_form_definition_instance - entry["uuid"] = old_uuid - uuid_mapping[old_uuid] = old_uuid - - if resource == "forms" and existing_form_instance: - serializer_kwargs["instance"] = existing_form_instance - - if resource in ("formVariables", "formLogic"): - # by now, the form resource has been created (or it was an existing one) - _form = existing_form_instance or created_form - serializer_kwargs["context"].update( - { - "forms": {str(_form.uuid): _form}, - "form_definitions": { - str(fd.uuid): fd - for fd in FormDefinition.objects.filter( - formstep__form=_form - ) - }, - } - ) - if "service_fetch_configuration" in entry: - # The transferring between systems case is very tricky - # better not import these, we don't know where this came from. - # services and ids may point to different things - # in different OF instances. - del entry["service_fetch_configuration"] - - if resource == "formLogic": - # by now, the form resource has been created (or it was an existing one) - _form = existing_form_instance or created_form - serializer_kwargs["context"].update( - { - "form_variables": FormVariableWrapper(_form), - "form_steps": { - form_step.uuid: form_step - for form_step in _form.formstep_set.all().order_by("order") - }, - } - ) - - deserialized = serializer(**serializer_kwargs) - - if resource == "formLogic" and "order" not in entry: - entry["order"] = 0 - - try: - is_create = ( - deserialized.instance is None or not deserialized.instance.pk - ) - deserialized.is_valid(raise_exception=True) - - if resource == "formDefinitions": - apply_component_conversions( - deserialized.validated_data["configuration"] - ) - - apply_definition_conversions( - deserialized.validated_data["configuration"] - ) - - if resource == "formLogic": - clear_old_service_fetch_config(deserialized.validated_data) - - instance = deserialized.save() - if resource == "forms": - created_form = deserialized.instance - if resource == "formSteps": - # Once the form steps have been created, we create the component FormVariables - # based on the form definition configurations. - FormVariable.objects.create_for_form(created_form) - if resource == "formDefinitions": - _form_definitions.append(instance) - if resource == "formDefinitions" and is_create: - uuid_mapping[old_uuid] = str(instance.uuid) - - # The FormSerializer/FormStepSerializer/FormLogicSerializer have the uuid as a read only field. - # So the mapping between the old uuid and the new needs to be done after the instance is saved. - if hasattr(deserialized.instance, "uuid") and "uuid" in entry: - uuid_mapping[old_uuid] = str(deserialized.instance.uuid) - except ValidationError as e: - if ( - resource == "forms" - and "slug" in e.detail - and e.detail["slug"][0].code == "unique" - ): - entry["slug"] = ( - f"{entry['slug']}-{''.join(random.choices(string.hexdigits, k=6))}" - ) - - deserialized = serializer( - data=entry, - context={ - "request": request, - "form": created_form, - "is_import": True, - }, - instance=existing_form_instance, - ) - deserialized.is_valid(raise_exception=True) - deserialized.save() - created_form = deserialized.instance - uuid_mapping[old_uuid] = str(deserialized.instance.uuid) - - else: - raise e - - return created_form - - -def apply_component_conversions(configuration): - """ - Apply the known formio component conversions to the entire form definition. - """ - log = logger.bind(action="forms.apply_component_conversions") - for component in iter_components(configuration): - if not (component_type := component.get("type")): # pragma: no cover - continue - if not (converters := CONVERTERS.get(component_type)): - continue - for identifier, apply_converter in converters.items(): - log.debug( - "apply_converter", component_type=component_type, identifier=identifier - ) - apply_converter(component) - - -def apply_definition_conversions(configuration: JSONObject) -> None: - for converter in DEFINITION_CONVERTERS: - converter(configuration) - - -def remove_key_from_dict(dictionary, key): - for dict_key in list(dictionary.keys()): - if key == dict_key: - del dictionary[key] - elif isinstance(dictionary[dict_key], dict): - remove_key_from_dict(dictionary[dict_key], key) - elif isinstance(dictionary[dict_key], list): - for value in dictionary[dict_key]: - if isinstance(value, dict): - remove_key_from_dict(value, key) - - -def clear_old_service_fetch_config(rule: dict) -> None: - for action in rule["actions"]: - if action["action"]["type"] != LogicActionTypes.fetch_from_service: - continue - - if "value" not in action["action"] or action["action"]["value"] == "": - continue - - # See comment above in `import_form_data` where we check if the variable has a - # `service_fetch_configuration` attribute. - # We can't reliably relate the service fetch configured to an existing configuration. - # So we don't add any existing service fetch config to the variables - action["action"]["value"] = "" - - -class FileComponentOptions(TypedDict, total=False): - key: Required[str] - document_type_description: str - organization_rsin: str - confidentiality_level: str - title: str - - -def move_file_registration_options( - form_data: dict, form_definitions: Collection[FormDefinition] -): - relevant_backends = [ - backend - for backend in form_data.get("registration_backends", []) - if backend.get("backend") - in ( - OBJECTS_API_PLUGIN_IDENTIFIER, - STUF_ZDS_PLUGIN_IDENTIFIER, - ZGW_APIS_PLUGIN_IDENTIFIER, - ) - ] - if not relevant_backends: - return - - # collect all file components, including the ones inside edit grids - file_component_options: dict[str, FileComponentOptions] = {} - for fd in form_definitions: - for component in fd.configuration_wrapper: - if component["type"] != "file": - continue - if not (registration := component.get("registration")): - continue - opts: FileComponentOptions = {"key": component["key"]} - - # NOTE: we ignore the catalogue information - the backend-level catalogue - # option is used and this is validate at the serializer level - document_type_description = (registration.get("documentType") or {}).get( - "description" - ) - organization_rsin = registration.get("bronorganisatie") - confidentiality_level = registration.get("docVertrouwelijkheidaanduiding") - title = registration.get("titel") - - if document_type_description: - opts["document_type_description"] = document_type_description - if organization_rsin: - opts["organization_rsin"] = organization_rsin - if confidentiality_level: - opts["confidentiality_level"] = confidentiality_level - if title: - opts["title"] = title - - if len(opts.keys()) != 1: - file_component_options[component["key"]] = opts - - if not file_component_options: - return - - files = list(file_component_options.values()) - - def _file_for_stuf_zds(opts: FileComponentOptions): - if title := opts.get("title"): - return {"key": opts["key"], "title": title} - return None - - files_for_stuf_zds = [o for opts in files if (o := _file_for_stuf_zds(opts))] - - for backend in relevant_backends: - options = backend.get("options") or {} - if "files" in options: - continue - - plugin_id = backend.get("backend") - if plugin_id in (OBJECTS_API_PLUGIN_IDENTIFIER, ZGW_APIS_PLUGIN_IDENTIFIER): - options["files"] = files - elif plugin_id == STUF_ZDS_PLUGIN_IDENTIFIER: - options["files"] = files_for_stuf_zds - else: # pragma: no cover - raise ValueError(f"Unknown registration plugin '{plugin_id}'.")