8.6.0 Performance and bugfix backports

This unscheduled TYPO3-v8 release is an exception.

In this release I have backported the latest performance features from the current version, as well as a few bug fixes.

9.5.0 Performance, sorting publishing, cleanups and fixes

🚀 🚀 🚀

I've got something great for you. Really! It's this release. As you might have guessed from the new version number, this is a feature release and those features are damn yummi 😋

But first, let's see which nags have been addressed.

🐛 bugs zapped:

• Fixed the parent record association of records fetched as translation
• Use = instead of LIKE for constraints only containing Numerals (#84 thanks to @friggingee)
• Use already existing records when searching for records by a single identifier
• Removed some loggers which did not log anything useful
• Only add fields to sys_redirects when EXT:redirects is active
• Do not try to process files which do not exist (Root cause still to be fixed 😑)

✨ Feature time!

Now for those nice features i promised you. To be honest, the first feature is hidden inside the second bug fix, which alters the queries depending on the values used in the constraints. Simply using a = instead of LIKE can speed up your application because the database does not have to convert a whole table column to string first and neglect the index. I've measured between -3% and -44% execution time for the Publish Overview Module. This is simply awesome!

Next up: "Respect rootLevel, allowedTables and skip empty tables or missing PIDs"
Oh yeah, this is party a bug fix, partly a feature. Developers know, that TYPO3 has different mechanism which allow or deny specific kinds of records to be created and stored on specific pages or places in the page tree. The content publisher did not adhere to these rules when searching for records stored on pages, which resulted in a vast number of queries which could have been avoided easily. These rules are known as the rootLevel restriction and allowedTables depending on page types.
Additionally, the content publisher will peek into tables which are queried. With the information gathered by peeking into the table, the content publisher can now decide to skip further queries to that table because it is either empty or does not contain a special value (the searched PID). This feature reduces even more superfluous queries, which would not yield any result.

The last big change is called "publishSorting". This feature will watch the publishing process and detect changes in the order of published pages and records. If a change was detected, the publish sorting feature will publish the sorting of all records which are affected by this sorting change (but no other property of said records). This will help to maintain the order of records without having to manually publish all records affected by that sorting change.

9.4.0 Redirects Support

Hey there. I am proud to announce a new Content Publisher feature release.
We have developed a new feature and included several bug fixes. See the list below:

Features

Redirects Support

A new optional extension EXT:redirects has been created for TYPO3 v9 that allows editors to create redirects.
In TYPO3 v10, this extension has been further integrated into the core by e.g. automatically creating redirects when the slug of a page changes.
Since editors expect the Foreign System to behave like Local, these redirects must also be published.
The difficulty here is that redirects are usually restricted to one domain, which is different on the remote system. To resolve this difference, the content publisher changes the domain to match the domain on the remote system when publishing the redirect.

It is also possible to publish single redirects with the new Redirects Module.

The documentation will tell you more about it: https://github.com/in2code-de/in2publish_core/blob/develop/Documentation/Features/RedirectsSupport.md

Deletion state changes

Records in the Publish Overview Module which are deleted did not show any changed record properties. We changed this, so that editors at least see the change of the deleted flag.

Important bug fixes

• Esure to publish the default language version of a translated page (for SRP [EE], Workflows [EE] and Publish Overview)
• Share the RecordFactory's runtime cache (this might increase the performance of the Publish Overview Module in some cases)
• Added a new test to the Publish Tools Module Tests to assert that the site configuration contains identical language configurations.

Bug fixes

• The attached extension ZIP will not contain the Build folder anymore
• Assert that the table sys_domain (only TYPO3 v9) exists before querying it
• Add PID detection for new records
• Add missing colored rectangle if the Overview Module to reduce layout shifts for records in a certain state
• Do not throw an exception if the rootline of a page can not be found if the page is disconnected from the page tree (The parent page does no exist)
• Ensure that string function arguments have the correct type (in replaceGeneralMarkers)
• Skip redirects support if EXT:redirects is not installed
• Don't try to create URLs to deleted pages (TYPO3 can't do that)
• Never relate sys:records to pages based on their PID
• Added missing unique exception code
• Provide missing icons in TYPO3 v9
• Allow deleted redirects to be published

9.3.1 Three little bugsies

sys_file pointer fallback
In 5 years of publishing content it never occurred, that a sys_file_processedfile record did not have a valid sys_file reference on local. What an ultra-edge case! Anyway, the foreign's record identifier will be used instead.

treatRemovedAndDeletedAsDifference only for deleted
Check carefully if a foreign record is deleted and the local one does not exist. There have been cases where records have been incorrectly marked as deleted-published-removed.

TCA type user
This field is not supported for relation resolving. And it should not be considered as capable of this.

9.3.0 Even more features

41 files changed, 858 insertions(+), 79 deletions(-)

Holy cow, a lot has been going on here. Let me break this down for you.
There are a total of 3 new features and 6 bug fixes.

The features are...

Publishing a page via page tree context menu

The page tree context menu (which opens on right click or left click on the page icon) can now be used to publish a page.
Since the menu entry can not know about the actual record state (new, changed, deleted, ...) it will always be active. Please check the known issue: https://github.com/in2code-de/in2publish_core/blob/develop/Documentation/KnownIssues.md#context-menu-publishing
Since this feature is not perfect, it is disabled by default. You can activate it by setting features.contextMenuPublishEntry.enable = TRUE.

Caches of pages which contain a file will be cleared when the file is published

Publishing a file via the Publish Files module has in the past only transferred the file and the corresponding database entries. Now, after publishing the file, every cache is cleared from every page that uses this file directly.

You can now publish a page which was deleted and has been removed finally via the recycler

Some people are using the recycler extension to clean up their TYPO3 databases manually. However, this could not be published since the content publisher does not differenciate between records that are marked as deleted and those, which are actually deleted from the database.
Enable factory.treatRemovedAndDeletedAsDifference to tell the Content Publisher, that you want to see and publish these records.
The new record state which was introduced for this state is black.
BTW: Records which are deleted from the database do not need a workflow state, because they will not change any visible content.

Let's talk about (some) bug fixes

SQL escape sequences in FlexForms

TYPO3 allows the usage of {#tx_mytable} to quote a table and column name in TCA. This feature was supported for normal TCA, but not for FlexForms, which are a kind of dynamic TCA.

Cap the severity level of performance tests to WARNING

Several users have reported problems with performance test results. In fact, some are using these tests on the command line in their CI/CD environment. Unfortunately, the results vary widely on these systems, so the pipelines broke and the applications were not deployed.
Since the performance tests don't actually check if the whole thing works, but how fast the environment responds to their commands, there is no sane reason to let them fail.

Merge duplicate File Entries in the Publish Files Module

An uploaded file with the same identifier as a file, which did exist in that folder, led to two entries in the Publish Files Module with the same file name. That's bad UX, despite being technically correct. So here's the fix: We hide all further rows with the same file name behind the first one. Publishing that one will publish all of these hidden records, too, as well as the file itself. Hooray!

Remaining bug fixes:

• Using DBAL count instead of SQL count
• Prevent runinng into doctrine/dbal regression by skipping empty addition where clauses
• Do not generate preview URLs for Page ID 0

9.2.0 env vars, internal_type file_reference and config debug

3️⃣ 🥳 Triple Feature Release

TCA type group internal_type file_reference

The first feature is also a community feature. @pixelbrackets contributed the changes, required for the Content Publisher to support the file_reference type of TCA group relations (which are used by flux, for example). Thank you very much for contributing and testing, @pixelbrackets 🥰

Env vars in the configuration

tl;dr: You can use %env(MY_ENV_VAR)% anywhere in your configuration to reference environment variables.

Another feature which roots in the thriving Content Publisher Community is the support for Environment Variables in the configuration. We decided to give that Feature Request a bit of an upgrade and implemented Configuration Post Processors as a new ConfigContainer feature, which allows you to write your own custom features for the configuration. Implementing your own PostProcessor is as easy as registering your class and implementing an interface. Guides/ConfigurationPostProcessing.md will tell you everything you need to know.

The first implemented config post processor is the DynamicValuesPostProcessor including the EnvVarProvider, which allows you to use symfony-like syntax anywhere in your configuration (not just the yaml file, but also in PageTS, UserTS and the ManagedConfiguration from the Enterprise Edition).
The DynamicValuesPostProcessor is also extensible with more Providers, which can read custom data sources. This feature is documented in Guides/DynamicConfiguration.md

Better debugging options

The Tools module is the biggest support during setup and troubleshooting. ANd it got even better.

• The configuration inspection section got new dumps which show your current ConfigContainer setup.
• You can inspect the actual values returned by each ConfigProvider
• You can enter a Page ID to inspect the configuration for the given page (Especially useful when having conditions or debugging the PageTS configuration)

Other Fixes

• Prevent exceptions during TYPO3 bootstrapping
• Allow empty database passwords (please don't use that, except for local development)
• Fixed return URLs of "edit record links" rendered in any content publisher module

As always, stay safe and healty!

9.1.0 Translated record handling and support info

Features:

Support Info

• Information where to get support is displayed in the tools module
• End-user support information can be added by the system maintainer

Translation Handling (not a big thing)

• Translated records are shown with the sys_language flag.
• Translated records will always be directly below the original (they have been displayed at the end until now)
• Translated records are not limited by excludeRelatedTables (but their translation parents are, so these settings still are in effect)
  • Fixes a bug where page translation were not published.

Bug fixes

• Decoupled the app from the internal implementation of ArrayObject
• Use language parameter instead of localized page ID to build URLs (for correct cHash)
• Prevent PageTS caching before ext_tables and the TCA is loaded (flux support)
• Allow FlexForm config arrays without TCEforms index (flux support)
• Inherit the correct FQCN from the changed EXT:logs controller (logs integration)
• Do not attempt to modify preview URLs of files which already are full qualified (non-public FAL storages)
• Run tasks after publishing files and folders

9.0.2 fixed RCE option type

A small type error sneaked through the release tests. Fixed it right away!

9.0.1 Stability improvements

Bugfixes:

• Ensure Envelope UIDs are always transported as integer
• Fix variable type problem with string functions in Record
• Added missing type hints
• Fix page-to-page relation filter when adding related records

"Under-the-radar-feature":

• Permformance Tests in the Backend
  • These Tests will read/write some data and database entries as well as initialize some required drivers for the publishing. These Tests are not required to be green, but expect decreased overall performance, when they are not.

9.0.0 TYPO3 v10 compatibility

Almost exactly at the time of the TYPO3 v10 release, we proudly publish in2publish_core 9.0, which is now compatible with TYPO3 v10.
Most changes in this release increase the stability of the product across the supported TYPO3 versions.

Some changes i want to highlight:

• Thx to @YKWeyer for his contribution which adds a new voting signal that can be used to ignore record relations defined in FlexForms
• Record l10nParents will be included in relations, even if their table is excluded.
• Full rework of the URI handling. The content publisher can now build URLs for the foreign system based on the sites configuration, even for record previews defined in PageTS
  • Also included: Support for TYPO3 sites with an URL path (e.g. https://example.com/content/)
• "Cloud"-Mode option: You can disable the foreign key fingerprint check (e.g. if you are hosting in a cloud/docker environment)

Stats: 297 files changed, 4750 insertions(+), 3751 deletions(-)