[Regression] Clean up temporary files created by XML data parser plugin

Open on Drupal.org →

Created on 7 November 2024, 7 months ago

Problem/Motivation

This is a regression that has been introduced in ✨ XML parser ignores the data-fetcher plugin Needs work .

The XML data parser plugin gets the XML contents from the data fetcher plugin and then writes these contents to a temporary file, which is then used by the XML reader. The temporary file is created using \Drupal\Core\File\FileSystem::tempnam(), which in turn uses PHP's tempnam(). The temporary files are never deleted by the plugin, resulting in a lot of stray files in temporary directories.

Note: Unlike files created using tmpfile(), which are deleted by PHP automatically at some point, files created using tempnam() are not and have to be deleted by the implementing code.

Steps to reproduce

Set up a migration using the XML data parser plugin.
Verify that your temporary directory contains no files starting with file.
Run the migration, e.g. using drush mim.
Wait until the migration is complete.
Verify that your temporary directory contains files starting with file that contain the contents of the XML file used for import. This should be at least one file, but may be more, e.g. if you performed a rollback first.

Sample source config using http data fetcher, file data fetcher works as well:

source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: xml
  urls: 'https://updates.drupal.org/release-history/drupal/current'
  item_selector: '/project/releases/release'

Proposed resolution

Delete the file on destruction of the plugin. Or maybe there is a better way using the API, although I didn't see it?

Remaining tasks

User interface changes

None.

API changes

None.

Data model changes

None.

🐛 Bug report

Status

Active

Version

6.0

Component

Plugins

Created by

🇩🇪Germany FeyP

Live updates comments and jobs are added and updated live.

Sign in to follow issues

Merge Requests

!124[Regression] Clean up temporary files created by XML data parser plugin
Open
🇩🇪Germany stefan.korn
updated 5 days ago
!114[Regression] Clean up temporary files created by XML data parser plugin
Open
🇩🇪Germany FeyP
updated 7 months ago

Comments & Activities

Issue created by @FeyP
Merge request !114Issue #3486188: [Regression] Clean up temporary files created by XML data parser plugin → (Open) created by FeyP
Comment 7 months ago →
🇩🇪Germany FeyP
Pushed a quick and dirty fix where the file is deleted in a destructor to the MR.
Pipeline finished with Success
7 months ago
#332198
Issue was unassigned.
Status changed to Needs review 5 days ago11:19am 26 May 2025
Comment 5 days ago →
🇩🇪Germany stefan.korn Jossgrund
One has to say, that this only happens when drupal's temporary directory is different from systems temporary directory (/tmp in most cases).

That is probably a reason why not all users are seeing this problem. But nonetheless this is a serious bug. If you heavily use the XML Plugin it can fill up the disk space.
Merge request !124Issue #3486188: [Regression] Clean up temporary files created by XML data... → (Open) created by stefan.korn
Pipeline finished with Success
5 days ago
Total: 276s
#506784
Comment 5 days ago →
🇩🇪Germany stefan.korn Jossgrund
had some issues with the __destruct variant in a scenario where we use some custom code to start a migration via backend.

So providing an alternative solution without __destruct but inheriting nextSource() function to unlink the temp file there.

I do not say that this solutions is better than the other one, just in our custom setup it works better.

So leaving decision up to maintainers how to fix this.

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024