Error while using FilesystemIterator

Created on 22 January 2025, 8 days ago

Problem/Motivation

While trying to use this module with Feeds module and the DirectoryFetcher I noticed that there is a scenario when S3fsStream fails with low level method.

Specifically, when FilesystemIterator is used to wrap a stream, when the iterator is passed to a foreach statement the rewind method invokes S3fsStream::dir_opendir passing null as uri.

Steps to reproduce

$iterator = new \FilesystemIterator('public://path/to/bucket/dir');

$iterator->rewind();

Proposed resolution

return False if uri is null.

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Active

Version

3.7

Component

Code

Created by

🇮🇹Italy Bladedu Italy

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @Bladedu
  • Merge request !71Issue #3501484 → (Open) created by Bladedu
  • Pipeline finished with Failed
    8 days ago
    Total: 549s
    #402988
  • Pipeline finished with Failed
    8 days ago
    Total: 966s
    #402987
  • 🇺🇸United States cmlara

    How is $uri null here? The relevant specs appear to only permit a string value. PHPStan alerts similar with warning "Call to function is_null() with string will always evaluate to false."

    This implies there is a bug somewhere else in the stack that needs to be identified and resolved, setting back to needs work to identify the root fault and for eventual tests to be included with the fix.

    @see https://www.php.net/manual/en/streamwrapper.dir-opendir.php
    @see Drupal\Core\StreamWrapper\PhpStreamWrapperInterface

  • 🇮🇹Italy Bladedu Italy

    It might be true the bug is somewhere else (although after few hours I couldn't find it), but have a look at core dir_opendir method signature here: https://git.drupalcode.org/project/drupal/-/blob/11.x/core/lib/Drupal/Co...
    Although the comment says it's going to be a string, the method signature does not have the type enforcing.

    Any hint on debugging this would be helpful

  • 🇺🇸United States cmlara

    Any hint on debugging this would be helpful

    I would suggest starting with xdebug and a conditional breakpoint for when $uri is null.

    I suspect we will find this is likely a child call where the stack will show a parent call somewhere else in the streamWrapper code is passing NULL. I am not expecting this will be an issue in PHP proper, although if the stack trace stops with just the dir_opendir() call that could be a possibility.

  • 🇺🇸United States cmlara

    Had a moments to run the steps to reproduce through Drush:

    $ ddev drush --xdebug php
    Psy Shell v0.12.7 (PHP 8.3.14 — cli) by Justin Hileman
    s3fs test site (Drupal 11.1.0)
    > $iterator = new \FilesystemIterator('s3://');
    = FilesystemIterator {#8904
        path: "s3:/",
        filename: "2024-12",
        basename: "2024-12",
        pathname: "s3://2024-12",
        extension: "",
        realPath: false,
        aTime: 1969-12-31 16:00:00,
        mTime: 1969-12-31 16:00:00,
        cTime: 1969-12-31 16:00:00,
        inode: 0,
        size: 0,
        perms: 040777,
        owner: 0,
        group: 0,
        type: "dir",
        writable: true,
        readable: true,
        executable: true,
        file: false,
        dir: true,
        link: false,
      }
    
    > $iterator->rewind()
    
       TypeError  Drupal\s3fs\StreamWrapper\S3fsStream::resolvePath(): Argument #1 ($path) must be of type string, null given, called in modules/custom/s3fs/src/StreamWrapper/S3fsStream.php on line 1123.
    

    Here is the relevant portion of the stack trace:

    S3fsStream.php:1123, Drupal\s3fs\StreamWrapper\S3fsStream->dir_opendir()
    StreamWrapper.php:477, Aws\S3\StreamWrapper->Aws\S3\{closure:/var/www/html/vendor/aws/aws-sdk-php/src/S3/ StreamWrapper.php:475-479}()
    StreamWrapper.php:941, Aws\S3\StreamWrapper->boolCall()
    StreamWrapper.php:475, Aws\S3\StreamWrapper->dir_rewinddir()
    35:1, FilesystemIterator->rewind()
    

    The parent AWS's code is passing $this->openedPath as null. We intentional don't call parent::dir_opendir() in our dir_opendir() where it would be set. I don't believe we would want to use parent::opendir() as we depend upon the database. This is a private variable so we can't just set it in our dir_opendir().

    Above stack should be enough to determine a proper solution.

    Cursory thoughts:
    We likely need to implement dir_rewinddir() ourselves.

Production build 0.71.5 2024