Diffs with different line endings leads to Invalid $mode 3 specified

Created on 26 September 2023, about 1 year ago
Updated 25 July 2024, 5 months ago

Problem/Motivation

Diffs with different line endings lead to InvalidArgumentException : Invalid $mode 3 specified

This is because \Drupal\Component\Diff\DiffOpOutputBuilder::hunkOp does not handle \SebastianBergmann\Diff\Differ::DIFF_LINE_END_WARNING

Steps to reproduce

See unit test in MR.

Can be reproduced in diff module as per this test

Proposed resolution

Ignore line endings or handle them gracefully.

Remaining tasks

Check how line endings were handed in 10.0 and below.

User interface changes

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Needs work

Version

11.0 🔥

Component
Base 

Last updated 1 day ago

Created by

🇦🇺Australia mstrelan

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @mstrelan
  • last update about 1 year ago
    30,342 pass, 2 fail
  • @mstrelan opened merge request.
  • Status changed to Needs work about 1 year ago
  • last update about 1 year ago
    30,353 pass, 2 fail
  • Status changed to Needs review about 1 year ago
  • last update about 1 year ago
    30,365 pass
  • Status changed to RTBC about 1 year ago
  • 🇦🇺Australia acbramley

    Simple fix, correctly fixes the failed test and fixes our issues on a client project.

  • last update about 1 year ago
    30,367 pass
  • last update about 1 year ago
    30,362 pass
  • last update about 1 year ago
    30,363 pass
  • Status changed to Needs review about 1 year ago
  • 🇦🇺Australia larowlan 🇦🇺🏝.au GMT+10

    Left a question on the MR

  • Status changed to RTBC about 1 year ago
  • 🇺🇸United States smustgrave

    Restoring status.

  • Status changed to Needs work about 1 year ago
  • 🇮🇹Italy mondrake 🇮🇹

    Thanks for working on this. It seems my initial implementation fell short on a number of things. Sorry for that.

    The good thing is that we are adding test coverage for cases that were totally untested in the prior implementation - a more reliable pattern against possible future regressions.

    In a couple of similar issues earlier, 🐛 DiffOpOutputBuilder does not correctly match the 10.0.x implementation and produces unexpected output Fixed and 🐛 DiffOpOutputBuilder does not correctly match the 10.0.x implementation and produces unexpected output (part 2) Needs review , @alexpott also added the test case to the @legacy test for DiffEngine, so to show that the change is consistent with earlier behavior. I think it should be done here, too. NW for that.

  • last update about 1 year ago
    30,374 pass
  • Status changed to Needs review about 1 year ago
  • 🇦🇺Australia mstrelan

    Added test coverage to DiffEngineTest

  • last update about 1 year ago
    30,374 pass
  • Status changed to Needs work about 1 year ago
  • 🇮🇹Italy mondrake 🇮🇹

    OK, so now we have the regression fixed.

    Since this is making a 'change' out of two exactly same lines that only differ by the EOL (being UNIX-like or WINDOWS-like), I was wondering: this is technically correct, but for a human it would make no difference. So I checked usage of Diff, and I find one only in core, where the strings are certainly stripped of the '\n' (but I am unsure about the '\r'), BEFORE they are sent for diffing. So this one might be an edge case. I think that would deserve some comments somewhere re. how the array of strings passed to Diff should be preprocessed for EOL markings.

    From ConfigManager:

      /**
       * {@inheritdoc}
       */
      public function diff(StorageInterface $source_storage, StorageInterface $target_storage, $source_name, $target_name = NULL, $collection = StorageInterface::DEFAULT_COLLECTION) {
        if ($collection != StorageInterface::DEFAULT_COLLECTION) {
          $source_storage = $source_storage->createCollection($collection);
          $target_storage = $target_storage->createCollection($collection);
        }
        if (!isset($target_name)) {
          $target_name = $source_name;
        }
        // The output should show configuration object differences formatted as YAML.
        // But the configuration is not necessarily stored in files. Therefore, they
        // need to be read and parsed, and lastly, dumped into YAML strings.
        $source_data = explode("\n", Yaml::encode($source_storage->read($source_name)));
        $target_data = explode("\n", Yaml::encode($target_storage->read($target_name)));
    
        // Check for new or removed files.
        if ($source_data === ['false']) {
          // Added file.
          // Cast the result of t() to a string, as the diff engine doesn't know
          // about objects.
          $source_data = [(string) $this->t('File added')];
        }
        if ($target_data === ['false']) {
          // Deleted file.
          // Cast the result of t() to a string, as the diff engine doesn't know
          // about objects.
          $target_data = [(string) $this->t('File removed')];
        }
    
        return new Diff($source_data, $target_data);
      }
    
    

    Also, I think additional test cases should be provided for DiffFormatterTest, that is currently only testing array of strings free from any EOL markings.

  • 🇺🇸United States moshe weitzman Boston, MA

    It seems like the additional tests are not coming quickly. Would maintainers consider fixing the bug without tests? Otherwise we are forcing everyone to live with a bug because we are worried that the bug might return one day.

  • 🇺🇸United States micahw156

    For what it's worth, we've been running the patch from #3 above in production for nine months with no ill effects.

Production build 0.71.5 2024