Multiline YAML syntax is buggy with single newline character

Created on 19 October 2021, almost 3 years ago
Updated 3 July 2024, 2 months ago

Problem/Motivation

Following ✨ Export configuration YAML strings as multiline Fixed , multiline syntax is used for yml that contains newline characters. This has caused a problem with a certain migration yml that describes doing a find-replace involving newlines. After a few configuration imports and exports, "\n" gets transformed into ''. The configuration for the process plugin starts out like this:

  plugin: str_replace
  source: post_content
  regex: TRUE
  search: "/(\r\n)+/"
  replace: "\n"

Some time later (apparently after any other configuration is changed), this will get exported using the multiline syntax:

  plugin: str_replace
  source: post_content
  regex: TRUE
  search: "/(\r\n)+/"
  replace: |

Then some time after that, the config will get exported like this:

  plugin: str_replace
  source: post_content
  regex: TRUE
  search: "/(\r\n)+/"
  replace: ''

which is very different from what we started with.

Steps to Reproduce

The bug takes two "round trips" of encoding and decoding to reveal itself. Here are some commands that can be run in drush.

drush ev 'print json_encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(["foo" => "\n"])))))'

Expected output is {"foo":"\n"}
Actual output: {"foo":""}

Compare this to adding bar next to the newline, which works fine.

drush ev 'print json_encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(["foo" => "bar\n"])))))'

Expected output is {"foo":"bar\n"}
Actual output: {"foo":"bar\n"}

What happens with other strings that are composed only of newlines?

drush ev 'print json_encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(["foo" => "\n\n"])))))'

Expected output is {"foo":"\n\n"}
Actual output: {"foo":""}

drush ev 'print json_encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(["foo" => "\n\n\n"])))))'

Expected output is {"foo":"\n\n\n"}
Actual output: {"foo":"\n"}

drush ev 'print json_encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(\Drupal\Component\Serialization\YamlSymfony::decode(\Drupal\Component\Serialization\YamlSymfony::encode(["foo" => "\n\n\n\n"])))))'

Expected output is {"foo":"\n\n\n\n"}
Actual output: {"foo":"\n\n"}

Proposed resolution

TBD

Remaining tasks

TBD

User interface changes

None

API changes

TBD

Data model changes

TBD

Release notes snippet

TBD

πŸ› Bug report
Status

Needs work

Version

11.0 πŸ”₯

Component
ConfigurationΒ  β†’

Last updated 1 day ago

Created by

πŸ‡ΊπŸ‡ΈUnited States danflanagan8 St. Louis, US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡©πŸ‡°Denmark ressa Copenhagen

    Thanks for creating this issue @danflanagan8. I found it because you commented in the Migrate Plus issue πŸ› str_replace does not support strings with \n Closed: won't fix .

    I found it while looking for hints to my issue πŸ› Preserve line breaks, to support importing Markdown format Active , where I am losing line breaks during a migration. Perhaps it is related to the YAML challenges outlined here?

    I had hoped that a workaround could be to add an extra line break, something like this:

        plugin: str_replace
        regex: true
        source: body
        search: /(\n)/
        replace: $1$1
    

    But it looks like the source value is put on a single line, before processing by Migrate:

    $ drush migrate:import sub_nodes --update --migrate-debug
    1^ "---------------------------------------------------------------------"
    2^ "|                             $Source                               |"
    3^ "---------------------------------------------------------------------"
    4^ array:6 [
      "nid" => 7086
      "title" => "How to use Drupal"
      "body" => """
        You can use Drupal for web site building. Here are some benefits (this list works):\n
        - Open source\n
        - Big community\n
        - A third reason\n
        Here are some benefits (end up on the same line ...):\n
        - Open source - Big community - A third reason\n
    [...]
    

    A bit of the resulting source, with double line breaks via str_replace, but only for some lines, sadly not the list items:

         Here are some benefits (end up on the same line ...):\n
          \n
          - Open source - Big community - A third reason\n
          \n
    
  • πŸ‡©πŸ‡°Denmark ressa Copenhagen

    Oops, restoring referenced issue.

Production build 0.71.5 2024