Replace custom scripting with Twig

Problem/Motivation

Mail Edit has a simple but custom scripting language built in for providing conditionals and loops.

A sample of the code for parsing it looks like this:

for ($n = 0; $n < 5 && $result = preg_match_all('/{{(?P<condition>[^?#@]+?)(?P<operator>\?|#|@)(?P<text>(({{(({{(({{(({{((.|\n)*?)}}|.|\n)*?)}}|.|\n)*?)}}|.|\n)*?)}}|.|\n)*?))}}/', $template, $clauses); ++$n) {
  foreach ($clauses[0] as $i => $clause) {
  $replacement = '{{SYNTAX_ERROR}}';
  if ($clauses['operator'][$i] == '?') {
  if (!preg_match('/^(?P<true>(({{((({{((({{((({{((.|\n)*?)}})|.|\n)*?)}})|.|\n)*?)}})|.|\n)*?)}})|(\[[^]]*?\])|[^:[])*):(?P<false>(.|\n)*)$/', $clauses['text'][$i], $replacements)) {
    $template = str_replace($clause, $replacement, $template);
    continue;
    }
    if (preg_match('/^(?P<operand_1>.*?)\s*(?P<operator>==|!=|\<\>|\>=|\<=|\>|\<)\s*(?P<operand_2>.*?)$/', $clauses['condition'][$i], $condition)) {

The rest of the function can be found here: https://git.drupalcode.org/project/mail_edit/-/blob/7.x-1.x/mail_edit.al...

This is currently working code used by many sites in Drupal 7. However, I am concerned that it is difficult to maintain, difficult to debug, and especially difficult to extend / add new functionality.

Proposed resolution

Drupal 8 and 9 ship with a templating system built in - Twig. There is prior art of Twig being used in the UI:

Drupal core allows the use of Twig in Views: https://git.drupalcode.org/project/drupal/-/blob/9.2.x/core/modules/view...

When editing the settings for a field's output:

Rewrite results

Override the output of this field with custom text

The text to display for this field. You may enter data from this view as per the "Replacement patterns" below. You may include Twig or the following allowed HTML tags: <a> <abbr> <acronym> <address> <article> <aside> <b> <bdi> <bdo> <big> <blockquote> <br> <caption> <cite> <code> <col> <colgroup> <command> <dd> <del> <details> <dfn> <div> <dl> <dt> <em> <figcaption> <figure> <footer> <h1> <h2> <h3> <h4> <h5> <h6> <header> <hgroup> <hr> <i> <img> <ins> <kbd> <li> <mark> <menu> <meter> <nav> <ol> <output> <p> <pre> <progress> <q> <rp> <rt> <ruby> <s> <samp> <section> <small> <span> <strong> <sub> <summary> <sup> <table> <tbody> <td> <tfoot> <th> <thead> <time> <tr> <tt> <u> <ul> <var> <wbr>

Emphasis is mine. I tested it, and the if conditional in Twig works.

Webform allows the use of Twig in mail templates: https://git.drupalcode.org/project/webform/-/blob/6.x/src/Twig/WebformTw...

I did some testing, and it looks like Tokens and Twig can be used side by side, so that shouldn't be a blocker.

The actual question is: should we port the existing scripting language, or migrate to Twig?

Remaining tasks

Add a "Twig 101" block similar to the "Dynamic Text Information" block from Drupal 7 on the mail template form, listing both common Twig features (if, for, etc.) and available variables. The same variables can be passed to both Twig and Tokens.

Add Twig validation to the mail template forms.

Add Twig rendering just before or just after token replacement.

Possibly optional: Borrow from Webform and add a select field in the templates that lets the user choose between plain text or Twig (Tokens would be available either way)

Add security, e.g. , permissions to use Twig in templates.

Write some tests to validate security, maybe some for functionality.

Create migrations from D7 mail templates that use scripting to D8+ mail templates Twig.

User interface changes

From D7, the user interface would change by replacing the "Dynamic Text Information" block with information about Twig, such as which variables are available, and a short "Twig 101" summary, followed by a link to the actual Twig documentation.