🇩🇪Germany @macdev_drupal

Wiesbaden
Account created on 26 March 2014, about 11 years ago
#

Recent comments

🇩🇪Germany macdev_drupal Wiesbaden

I have built a prototype that collects textfields and textareas.
As we do work with Saved Submissions I put the focus on there after spending some time within the unsaved method :-) by accident.
I added to checkboxes

Dont know if the uncombined version still works but the combined seems to do so.

<?php

namespace Drupal\spamaway\Plugin\WebformHandler;

use Drupal\Core\Form\FormStateInterface;
use Drupal\webform\Plugin\WebformHandlerBase;
use Drupal\webform\WebformSubmissionInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * Checks Anti SPAM of a submission.
 *
 * Checks if the current submission has been done before based on these criteria:
 * - within a specific period of time (configuratble)
 * - similar data has been posted for specific fields (based on similar_text percentage threshold)
 * - x amount of similar posts have been made.
 *
 * @WebformHandler(
 *   id = "spamaway_anti_spam_forms",
 *   label = @Translation("SpamAway - Anti spam handler"),
 *   category = @Translation("Anti-SPAM"),
 *   description = @Translation("SpamAway - Anti Spam based on repeated (similar) submissions."),
 *   cardinality = \Drupal\webform\Plugin\WebformHandlerInterface::CARDINALITY_SINGLE,
 *   results = \Drupal\webform\Plugin\WebformHandlerInterface::RESULTS_PROCESSED,
 * )
 */
class AntiSpamHandler extends WebformHandlerBase {

  const SPAMAWAY_SUBMISSION_TABLE = 'spamaway_webform_submission';

  /**
   * @var \Drupal\Core\Database\Connection $connection
   */
  protected $connection;

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    $instance = parent::create($container, $configuration, $plugin_id, $plugin_definition);
    $instance->connection = $container->get('database');
    return $instance;
  }

  /**
   * Gets default configuration for this plugin.
   *
   * @return array
   *   An associative array with the default configuration.
   */
  public function defaultConfiguration() {
    return [
      'spamaway_anti_spam_field_names' => 'message',
      'spamaway_anti_spam_hash' => 'sha256',
      'spamaway_anti_spam_threshold_percentage' => 80,
      'spamaway_anti_spam_period' => 0,
      'spamaway_anti_spam_allowed_count' => 5,
      'spamaway_anti_spam_ip_period' => 36000,
      'spamaway_anti_spam_allowed_ip_count' => 4,
      'spamaway_anti_spam_logging' => 0,
      'spamaway_query_limit' => 200,
      'spamaway_ip_check_enabled' => TRUE,
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function buildConfigurationForm(array $form, FormStateInterface $form_state) {

    $form['spamaway_anti_spam_field_names'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Field names'),
      '#description' => $this->t('A comma seperated list of field names to take into consideration for similarity. You can also add \'ip\' to check on IP address and combine fields using a + seperator. Ex: field_a,field_b+field_c,field_d+ip'),
      '#default_value' => $this->configuration['spamaway_anti_spam_field_names'] ??
        $this->defaultConfiguration['spamaway_anti_spam_field_names'],
    ];

    $form['spamaway_auto_detect_text_fields'] = [
      '#type' => 'checkbox',
      '#title' => $this->t('Auto-detect text fields'),
      '#description' => $this->t('Automatically detect and check all text and textarea fields.'),
      '#default_value' => $this->configuration['spamaway_auto_detect_text_fields'] ?? FALSE,
      ];

    $form['spamaway_auto_detect_combined_fields'] = [
      '#type' => 'checkbox',
      '#title' => $this->t('Combine auto-detected fields'),
      '#description' => $this->t('Combine all detected fields into a single check string (e.g. for full name + message)'),
      '#default_value' => $this->configuration['spamaway_auto_detect_combined_fields'] ?? FALSE,
    ];


    $form['spamaway_anti_spam_hash'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Hash algorithm'),
      '#description' => $this->t('The hash algorithm used for storing field name values to check against. See php hash for supported algoritms. This is only used if the webform is not storing data itself.'),
      '#default_value' => $this->configuration['spamaway_anti_spam_hash'] ??
        $this->defaultConfiguration['spamaway_anti_spam_hash'],
    ];

    $form['spamaway_anti_spam_threshold_percentage'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Threshold percentage'),
      '#suffix' => '%',
      '#description' => $this->t('A comma seperated list of threshold percentages for each field name (or one single value used for all field names). This is only use if the webform stores data itself.'),
      '#default_value' => $this->configuration['spamaway_anti_spam_threshold_percentage'] ??
        $this->defaultConfiguration['spamaway_anti_spam_threshold_percentage'],
    ];

    $form['spamaway_anti_spam_period'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Period of time'),
      '#description' => $this->t('The period for within similar submissions must have been submitted in seconds. Use 0(default) to disable this condition.'),
      '#suffix' => 'seconds',
      '#default_value' => $this->configuration['spamaway_anti_spam_period'] ??
        $this->defaultConfiguration['spamaway_anti_spam_period'],
    ];

    $form['spamaway_anti_spam_allowed_count'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Allowed count'),
      '#description' => $this->t('The number of similar submissions allowed before we consider it spam. You can also use a comma seperated list of counts for each field name you specified.'),
      '#default_value' => $this->configuration['spamaway_anti_spam_allowed_count'] ??
        $this->defaultConfiguration['spamaway_anti_spam_allowed_count'],
    ];

    $form['spamaway_anti_spam_ip_period'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Period of time for IP'),
      '#description' => $this->t('The period for within submissions with the same IP address are considered spam in seconds. Defaults to 10 minutes'),
      '#suffix' => 'seconds',
      '#default_value' => $this->configuration['spamaway_anti_spam_ip_period'] ??
        $this->defaultConfiguration['spamaway_anti_spam_ip_period'],
    ];

    $form['spamaway_anti_spam_allowed_ip_count'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Allowed IP count'),
      '#description' => $this->t('The number of similar submissions allowed by the same IP address before we consider it spam. Defaults to 4'),
      '#default_value' => $this->configuration['spamaway_anti_spam_allowed_ip_count'] ??
        $this->defaultConfiguration['spamaway_anti_spam_allowed_ip_count'],
    ];

    $form['spamaway_ip_check_enabled'] = [
      '#type' => 'checkbox',
      '#title' => $this->t('IP check enabled'),
      '#description' => $this->t('Enable IP check validation.'),
      '#default_value' => $this->configuration['spamaway_ip_check_enabled'] ??
        $this->defaultConfiguration['spamaway_ip_check_enabled'],
    ];

    $form['spamaway_anti_spam_logging'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Enable logging'),
      '#default_value' => $this->configuration['spamaway_anti_spam_logging'] ??
        $this->defaultConfiguration['spamaway_anti_spam_logging'],
    ];

    $form['spamaway_query_limit'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Query limit'),
      '#description' => $this->t('Cannot be higher than 200'),
      '#default_value' => $this->configuration['spamaway_query_limit'] ??
        $this->defaultConfiguration['spamaway_query_limit'],
    ];

    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function submitConfigurationForm(array &$form, FormStateInterface $form_state) {
    $this->configuration['spamaway_anti_spam_field_names'] = $form_state->getValue('spamaway_anti_spam_field_names');
    $this->configuration['spamaway_anti_spam_hash'] = $form_state->getValue('spamaway_anti_spam_hash');
    $this->configuration['spamaway_anti_spam_period'] = $form_state->getValue('spamaway_anti_spam_period');
    $this->configuration['spamaway_auto_detect_text_fields'] = $form_state->getValue('spamaway_auto_detect_text_fields');
    $this->configuration['spamaway_auto_detect_combined_fields'] = $form_state->getValue('spamaway_auto_detect_combined_fields');
    $this->configuration['spamaway_anti_spam_allowed_count'] = $form_state->getValue('spamaway_anti_spam_allowed_count');
    $this->configuration['spamaway_anti_spam_threshold_percentage'] = $form_state->getValue('spamaway_anti_spam_threshold_percentage');
    $this->configuration['spamaway_anti_spam_ip_period'] = $form_state->getValue('spamaway_anti_spam_ip_period');
    $this->configuration['spamaway_anti_spam_allowed_ip_count'] = $form_state->getValue('spamaway_anti_spam_allowed_ip_count');
    $this->configuration['spamaway_anti_spam_logging'] = $form_state->getValue('spamaway_anti_spam_logging');
    $this->configuration['spamaway_query_limit'] = $form_state->getValue('spamaway_query_limit');
    $this->configuration['spamaway_ip_check_enabled'] = $form_state->getValue('spamaway_ip_check_enabled');
  }

  /**
   * {@inheritdoc}
   */
  public function postSave(WebformSubmissionInterface $webform_submission, $update = TRUE) {

    // Store the ip address into our custom table.
    if ($this->isIpCheckEnabled()) {
      $this->connection->insert(self::SPAMAWAY_SUBMISSION_TABLE)->fields([
        'webform_id' => $webform_submission->getWebform()->id(),
        'created' => $webform_submission->getCreatedTime(),
        'submission' => $webform_submission->serial(),
        'field_name' => 'ip',
        'value' => $webform_submission->getRemoteAddr()
      ])->execute();
    }

    // Store the submission fields we need into our custom table.
    if (!$this->isSaveSubmissionsEnabled()) {
      $field_names = $this->getFieldNames();
      foreach ($field_names as $field_name) {
        $value = $this->getFieldValue($field_name, ['ip' => $webform_submission->getRemoteAddr()] + $webform_submission->getData());
        if (!empty($value)) {
          $this->connection->insert(self::SPAMAWAY_SUBMISSION_TABLE)->fields([
            'webform_id' => $webform_submission->getWebform()->id(),
            'created' => $webform_submission->getCreatedTime(),
            'submission' => $webform_submission->serial(),
            'field_name' => $field_name,
            'value' => $value
          ])->execute();
        }
      }
    }

  }

  /**
   * {@inheritdoc}
   */
  public function postDelete(WebformSubmissionInterface $webform_submission) {
    // Delete all entries from the custom table.
    $this->connection->delete(self::SPAMAWAY_SUBMISSION_TABLE)
      ->condition('webform_id', $webform_submission->getWebform()->id())
      ->condition('submission', $webform_submission->serial())
      ->execute();
  }

  /**
   * Helper to get an array of clean values from a string.
   *
   * @param string $seperator
   * @param string $array
   *
   * @return array
   */
  protected function explodeTrimmed($seperator, $string) {
    return array_filter(array_map('trim', explode($seperator, $string)));
  }

  /**
   * Get the field names from the config settings.
   *
   * @return array
   */
  protected function getFieldNames() {
    if (!empty($this->configuration['spamaway_auto_detect_text_fields'])) {
      $elements = $this->getWebform()->getElementsDecoded();
      $field_names = [];

      // Rekursive Extraktion
      $extractTextFields = function(array $elements, string $prefix = '') use (&$extractTextFields, &$field_names) {
        foreach ($elements as $key => $element) {
          if (strpos($key, '#') === 0) {
            continue;
          }

          // Aktueller flacher Feldname
          $field_key = $element['#webform_key'] ?? $key;

          if (isset($element['#type']) && in_array($element['#type'], ['textfield', 'textarea'])) {
            $field_names[] = $field_key;
          }

          if (is_array($element)) {
            $extractTextFields($element, $prefix);
          }
        }
      };

      $extractTextFields($elements);

      if (!empty($this->configuration['spamaway_auto_detect_combined_fields'])) {
        $combined = implode('+', $field_names);
        \Drupal::logger('spamaway_spam')->debug('Auto-detected (combined) flat fields: @combo', ['@combo' => $combined]);
        return [$combined];
      }
      else {
        \Drupal::logger('spamaway_spam')->debug('Auto-detected flat fields: <pre>@fields</pre>', [
          '@fields' => print_r($field_names, TRUE),
        ]);
        return $field_names;
      }
    }

    // Manuelle Eingabe (auch hier flach erwartet)
    return $this->explodeTrimmed(',', $this->configuration['spamaway_anti_spam_field_names']);
  }





  /**
   * Returns the hashed value of a field name (or field key combo).
   */
  protected function getFieldValue($field_name, $data) {
    $value = '';
    $keys = $this->explodeTrimmed('+', $field_name);

    foreach ($keys as $key) {
      if (!empty($data[$key])) {
        $value .= $data[$key];
      }
    }

    return $value;
  }

  protected function isSaveSubmissionsEnabled() {
    return ($this->getWebform()->getSetting('results_disabled') === FALSE);
  }

  protected function isLoggingEnabled() {
    return ($this->configuration['spamaway_anti_spam_logging'] ?? 0);
  }

  protected function isIpCheckEnabled() {
    return ($this->configuration['spamaway_ip_check_enabled'] ?? 0);
  }

  protected function hashValue($value) {
    $hashed = hash($this->configuration['spamaway_anti_spam_hash'], $value, false);
    return $hashed;
  }

  /**
   * Validate webform submission webform .
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   * @param \Drupal\webform\WebformSubmissionInterface $webform_submission
   *   A webform submission.
   */
  public function validateForm(array &$form, FormStateInterface $form_state, WebformSubmissionInterface $webform_submission) {

    if (\Drupal::currentUser()->hasPermission('spamaway bypass spam detection')) {
      if ($this->isLoggingEnabled()) {
        $this->getLogger('spamaway_spam')->debug($this->t('Spam detection was bypassed for @user', ['@user' => \Drupal::currentUser()->getAccountName()]));
      }
      //return;
    }

    // Set hard limit
    if ($this->configuration['spamaway_query_limit'] > 200) {
      $this->configuration['spamaway_query_limit'] = 200;
    }

    ////////////////////////////////////////////////////////////////////////////
    // Make sure the last x seconds no submission was done by the same IP
    // address. Otherwise consider it spam.
    if ($this->isIpCheckEnabled()) {
      $this->baseIpCheck($webform_submission, $form_state);
    }

    if ($this->isSaveSubmissionsEnabled()) {
      $this->validateFormWithSavedSubmissions($form, $form_state, $webform_submission);
    } else {
      $this->validateFormCustomSubmissions($form, $form_state, $webform_submission);
    }

  }

  private function baseIpCheck(WebformSubmissionInterface $webform_submission, FormStateInterface $form_state) {
    $query_ip = $this->connection->select(self::SPAMAWAY_SUBMISSION_TABLE, 'w');
    $query_ip->condition('webform_id', $webform_submission->getWebform()->id());
    $query_ip->condition('field_name', 'ip');
    $query_ip->condition('value', $webform_submission->getRemoteAddr());
    $query_ip->where('created > UNIX_TIMESTAMP() - :limit', [':limit' => $this->configuration['spamaway_anti_spam_ip_period']]);
    $count_ip = $query_ip->countQuery()->execute()->fetchField();
    if ($count_ip > $this->configuration['spamaway_anti_spam_allowed_ip_count']) {
      $this->spamDetected($form_state, $this->t('Spam detected by IP check on @webform from @ip within @period (count @count / @allowed)', [
        '@webform' => $webform_submission->getWebform()->id(),
        '@ip' => $webform_submission->getRemoteAddr(),
        '@period' => $this->configuration['spamaway_anti_spam_ip_period'],
        '@count' => $count_ip,
        '@allowed' => $this->configuration['spamaway_anti_spam_allowed_ip_count']
      ]));
    }
  }

  /**
   * Validate webform submission webform .
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   * @param \Drupal\webform\WebformSubmissionInterface $webform_submission
   *   A webform submission.
   */
  protected function validateFormWithSavedSubmissions(array &$form, FormStateInterface $form_state, WebformSubmissionInterface $webform_submission) {
    \Drupal::logger('spamaway_spam')->debug('✅ validateFormWithSavedSubmissions() wurde aufgerufen');

    $webform_id = $webform_submission->getWebform()->id();
    $field_names = $this->getFieldNames();
    $form_data = $form_state->getValues();
    \Drupal::logger('spamaway_spam')->debug('📦 Form Values: <pre>@data</pre>', [
      '@data' => print_r($form_state->getValues(), TRUE),
    ]);

    if (empty($field_names)) {
      return;
    }

    $flat_names = [];
    foreach ($field_names as $field_entry) {
      foreach (explode('+', $field_entry) as $name) {
        $parts = explode('.', $name);
        $flat_names[] = array_pop($parts);
      }
    }

    // Formularwerte kombinieren (für Vergleich)
    $combined_key = implode('+', $field_names);
    $submitted_value = $this->getFieldValue($combined_key, $form_data);
    \Drupal::logger('spamaway_spam')->debug('📤 Submitted raw combined string: @string', [
      '@string' => $submitted_value,
    ]);
    $threshold = (int) $this->configuration['spamaway_anti_spam_threshold_percentage'];
    $allowed_count = (int) $this->configuration['spamaway_anti_spam_allowed_count'];

    // Zeitraum
    $query = $this->connection->select('webform_submission', 's');
    $query->addField('s', 'sid');
    $query->condition('s.webform_id', $webform_id);
    $query->orderBy('s.created', 'DESC');
    $query->range(0, $this->configuration['spamaway_query_limit']);

    $period = time() - $this->configuration['spamaway_anti_spam_period'];
    if ($this->configuration['spamaway_anti_spam_period'] && $period > 0) {
      $query->condition('s.created', $period, '>');
    }

    $sids = $query->execute()->fetchCol();

    if (empty($sids)) {
      \Drupal::logger('spamaway_spam')->debug('📭 Keine früheren SIDs gefunden');
      return;
    }

    // Alle Daten dieser SIDs aus webform_submission_data holen
    $data_query = $this->connection->select('webform_submission_data', 'd');
    $data_query->fields('d', ['sid', 'name', 'value']);
    $data_query->condition('d.sid', $sids, 'IN');
    $data_query->condition('d.name', $flat_names, 'IN');
    $data_rows = $data_query->execute()->fetchAll();

    \Drupal::logger('spamaway_spam')->debug('💾 Gefundene Datenzeilen: <pre>@data</pre>', [
      '@data' => print_r($data_rows, TRUE),
    ]);

    // Gruppieren nach SID
    $grouped = [];
    foreach ($data_rows as $row) {
      $grouped[$row->sid][$row->name] = $row->value;
    }

    $similar_count = 0;

    foreach ($grouped as $sid => $values) {
      $db_raw_string = '';
      foreach ($flat_names as $name) {
        $db_raw_string .= $values[$name] ?? '';
      }

      \Drupal::logger('spamaway_spam')->debug('📥 DB raw combined string (SID @sid): @string', [
        '@sid' => $sid,
        '@string' => $db_raw_string,
      ]);

      $submitted_hash = $this->hashValue($submitted_value);
      $db_value = $this->hashValue($db_raw_string);

      \Drupal::logger('spamaway_spam')->debug('🧩 Zusammengesetzter Submitted-Wert aus @fields: @value', [
        '@fields' => implode(', ', $field_names),
        '@value' => $submitted_value,
      ]);

      $percent = 0;
      similar_text($submitted_value, $db_raw_string, $percent);


      \Drupal::logger('spamaway_spam')->debug('🔍 Vergleich mit SID @sid: @percent% (@submitted vs. @db)', [
        '@sid' => $sid,
        '@percent' => $percent,
        '@submitted' => $submitted_value,
        '@db' => $db_raw_string,
      ]);

      if ($percent >= $threshold) {
        $similar_count++;
        if ($similar_count >= $allowed_count) {
          $this->spamDetected($form_state, $this->t('Spam erkannt bei @webform: @count ähnliche Einträge ≥ @threshold%', [
            '@webform' => $webform_id,
            '@count' => $similar_count,
            '@threshold' => $threshold,
          ]));
          break;
        }
      }
    }
  }

  /**
   * Validate webform submission webform using the custom submissions table.
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   * @param \Drupal\webform\WebformSubmissionInterface $webform_submission
   *   A webform submission.
   */
  public function validateFormCustomSubmissions(array &$form, FormStateInterface $form_state, WebformSubmissionInterface $webform_submission) {

    ////////////////////////////////////////////////////////////////////////////
    // Check if the submitted message is similar to previous submitted messages
    \Drupal::logger('spamaway_spam')->debug('== validateFormCustomSubmissions() aufgerufen ==');
    $field_names = $this->getFieldNames();
    if (empty($field_names)) {
      return;
    }

    // Get the field values ...
    $query = $this->connection->select(self::SPAMAWAY_SUBMISSION_TABLE, 's');
    $query->addField('s', 'value', 'value');
    $query->addField('s', 'field_name', 'field_name');
    $query->orderBy('s.created', 'DESC');

    // for this form ...
    $query->condition('s.webform_id', $webform_submission->getWebform()->id());

    // limited to X last submissions ...
    $query->range(0, $this->configuration['spamaway_query_limit']);

    // withing a period of time ...
    $period = time() - $this->configuration['spamaway_anti_spam_period'];
    if ($this->configuration['spamaway_anti_spam_period'] && $period > 0) {
      $query->condition('s.created', $period, '>');
    }

    // limited by the fields we need ...
    $query->condition('s.field_name', $field_names, 'IN');
    $rows = $query->execute()->fetchAll();

    // Keep track of how many times we have the same field.
    $matching_count = array_combine($field_names, array_fill(0, count($field_names), 0));

    // How many times are we allowed to have the same field.
    $allowed_count = $this->explodeTrimmed(',', $this->configuration['spamaway_anti_spam_allowed_count']);
    if (count($allowed_count) == 1 || count($allowed_count)!=count($field_names)) {
      // If we have only one threshold or the numbers don't match the number
      // of fields we need take the first threshold for everything.
      $allowed_count = array_fill(0, count($field_names), $allowed_count[0]);
    }
    $allowed_count = array_combine($field_names, $allowed_count);

    foreach ($rows as $row) {

      $submitted_value = $this->getFieldValue($row->field_name, ['ip' => $webform_submission->getRemoteAddr()] + $form_state->getValues());

      if ($submitted_value === $row->value) {
        $matching_count[$row->field_name]++;

        if ($matching_count[$row->field_name] < $allowed_count[$row->field_name]) {
          continue;
        }

        $this->spamDetected($form_state, $this->t('Spam detected by hash check on @webform due to similar post within @period', [
          '@webform' => $webform_submission->getWebform()->id(),
          '@ip' => $webform_submission->getRemoteAddr(),
          '@period' => $this->configuration['spamaway_anti_spam_ip_period']
        ]));

        break;
      }
    }
  }

  protected function spamDetected(FormStateInterface $form_state, $message) {
    // We had (a) very similar submission(s) before so we ignore the new submission.
    $form_state->setErrorByName('', $this->t('Spam detected. Please contact the site administrator if the issue persists.'));
    if ($this->isLoggingEnabled()) {
      $this->getLogger('spamaway_spam')->debug($message);
    }
  }

  protected function getFlattenedFieldNames($field_names) {
    $flattened_field_names = [];

    foreach ($field_names as $field_name) {
      // Teile den Namen des Feldes anhand des Punktes
      $parts = explode('.', $field_name);

      // Nehme nur den letzten Teil des Namens
      $flattened_field_names[] = array_pop($parts);
    }

    return $flattened_field_names;
  }





}



I haven't done many tests until now but I think a value between 65-75% would catch similiar spam for average contact forms.

The code above is a bit a mess, and I still have to clean it up to create a patch or so.
The hard thing for me was to figure out the when you want to get the field types you have to deal with nesting like flex_box.name or street.streetname and when you get the values from the db and form the submission you just have flat arrays.

As said I first hacked around in the unsaved submissions part but did not test it so maybe just ignore that part.

🇩🇪Germany macdev_drupal Wiesbaden

We got 200+ Spambot Submissions this night. We have rate limting active so that they did submit one post every three minutes that came through. It some kind of bitcoin spam:

 betreff: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  e_mail1: emiliogervacio02@gmail.com
  hausnummer: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  ihr_anliegen: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  nachname: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  ort: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  plz: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  strassennamen: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  telefonnummer: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'
  vorname: '1.333810 BTC in Your Account! That’s $118683 Ready to Be Collected:::: https://da.gd/SE68TK'

Sadly Modsecurity did not capture the post requests as no rule was triggered.

I wonder if it would be an option to add a factor to the antibot-key dynamically based on time of the form get.
At the moment it looks like the key is static for the same ip-adress. Okay, this would impact caching but then the bot would need to use the new key each time a form is submitted.

🇩🇪Germany macdev_drupal Wiesbaden

Okay, I tried using an action handler: Lock submission when submitted. This is one of the options provided by Webform.

After pressing the submit button, the submission is successfully locked. However, when the user opens the tokenized URL and re-edits the form, the lock is removed, and the values can be changed.

I believe this might be a bug. Or is it intended behavior that a locked submission can still be edited?

🇩🇪Germany macdev_drupal Wiesbaden

We are facing a similar issue. It would be beneficial to have an option to mark a Webform submission as "frozen" once a draft is submitted.

Currently, we provide an anonymous user with a tokenized URL to allow them to return later and continue editing the draft. However, once the form is finally submitted, the submission should no longer be editable, at least by the anonymous user.

The problem is that as long as the token URL remains valid, the submission can still be edited, even after it has been marked as submitted. This creates a situation where users can modify what should be a finalized submission.

Having a way to lock or freeze submissions after they are marked as complete would resolve this issue.

Maybe this could be achieved with a webform handler? Maybe there already is one?

🇩🇪Germany macdev_drupal Wiesbaden

Drush hands over to the local module. So I think this should be addressed there.
Just checked
web/core/modules/locale/locale.fetch.inc
web/core/modules/locale/locale.batch.inc
the methods there are just triggered from drush locale:check and the business logic is in those methods.
Maybe there could be a fall back, before deleting / renaming .po files locally.

🇩🇪Germany macdev_drupal Wiesbaden

Same here. It fails for a part of our code base:

hm-medien.de > [warning] Trying to access array offset on value of type null HandlerBase.php:724
hm-medien.de > [error] A valid cache entry key is required. Use getAll() to get all table data.
hm-medien.de > [error] Update failed: views_post_update_taxonomy_filter_user_context
hm-medien.de > [error] Update aborted by: views_post_update_taxonomy_filter_user_context

Will investigate further.
Had some config changes after core / module Updates in views config ymls which have not been commited yet.
On the main part of our code base the update did run well.

🇩🇪Germany macdev_drupal Wiesbaden

+1

Currently trying to Upgrade the config modules and after there seems to be a a patch for splits and config ignore, since config_split_ignore is not using the new api, I am now stuck again with:

Problem 1
- Root composer.json requires drupal/config_filter ^2.6, found drupal/config_filter[dev-2.x, 2.6.0, 2.x-dev] but these were not loaded, likely because it conflicts with another require.
Problem 2
- drupal/webform_config_ignore is locked to version 1.3.0 and an update of this package was not requested.
- drupal/webform_config_ignore 1.3.0 requires drupal/config_filter ^1 -> found drupal/config_filter[dev-1.x, 1.0.0-beta3, ..., 1.x-dev (alias of dev-1.x)] but it conflicts with your root composer.json require (^2.6).

🇩🇪Germany macdev_drupal Wiesbaden

Sorry, I think it just messed up things here because we did have 8.x-1.21 running with 10.2 (but not tested yet) and the incoming merge with 8.x-1.22 broke it, and I couldn't figure out the problem, which obviously should have something to do with one of the updated modules.
You are right.
Maybe it's better to do it like what we did when upgrading from D9 to D10 like here: "drupal/csv_serialization": "4.0 as 3.0".
This will make composer.json merge conflicts easier to handle if other modules will follow to lock support.

🇩🇪Germany macdev_drupal Wiesbaden

Hi, I am trying to explain our use case.
We have a large multisite installation and custom and contrib modules which are used only by specific customers.
So, for example, we enable basic_cart only for some of them by enabling a split.
This is where config_split_ignore comes in.
It is used primarily to protect the settings of a split against being overridden. This couldn't be done with config_ignore because the general configuration doesn't know anything about the split.

dependencies:
  module:
    - config_split_ignore
third_party_settings:
  config_split_ignore:
    entities:
      - basic_cart.checkout
      - config_pages.type.basic_cart_settings
dependencies:
  module:
    - config_split_ignore
third_party_settings:
  config_split_ignore:
    entities:
      - 'he_dba.settings:custom_tables'
id: config_map_feuerwehr

If the settings of a split are not in the config because one cannot set them to ignore, they will be deleted at a drush cim during the deployment of a new release, as long as they wouldn't be exported with drush cex. Exporting them would, on the other hand, lead to conflicts with other site settings that differ.

Hope this explains the pain we are in a bit :-)

🇩🇪Germany macdev_drupal Wiesbaden

Looks like the project is now "unsupported". Any chance that someone will be able to merge the D10 automated fixes into a release? There are now broken dependencies form basic_cart module.

🇩🇪Germany macdev_drupal Wiesbaden

Are there any plans to do so @maximpodorov - as config_ignore 2.x seems to be unsupported now?

🇩🇪Germany macdev_drupal Wiesbaden

Yeah I understand.
We just stumbeld upon this when a content creator was trying to set up a webform in a different language in a non multilingual drupal site.
Our distro doesn't support that at the moment but people always try to hack funny things.
So maybe we will use a config page and use a form alter hook to let them translate it in the UI somehow.

🇩🇪Germany macdev_drupal Wiesbaden

Same here. We got several requests like this which came through the form within under a minute. So either it is remote post or a client with selenium I guess. But they would have to record this before. Anyway they got past Antibot.
The requests contain some SQL Injection patterns.

Cookie: M1R4X=1700663570xCaY4D9A6Vmrb; SSESS90efd8bd784fa77255ffa5094c2e07f7=dp1%2CS2R0Iv-hC5RvWNZJrsVnoCE6J2ey9ZCFZfjvrOMPL0Ns

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36

--e862924b-I--
anrede%5bselect%5d=%5fother%5f&anrede%5bother%5d=1&vorname=gjIjhtUZ&nachname=gjIjhtUZ&ihre%5fe%5fmail%5fadresse=testing%40example%2ecom&url%5fbzw%5fquellenangabe%5fder%5fbarriere=1&beschreibung=1&antibot%5fkey=c5vdMtXZaktbnbWNUD7JIFgSxV31bEBrDmJ%5f2VHUuYI&form%5fbuild%5fid=%2d1+OR+2%2b215%2d215%2d1%3d0%2b0%2b0%2b1+%2d%2d+&form%5fid=webform%5fsubmission%5fbarriere%5fmelden%5fparagraph%5f7180%5fadd%5fform

HTTP/1.1 303 See Other

Location: https://drupal.website.de/webform/barriere_melden/confirmation?token=mkgaCOgK_gh6Q4oQO04FcY_95SavIlL7yffSJoY0uZw

We did capture this with modsecurity.
Maybe somone with deeper knowledge of Antibot could shine some light on this.
And eventually improve the module.

🇩🇪Germany macdev_drupal Wiesbaden

Got it working for a menu link field that way.

      $form['title']['widget'][0]['value']['#maxlength'] = 25;
      $form['title']['widget'][0]['value']['#attributes']['data-maxlength'] = 25;
      $form['title']['widget'][0]['value']['#maxlength_js'] = TRUE;

Guess you just need to add that 'data-maxlength' to your example above.
For my case it needed to be inside of $element['widget'][0]['value'] to get it working.
The documentation says it different.

🇩🇪Germany macdev_drupal Wiesbaden

Sorry for bothering you Jake! Did test it on simplytest.me and it works like designed. So it must be somewhere in our 34+ custom modules an 50k+ lines off code. Closing the Issue.

🇩🇪Germany macdev_drupal Wiesbaden

Okay, changed description_display to 'before' here but doesn't make a difference. https://git.drupalcode.org/project/drupal/-/blob/9.5.x/core/lib/Drupal/C...
Will investigate this further.

🇩🇪Germany macdev_drupal Wiesbaden

Confirm - 2.0.2 is broken with 9.5.9 Drupal Core

🇩🇪Germany macdev_drupal Wiesbaden

Have the same error after updating to php 8.1.20 and drupal core 9.5.8
We set the identiy from settings.php, as we run more than 100 sites with the same drupal build, with:

$config['system.logging']['error_level'] = 'all';
$config['syslog.settings']['identity'] = '';
$config['syslog.settings']['format'] = '!base_url|!timestamp|!type|!ip|!request_uri|!referer|!uid|!link|!message';

After a drush cr is run the first visitor would se the red deprecation warning.
Therefore I would upvote for this to be fixed in core.

🇩🇪Germany macdev_drupal Wiesbaden

Not sure if it is the same problem but downgrading from 3.0.0-beta3 to 3.0.0-beta1 fix this issue for us.
It seems like the embed and includes do no loger work like that.

 {% block content %}
    {% include '@hw_radix_standard/navbar/navbar-brand.twig' with {
      prefix: site_name_prefix,

Twig\Error\LoaderError: Template "@hw_radix_standard/navbar/navbar-brand.twig" is not defined in "themes/custom/hw_radix_standard/templates/block/block--system-branding-block.html.twig" at line 26. in Twig\Loader\ChainLoader->getCacheKey() (line 98 of /var/www/html/vendor/twig/twig/src/Loader/ChainLoader.php).
Twig\Environment->getTemplateClass('@hw_radix_standard/navbar/navbar-brand.twig', NULL) (Line: 205)
Drupal\Core\Template\TwigEnvironment->getTemplateClass('@hw_radix_standard/navbar/navbar-brand.twig') (Line: 381)
Twig\Environment->loadTemplate('@hw_radix_standard/navbar/navbar-brand.twig', NULL) (Line: 333)
Twig\Template->loadTemplate('@hw_radix_standard/navbar/navbar-brand.twig', 'themes/custom/hw_radix_standard/templates/block/block--system-branding-block.html.twig', 26) (Line: 199)
__TwigTemplate_0c5736a4f96b51b76a13058fbc5c6d5f___75412211->block_content(Array, Array) (Line: 182)
Twig\Template->displayBlock('content', Array, Array) (Line: 79)
__TwigTemplate_8841fceebceb03af7747f1ab821ed205->doDisplay(Array, Array) (Line: 405)
Twig\Template->displayWithErrorHandling(Array, Array) (Line: 378)
Twig\Template->display(Array, Array) (Line: 190)
__TwigTemplate_0c5736a4f96b51b76a13058fbc5c6d5f___75412211->doDisplay(Array, Array) (Line: 405)
Twig\Template->displayWithErrorHandling(Array, Array) (Line: 378)
Twig\Template->display(Array) (Line: 69)
__TwigTemplate_0c5736a4f96b51b76a13058fbc5c6d5f->doDisplay(Array, Array) (Line: 405)
Twig\Template->displayWithErrorHandling(Array, Array) (Line: 378)
Twig\Template->display(Array) (Line: 390)
Twig\Template->render(Array) (Line: 55)
twig_render_template('themes/custom/hw_radix_standard/templates/block/block--system-branding-block.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('block', Array) (Line: 433)
Drupal\Core\Render\Renderer->doRender(Array) (Line: 446)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 204)
Drupal\Core\Render\Renderer->render(Array) (Line: 479)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, 'html', NULL, 1) (Line: 178)
__TwigTemplate_83b2ac2dc15fc1e2e161799ddba49516->block_branding_top(Array, Array) (Line: 182)
Twig\Template->displayBlock('branding_top', Array, Array) (Line: 60)
__TwigTemplate_83b2ac2dc15fc1e2e161799ddba49516->doDisplay(Array, Array) (Line: 405)
Twig\Template->displayWithErrorHandling(Array, Array) (Line: 378)
Twig\Template->display(Array) (Line: 390)
Twig\Template->render(Array) (Line: 55)
twig_render_template('themes/custom/hw_radix_standard/src/components/page/page.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('page', Array) (Line: 433)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 204)
Drupal\Core\Render\Renderer->render(Array) (Line: 479)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, 'html', NULL, 1) (Line: 132)
__TwigTemplate_b3a54b59fb991600ee3d43e44ae0a08d->doDisplay(Array, Array) (Line: 405)
Twig\Template->displayWithErrorHandling(Array, Array) (Line: 378)
Twig\Template->display(Array) (Line: 390)
Twig\Template->render(Array) (Line: 55)
twig_render_template('themes/custom/hw_radix_standard/src/components/system/html.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('html', Array) (Line: 433)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 204)
Drupal\Core\Render\Renderer->render(Array) (Line: 162)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 580)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 163)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object)
call_user_func(Array, Object, 'kernel.view', Object) (Line: 142)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object, 'kernel.view') (Line: 174)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 81)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 191)
Drupal\page_cache\StackMiddleware\PageCache->fetch(Object, 1, 1) (Line: 128)
Drupal\page_cache\StackMiddleware\PageCache->lookup(Object, 1, 1) (Line: 82)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 718)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

🇩🇪Germany macdev_drupal Wiesbaden

We had some similar issues when updateing to 1.4 because the "Enable on this site" Checkbox was not checked by default. Maybe this could be the same reason?
The logouts happend not predictably in this state: https://www.drupal.org/project/autologout/issues/3363875 🐛 Autologout disabled after 1.4 Upgrade Closed: works as designed

🇩🇪Germany macdev_drupal Wiesbaden

For me, it seems like this new Base_url field in entity_print 2.11 does not work for us.

We have differences between stages, we are coping with like installations which can not curl themselves at all, installations which can curl only via localhost, installations which can resolve their domain names and docker / ddev for development.
And then there is multisite, we have to deal with. Further, we are using it with webform, which has some special issues when private files and the signature field are used.

I got it working in ddev / docker, but not at installations which can call themselves only via localhost. It, too, seems that the file:/// syntax is not supported.
If the patch is applied against 2.11 it doesn't work either. I guess the base_url field form module needs to be overridden to bring back the previous behavior.
Meanwhile, I'll roll back to the patched 2.7 and try to figure out if a differently patched 2.11 would be an option.

🇩🇪Germany macdev_drupal Wiesbaden

Running with Core 9.4.10, entity_print 2.11.0 and webforms 6.1.4 with Patch #3 I do get the error:
Error generating document: Failed to generate PDF: The CSS selector 'summary::marker' is not valid

When I apply patch #2 it works.

Production build 0.71.5 2024