drush wfx does not respect --excluded-columns

Created on 18 September 2023, about 1 year ago
Updated 4 October 2023, about 1 year ago

Problem/Motivation

Adding --excluded-columns does not change the output of drush wfx
I'd like to filter out unnecessary columns as advertised by the help screen and as it works in the webGUI.

Steps to reproduce

Create some simple form.
drush wfx --excluded-columns="sticky"
Still see the "sticky" column being exported.

Proposed resolution

Patch attached works for me. The excluded columns do not seem to be formatted in the expected way. I added a condition to keep the old method as well as it might be necessary for the webGUI-based export. I don't know enough to judge this. Maybe it can be removed as well.

Remaining tasks

Review from someone more knowledgeable than me necessary. I also have two comments on other places in the code, maybe I don't understand it correctly (I am not a dev) or maybe they help someone optimize a few things:

It seems default options get set twice:

src/WebformSubmissionExporter.php:

 231   public function setExporter(array $export_options = []) {
 230     $export_options += $this->getDefaultExportOptions();

and
src/Commands/WebformSubmissionCommands.php:

158     // Get command options as export options.
159     $default_options = $submission_exporter->getDefaultExportOptions();
160     $export_options = Drush::redispatchOptions();
161     $export_options['access_check'] = FALSE;
162     // Convert dashes to underscores.
163     foreach ($export_options as $key => $value) {
164       unset($export_options[$key]);
165       $key = str_replace('-', '_', $key);
166       if (isset($default_options[$key]) && is_array($default_options[$key])) {
167         $value = explode(',', $value);
168       }
169       $export_options[$key] = $value;
170     }
171     $export_options += $submission_exporter->getDefaultExportOptions();
172     $submission_exporter->setExporter($export_options);

also, the call to $submission_exporter->getDefaultExportOptions() in line 171 seems unnecessary, as line 159 already does that. $export_options+=$default_options should suffice (and be faster).

I can provide a separate issue and patch for this as well, if this is the better way to solve this. Or some maintainer just changes it - I'm absolutely OK with this as well!

🐛 Bug report
Status

Fixed

Version

6.2

Component

Code

Created by

🇦🇹Austria tgoeg

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @tgoeg
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update about 1 year ago
    Patch Failed to Apply
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update about 1 year ago
    Patch Failed to Apply
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update about 1 year ago
    535 pass
  • Status changed to Needs review about 1 year ago
  • 🇦🇹Austria tgoeg

    Tests passing, so setting to needs review.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update about 1 year ago
    535 pass
  • 🇺🇸United States jrockowitz Brooklyn, NY

    The issue makes sense. The patch is a guess at fixing the issue via the drush command.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update about 1 year ago
    535 pass
  • 🇦🇹Austria tgoeg

    Right, with more knowledge of the code this is the correct place to fix it, i.e. only for the drush command and not globally as my attempt did.
    I built on top of that and removed the extraneous adding of default options, as setExporter() does it anyway and that function gets used by the webGUI-based export as well, so it should stay there (though it might be better to get the default options only once, before calling setExporter(), in both cases of drush and webGUI-based export.
    The measured overhead however is minimal, caching and PHP optimizations seem to take care of it.

    Tested it with and with my removed line.
    In both cases, the uuid column is not exported, as it is part of default excluded columns.

    Works for me!

  • 🇮🇳India bebalachandra

    Tested patch #5 and patch #6 on my local drupal 10.1. Firstly Please note that both patches are same, only the difference in patch #6 is removal of 171 line i.e " $export_options += $submission_exporter->getDefaultExportOptions();". Done testing with this line of code and without also. Both cases the result is same. "--excluded-colums" is working as expected after applying both patches. attaching Screenshot for reference. Let maintainer decide which is the best patch to consider. Both patches resolve the issue which is stated here.

    We can move this to RTBC+1

  • 🇺🇸United States jrockowitz Brooklyn, NY

    I am confused by #6. Removing the default export options will likely cause unexpected regressions for other people.

  • 🇮🇳India bebalachandra

    In that case, better we can consider #5 which is mainly focused on this ticket.

  • 🇦🇹Austria tgoeg

    Removing the default export options will likely cause unexpected regressions for other people.

    No, it won't, as they are not removed.
    As already laid out,
    src/Commands/WebformSubmissionCommands.php
    calls
    172 $submission_exporter->setExporter($export_options);
    (Additionally to
    171 $export_options += $submission_exporter->getDefaultExportOptions();)

    This in turn, in setExporter() in
    src/WebformSubmissionExporter.php:
    calls
    $export_options += $this->getDefaultExportOptions();

    So you end up calling getDefaultExportOptions(); twice (and adding it to the $export_options).

    As also laid out and tested, the default options do get set with only one call (which should be no surprise).

    Yours to decide.

    I can open up a second issue as well, if you like.

  • Status changed to RTBC about 1 year ago
  • 🇺🇸United States jrockowitz Brooklyn, NY

    Yep @togoeg stepped thru the code and confirmed the default options are always applied via \Drupal\webform\WebformSubmissionExporter::setExporter.

    The patch from #6 is RTBC. This can be backported to 6.1.x

  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 8.1 & MySQL 8
    last update about 1 year ago
    535 pass
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.2 & MySQL 8
    last update about 1 year ago
    535 pass
  • Status changed to Fixed about 1 year ago
  • 🇨🇦Canada Liam Morland Ontario, CA 🇨🇦
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024