Contrib.social

tome-1.9 breaks static generation on CLI (regression)

Open on Drupal.org →
Created on 19 January 2023, almost 2 years ago
Updated 30 January 2024, 11 months ago

Problem/Motivation

The update to >= tome-1.9 breaks static site generation for me. Using the webGUI however works.

Steps to reproduce

drush -l my.domain.org tome:static
[warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:312
Generating static HTML...
 120/120 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

[ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/,/contact,/facets-block-ajax,/sitemap.xml,/sitemap.xsl' '--return-json' '--process-count=5'
         '--uri=my.domain.org'" after failure...

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/rss.xml,/node,/veranstaltungen-colloquien-old,/veranstaltungen-colloquien,_entity:node:de:1'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/,/contact,/facets-block-ajax,/sitemap.xml,/sitemap.xsl' '--return-json' '--process-count=5'
         '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:9,_entity:node:de:8,_entity:node:de:21,_entity:node:de:22,_entity:node:de:23'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/rss.xml,/node,/veranstaltungen-colloquien-old,/veranstaltungen-colloquien,_entity:node:de:1'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:14,_entity:node:de:66,_entity:node:de:18,_entity:node:de:68,_entity:node:de:2'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:9,_entity:node:de:8,_entity:node:de:21,_entity:node:de:22,_entity:node:de:23'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:27,_entity:node:de:48,_entity:node:de:42,_entity:node:de:50,_entity:node:de:39'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:14,_entity:node:de:66,_entity:node:de:18,_entity:node:de:68,_entity:node:de:2'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:26,_entity:node:de:36,_entity:node:de:32,_entity:node:de:45,_entity:node:de:53'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:27,_entity:node:de:48,_entity:node:de:42,_entity:node:de:50,_entity:node:de:39'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:41,_entity:node:de:33,_entity:node:de:54,_entity:node:de:38,_entity:node:de:34'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:26,_entity:node:de:36,_entity:node:de:32,_entity:node:de:45,_entity:node:de:53'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:37,_entity:node:de:46,_entity:node:de:47,_entity:node:de:55,_entity:node:de:43'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:41,_entity:node:de:33,_entity:node:de:54,_entity:node:de:38,_entity:node:de:34'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:35,_entity:node:de:31,_entity:node:de:44,_entity:node:de:52,_entity:node:de:56'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:37,_entity:node:de:46,_entity:node:de:47,_entity:node:de:55,_entity:node:de:43'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:40,_entity:node:de:77,_entity:node:de:76,_entity:node:de:78,_entity:node:de:74'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:35,_entity:node:de:31,_entity:node:de:44,_entity:node:de:52,_entity:node:de:56'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:73,_entity:node:de:75,_entity:node:de:49,_entity:node:de:51,_entity:node:de:80'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:40,_entity:node:de:77,_entity:node:de:76,_entity:node:de:78,_entity:node:de:74'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:79,_entity:node:de:81,_entity:node:de:94,_entity:node:de:19,_entity:node:de:84'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:73,_entity:node:de:75,_entity:node:de:49,_entity:node:de:51,_entity:node:de:80'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:95,_entity:node:de:20,_entity:node:de:98,_entity:node:de:25,_entity:node:de:28'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:79,_entity:node:de:81,_entity:node:de:94,_entity:node:de:19,_entity:node:de:84'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:29,_entity:node:de:30,_entity:node:de:24,_entity:node:de:61,_entity:node:de:4'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:95,_entity:node:de:20,_entity:node:de:98,_entity:node:de:25,_entity:node:de:28'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:70,_entity:node:de:97,_entity:node:de:64,_entity:node:de:67,_entity:node:de:82'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:29,_entity:node:de:30,_entity:node:de:24,_entity:node:de:61,_entity:node:de:4'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:85,_entity:node:de:86,_entity:node:de:87,_entity:node:de:88,_entity:node:de:89'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:70,_entity:node:de:97,_entity:node:de:64,_entity:node:de:67,_entity:node:de:82'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:90,_entity:node:de:91,_entity:node:de:92,_entity:node:de:93,_entity:node:de:83'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:85,_entity:node:de:86,_entity:node:de:87,_entity:node:de:88,_entity:node:de:89'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:63,_entity:node:de:57,_entity:taxonomy_term:de:1,_entity:taxonomy_term:de:2,_entity:
         taxonomy_term:de:3' '--return-json' '--process-count=5' '--uri=my.domain.org'" after
         failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:90,_entity:node:de:91,_entity:node:de:92,_entity:node:de:93,_entity:node:de:83'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:taxonomy_term:de:4,_entity:taxonomy_term:de:8,_entity:taxonomy_term:de:9,_entity:taxonomy_te
         rm:de:10,_entity:user:de:3' '--return-json' '--process-count=5' '--uri=my.domain.org'" after
         failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:node:de:63,_entity:node:de:57,_entity:taxonomy_term:de:1,_entity:taxonomy_term:de:2,_entity:
         taxonomy_term:de:3' '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:user:de:6,_entity:user:de:12,_entity:user:de:17,_entity:user:de:19,_entity:user:de:403'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:taxonomy_term:de:4,_entity:taxonomy_term:de:8,_entity:taxonomy_term:de:9,_entity:taxonomy_te
         rm:de:10,_entity:user:de:3' '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:user:de:694,_entity:user:de:6534,_entity:user:de:9787,_entity:user:de:16707,_entity:user:de:
         22280' '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:user:de:6,_entity:user:de:12,_entity:user:de:17,_entity:user:de:19,_entity:user:de:403'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:user:de:22573,_entity:user:de:26816,_entity:user:de:35466,_entity:user:de:35468,_entity:user
         :de:39577' '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:user:de:694,_entity:user:de:6534,_entity:user:de:9787,_entity:user:de:16707,_entity:user:de:
         22280' '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/de,/de/contact,/de/facets-block-ajax,/de/sitemap.xml,/de/sitemap.xsl' '--return-json'
         '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '_entity:user:de:22573,_entity:user:de:26816,_entity:user:de:35466,_entity:user:de:35468,_entity:user
         :de:39577' '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Retrying "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/de/rss.xml,/de/node,/de/veranstaltungen-colloquien-old,/de/veranstaltungen-colloquien'
         '--return-json' '--process-count=5' '--uri=my.domain.org'" after failure...

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/de,/de/contact,/de/facets-block-ajax,/de/sitemap.xml,/de/sitemap.xsl' '--return-json'
         '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [ERROR] Error when running "'/usr/local/bin/drush -y -l my.domain.org' 'tome:static-export-path'
         '/de/rss.xml,/de/node,/de/veranstaltungen-colloquien-old,/de/veranstaltungen-colloquien'
         '--return-json' '--process-count=5' '--uri=my.domain.org'":
           sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found

 [OK] Exported static HTML and related assets.

(The warnings have always been there, this is not related, have not found a fix so far)

The steps 1 to 100 are much too fast, nothing is actually done. On a side note: The final statement should not be "OK".

Proposed resolution

Using the webGUI, it works.

If I undo the changes in
https://git.drupalcode.org/project/tome/-/commit/d50d1e8efdf8435cc5f71ae...
to modules/tome_static/src/Commands/StaticCommand.php

and re-add the escapeshellarg() parts:
$command = $this->executable . ' tome:static-export-path ' . escapeshellarg(implode(',', $chunk)) . ' --return-json --process-count=' . escapeshellarg($process_count) . ' --uri=' . escapeshellarg($uri);
it works again.

I am unsure what this does, but it is necessary. The paths I see getting handled by issuing "-v" do not contain any suspicious characters that would seem to need escaping.

🐛 Bug report
Status

Needs review

Version

1.9

Component

Tome Static

Created by

🇦🇹Austria tgoeg

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment almost 2 years ago →
    🇦🇹Austria paper boy

    Ran into the same problem with Tome 8.x-1.10 and Drupal 9.5.4 on a site still running on PHP7.4 (fpm and cli). I am not sure, if the same problem exists with PHP8.x.

    I'm working on a host with multiple PHP versions, so I'm calling Tome static export via CLI from project root directory with the following command:

    php7.4 vendor/bin/drush --root=/var/www/sites/mysite tome:static --uri=https://www.example.com

    Reverting back to Tome 8.x-1.8 fixes the problem.

  • Comment over 1 year ago →
    🇺🇸United States firewaller

    +1

  • Status changed to Needs review over 1 year ago
  • Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.3 & MySQL 5.7
    last update over 1 year ago
    Waiting for branch to pass
  • Comment over 1 year ago →
    🇦🇹Austria tgoeg

    I think I have found the problem.
    A careful look at my initial report above shows
    sh: 1: exec: /usr/local/bin/drush -y -l my.domain.org: not found
    That is to say, sh tries to exec() a binary that is called '/usr/local/bin/drush -y -l my.domain.org', which does not exist, the binary is only the first part of this string.
    That's what escapeshellargs() seems to have masked (not fixed, really).

    So to reproduce, one has to call drush <some drush parameter> tome:static or it will succeed.
    I have to use -l my.domain.org as this is a multisite setup.

    The attached patch fixes it for me (although I don't get any actual output on Drupal 10 + Drush 12 despite the patches in the issue queue, but I think this is another issue; at least it finishes now.).

    What it basically does is it conserves any provided parameters/switches in a separate array and does not merge them into the executable string.

    I guess the test modules/tome_base/tests/src/Unit/ExecutableFinderTraitTest.php needs to be adapted as well, as I changed the return type in ExecutableFinderTrait.php and also, it seems to not have covered this case.
    I am however just a sysadmin, so I cannot provide any tests (no clue how this works, sorry).

    Please also have a good review regarding the code presented - it might be a crude solution.

    However, feedback would be welcome if it basically tackles the root problem for others as well!

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    47 pass, 9 fail
  • Comment 11 months ago →
    🇦🇺Australia cafuego

    Your patch definitely fixes the issue at hand. It still applies to newer versions of tome as well, so I expect they cause the same problem if not patched.

    I'd try with newer versions of tome, but v1.12 won't install as-is in Drupal 9 and once I do make it install by bodging the info yaml, the drush commands seem to not get loaded.

  • Merge request !18Issue #3334879: Static geneation via CLI is broken. → (Open) created by cafuego
  • Open on Drupal.org →
    Core: 9.5.x + Environment: PHP 7.3 & MySQL 5.7
    last update 11 months ago
    Waiting for branch to pass
contrib.social Blog FAQ Discussions
Production build 0.71.5 2024