Contrib.social

Pths are exported multiple times with path counts > 1

Open on Drupal.org →
Created on 4 October 2024, 3 months ago

Problem/Motivation

We're investigating how to get Tome exports to execute in less time to make a static copy of our site, which is about 2000 pages. Like others, we've used the `path-count=1` parameter since anything higher results in menu blocks showing on the wrong pages and other issues.

I found a couple of issues to fix

1. Tome doesn't seem to keep track of paths which are queued for export. When the path and process counts are higher than one, those paths can be "discovered" as invoked paths. As a result, it'll be crawled and exported more than once, with potentially unexpected results in the end.

2. The invoked paths returned during static generation can include the same path more than once with these params, leading to more duplicated crawling

3. The menu active trail cache has to be cleared, not reset between requests for menu blocks to behave as expected. Otherwise, all the paths crawled by a process re-use cached versions of menu blocks from the first or earlier paths.

Steps to reproduce

Run the tome export with path-count > 1. You should see multiple passes of it exporting pages. When path-count is 1, we only see 2 export passes, one for pages and another for CSS, JS, image paths - mostly.

Proposed resolution

Merge request incoming.

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

🐛 Bug report
Status

Active

Version

1.0

Component

Code

Created by

🇺🇸United States omerida

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @omerida
  • Comment 3 months ago →
    🇺🇸United States omerida
  • Merge request !28Fixes bugs when path count is > 1 → (Open) created by omerida
  • Comment 3 months ago →
    🇺🇸United States omerida

    Attached is a patch file with the proposed changes:

    1. Adds an internal $init_paths array to tome_static/src/Commands/StaticCommand.php to track the initial paths requested for export. When we get invoked paths in exportPaths(), filter out any invoke_paths that are already in the initial paths.

    2. In tome_static/src/Commands/StaticExportPathCommand.php, run $invoke_paths through array_unique() to remove duplicates.

    3. In tome_static/src/RequestPreparer.php, I stepped through runs with path count > 1 and found that the menuActiveTrail has to be cleared, not reset, to ensure menu blocks are correctly generated for each path.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024