Philadelphia
Account created on 21 January 2010, over 15 years ago
  • Senior Drupal Solutions Architect at Purple 
#

Merge Requests

More

Recent comments

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

Ah, are you using Workspaces Extra? That module provides a custom controller for the revision history that removes the operations button.

So I don't think this is an issue with Parallel workspaces but probably an issue for that project to allow revisions to be reverted.

Our team does this for a few edge cases so I know it's a feasible workflow.

🇺🇸United States djdevin Philadelphia

I thought this was already happening, where whatever revision is reverted, that just gets copied to the latest revision in the current workspace.

Does it give you an error?

🇺🇸United States djdevin Philadelphia

After looking at this a bit it's still outputting incorrectly, the eligibleRegion should have had a Country subelement.

Turning off pivoting outputs as expected:

{
    "@context": "https://schema.org",
    "@graph": [
        {
            "@type": "Product",
            "name": "Thing",
            "offers": {
                "@type": "Offer",
                "price": "500",
                "eligibleRegion": {
                    "@type": "Country",
                    "name": "USA"
                }
            }
        }
    ]
}
🇺🇸United States djdevin Philadelphia

Thinking about how we could solve this.

I thought it would be possible to just associate both revisions with the workspace, so if one revision is disassociated the other one will still be there.

But, there's a unique index on the workspace_association table that prevents this:

PRIMARY KEY (`workspace`,`target_entity_type_id`,`target_entity_id`,`target_entity_id_string`)

It's strange that only one record per entity can exist here, but in a table like node_revision, multiple revisions can be associated with a workspace. Maybe we should update the index (and the code) to work off of the revision instead.

Another option is to update deleteAssociations() to either delete associations or update them to the most recent workspaced revision. But, that's probably confusing without renaming deleteAssociations to updateOrDeleteAssociations...

🇺🇸United States djdevin Philadelphia

Added a test that illustrates the problem.

🇺🇸United States djdevin Philadelphia

Let me take a look at it, just wanted to make an issue.

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

Agreed, tie it to system.performance.js.preprocess

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

Wasn't the issue. Closing.

🇺🇸United States djdevin Philadelphia

This might be a symptom and not the cause, I can't pin it down to the coalesce.

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

Yes, the dependencies are setup correctly (mod1 depends on mod2).

On a fresh install it works fine, issue seems to be introducing autowiring during an update - seems to be no way I can get the module enabled in time before that error throws.

🇺🇸United States djdevin Philadelphia

It was part of a multifaceted approach so I'm not sure exactly how much this was part of the gained performance. I used a Drupal queue and ran multiple instances of queue-process to delete a batch of 1000 entities at a time in parallel. A custom progress bar running periodically looked at the number of items at the start and the number of items left in the queue.

Didn't have significant issues with memory, I believe it was solely the efficiency loss from running thousands of delete queries instead of 1.

I was exaggerating a little bit and don't have hard benchmarks, but at 10M (million) it went from ~200 hours to 6 hours or so. They were also entities with ~200 fields (paragraphs) so there were a ton of single field delete queries which is where I identified the issue. So deleting a single entity cost 402 queries. Deleting 1000 of them cost 402,000 (2000 entity, revision, and 2*200*1000 field queries).

With the fix, deleting 1000 entities cost 2400 (1000*2 entity, revision and 200*2 fields and revisionqueries)

At 402,000 queries per batch of 1000, the issue is exacerbated if you are using something like AWS Aurora or MySQL with replication in a durable ACID configuration, and if the database isn't local to the network. Both contribute to delay when writing tons of data.

tl;dr: Bulk queries are recommended anyway.

🇺🇸United States djdevin Philadelphia

Probably not, but likely fixes my issue.

I have another issue with openid_connect since that form is not marked as workspace safe which I assume I could just do by adding it to Workspace safe forms.

🇺🇸United States djdevin Philadelphia

The discard issue is fixed if you use WSE in combination with this change, because then it gets all the revisions associated with a Workspace and not just the ones since the current version.

📌 Simplify getAssociatedRevisions() and getAssociatedInitialRevisions() from core Active

🇺🇸United States djdevin Philadelphia

I closed Option to show all revisions on Workspace-enabled content Active in favor of this

Just wanted to note that it would be useful to show the Workspace the revision belongs to, which I think is happening in the MR!

🇺🇸United States djdevin Philadelphia

Recreated MR with failing test.

🇺🇸United States djdevin Philadelphia

djdevin changed the visibility of the branch 3473358-some-cleanup-and to hidden.

🇺🇸United States djdevin Philadelphia

If you run this in devel/php it illustrates the problem (clean D10 site)

$node = \Drupal::entityTypeManager()->getStorage('node')->create([
'title' => 'Change in the live site',
'type' => 'page',
]);
$node->save();

$stage_ws = \Drupal::entityTypeManager()->getStorage('workspace')->load('stage');

$ws_manager = \Drupal::service('workspaces.manager');
$ws_manager->setActiveWorkspace($stage_ws);

$node->title = 'Change in stage site';
$node->save();

$ws_manager->switchToLive();

$nE = \Drupal::entityTypeManager()->getStorage('node')->load($node->id());

$nR = \Drupal::service('entity.repository')->getActive('node', $node->id());

// These outputs should be the same because we are in the live workspace so we should only see the non-workspace change.
echo $nE->label();
echo "\n";
echo $nR->label();
🇺🇸United States djdevin Philadelphia

Basically just follow the steps in https://www.drupal.org/project/drupal/issues/3438083#comment-15866369 Ability to edit content in live workspace when it has been edited in other workspaces Active

It looks like it should work, but it fails with this patch. I keep getting the latest revision instead of the latest non-workspace revision.

🇺🇸United States djdevin Philadelphia

Spoke too soon, this prevents the entity repository from getting swapped out so the loaded revisions are wrong.

🇺🇸United States djdevin Philadelphia

That approach seems to conflict with jsonapi_cross_bundles but I don't know why

The website encountered an unexpected error. Try again later.
Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException: Circular reference detected for service "workspaces_parallel.entity_repository", path: "redirect.request_subscriber -> redirect.checker -> access_manager -> paramconverter_manager -> paramconverter.jsonapi.entity_uuid -> workspaces_parallel.entity_repository -> jsonapi.exception_subscriber -> jsonapi.serializer -> serializer.normalizer.resource_identifier.jsonapi_extras -> Drupal\jsonapi\ResourceType\ResourceTypeRepositoryInterface -> jsonapi_cross_bundles.cross_bundle_resource_type_repository.inner". in Drupal\Component\DependencyInjection\Container->get() (line 149 of core/lib/Drupal/Component/DependencyInjection/Container.php).
🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

This is still an issue, the problem is that the form rebuilds when you add another condition, and then the arguments end up being a string instead of an array.

This only happens if you try to add more than 1 condition to an Entity reference views component.

🇺🇸United States djdevin Philadelphia

I updated the README.md.

🇺🇸United States djdevin Philadelphia

djdevin made their first commit to this issue’s fork.

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

Thanks!

🇺🇸United States djdevin Philadelphia
🇺🇸United States djdevin Philadelphia

I fixed a few issues from my PoC in #4 and published it here: https://www.drupal.org/project/workspaces_parallel

Caveat, there is no reintegration so any live changes just get overwritten when you publish (only content that was edited within the Workspace).

🇺🇸United States djdevin Philadelphia

Patch worked once rerolled for 10.3.x, not flooding the query log anymore.

There's a tag set on config:system.menu.[name], not sure if that gets flushed if a link changes.

🇺🇸United States djdevin Philadelphia

Seems to still be called a lot, even with a warm cache. Looking at RDS insights it's still a top query under load.

General log is full of the same query, even when trying the page as anon (no page cache, just dynamic cache).

(346 is the current node being viewed)

Could be something with our site but still investigating.

🇺🇸United States djdevin Philadelphia

djdevin created an issue.

🇺🇸United States djdevin Philadelphia

I was unable to test the MR because we haven't updated to 11.x yet and it doesn't apply to 10.3.x The modernizr tweak didn't seem to work for me, but we have a ton of libraries so it could easily be another library with weights.

Something else I'm also seeing related to performance is a ton of these queries (1000s) under heavy traffic:

Very likely old cached JS/CSS calls.

Seems I can just make up some random hash and pass all the query parameters, and it will make several DB calls. Could easily DDOS the site as 4xx errors are usually not cached. Since no JS file is ever created it keeps crunching.

https://drupal/sites/default/files/css/anything.css?delta=0&language=en&...

🇺🇸United States djdevin Philadelphia

Would be happy to have you on. Thanks!

🇺🇸United States djdevin Philadelphia

I just added a print_r(array_column($group['items'], 'data')); right before it gets passed to generateHash and the order is wrong causing the hash mismatch.

🇺🇸United States djdevin Philadelphia

Seeing this in 10.3.1.

Adding a breakpoint on AssetGroupSetHashTrait::generateHash() shows a differently weighted list when it is called on the current page vs. from the .js file, so the hash_equals() in AssetControllerBase fails and it doesn't cache the file.

I don't think generateHash() is the problem because the weights are already wrong for some reason.

Trying to reproduce outside of our specific site, but it's a list of 56 JS files and the middle of the list is out of order, resulting in a different hash.

We do have a lot of library dependencies so something is not taking those into account.

Reverting to 10.2.6 caches it to disk as expected.

🇺🇸United States djdevin Philadelphia

Rebased against 11.x (applies to 10.3.x cleanly as well)

deleteFromDedicatedTables() doesn't have any test coverage - since this isn't a bug I wonder if having coverage through SqlContentEntityStorageSchemaTest is enough?

Production build 0.71.5 2024