Fatal error: Allowed memory size of 268435456 bytes exhausted in config_views.views.inc on line 149

Oh my goodness, I think the solution may be simple. I don't know why I didn't see this before.

As we recurse through all the configuration, it appears we create new copies of the full compilation of the site's configuration, a large array to be sure. If instead we passed our compiled arrays by reference, we would avoid stamping out a new copy of the array in each instance.

So, in short, let's modify the method to include the compiled data by reference.

@cosmicdreams This change makes sense. Are you guessing that this will resolve it, or does it resolve it? I am not having the out of memory errors so I can not test.

When we had errors, it wasn't with a typical Drupal build. You needed to have a complex build, preferably with facets or a module like it that would have a lot of complexity with it's configuration. I got busy the last couple of days but I do have such a database I could test with. I'm just calling for some of the others who have tested in the past to give it another go.

I'm still seeing the same out of memory error with either the exclude facets patch in #16 or the by-reference changes in MR 2 / #20. Site has lots of config, and does use facets.

PHP Fatal error: Allowed memory size of 939524096 bytes exhausted (tried to allocate 24576 bytes) in /var/www/web/modules/contrib/config_views/config_views.views.inc on line 156

Thanks for the test, and sheesh that's a large memory size. I suppose this doesn't fix the run away memory usage after all.

Debugging this today with @beeyayjay and we discovered that the problem has to do with circular dependencies in a module's configuration schema. For example the Fallback Formatter module which our site uses has the config schema:

field.formatter.settings.fallback:
  type: mapping
  label: 'Fallback formatter settings'
  mapping:
    formatters:
      type: sequence
      label: 'Formatters'
      sequence:
        - type: mapping
          label: 'Formatter settings'
          mapping:
            status:
              type: boolean
              label: 'Status'
            weight:
              type: integer
              label: 'Weight'
            formatter:
              type: string
              label: 'Formatter'
            settings:
              type: field.formatter.settings.[%parent.formatter]
              label: 'Settings'

And when we added some debugging output to _views_views_config_process_schema to track the config definition label as it recurses, we see a progression like:

//Block////Field formatter/Settings
//Block////Field formatter/Settings/Fallback formatter settings
//Block////Field formatter/Settings/Fallback formatter settings/Formatters
//Block////Field formatter/Settings/Fallback formatter settings/Formatters/Formatter settings
//Block////Field formatter/Settings/Fallback formatter settings/Formatters/Formatter settings/Settings
//Block////Field formatter/Settings/Fallback formatter settings/Formatters/Formatter settings/Settings/Fallback formatter settings
//Block////Field formatter/Settings/Fallback formatter settings/Formatters/Formatter settings/Settings/Fallback formatter settings/Formatters
...

Our initial proof of concept fix was to keep a history array of the config definition labels and to confirm the current label is not present before proceeding. This fixed the memory error but broke certain config from showing in views. I'm still looking into why exactly it broke, but I'd guess some config definitions are nested more than 1 cycle deep.

Our second attempt allows the label to exist once, but not twice in the history. It fixed the memory issue and we don't see any problems where we're using config_views - but our choice of allowing 2 cycles is arbitrary and I'd imagine some modules could need more. So, some refinement may be needed. Or, maybe this is just helpful to reveal the nature of the problem and will illuminate a better solution.

FWIW, modules we ran into trouble with were Fallback Formatter, and Rules. Even with the attached fix Rules still cycles hundreds of times but doesn't crash.

That's great insight. I thought we had tried memorization already.

It seems like, in the case of this fallback field formatter, the view we wish it could create is:

View Fields:
- formatters

Where a formatter comprises of all the fields a formatter has.

This has me thinking that the logic should be broken up to have a different strategy for "mapping". Anything that is a mapping should be popped out of the all_definitions list and processed. Because it's like any entity/object that could be referred by other things. Kind of like how a view can have references.

That's just a quick thought without too much digging into all the impacts that would cause.

I revised the #25 patch, to compare not just the label, but an md5 hash of the entire $definition array. It can now stop on the first cycle without the issues @msielski described. I ran it with debug code to output the definition label when it breaks out of loops, and only labels associated with the problematic modules (Fallback Formatter, and Rules) were reported, so it seems to be working as expected.

A separate array of "mapping" objects that can be referenced by other definitions as @cosmicdreams described makes a lot of sense, but this patch should prevent memory errors in the meantime.

@beeyayjay this was a clever solution. I have tested the patch and confirm that it did not break anything. I have not encountered the original error from this issue so I can not say for sure whether it solved it, other than msielski and beeyayjay are on the same project that does have this problem and it apparently resolved it there.

Great progress. Thanks for the reminder to check this. I've got a large database that I can test against to give a second opinion. If everything checks out I'll merge.

Finally got a round to test this with my massive database filled with facets and complex configuration. I see proof positive of the enhancment here.

Without the patch: Site crashes upon attempting it's initial compilation of config (while enabling the module)
With the patch: Site takes a LONG time to compile the config but does not crash.

I have created a Merge Request containing your changes and am updating the issue to give credit to those involved.

Let's get this in today.

Thanks @cosmicdreams for moving this along. Nice work everybody.

Automatically closed - issue fixed for 2 weeks with no activity.