Exclude fast_404 paths and file extensions by default (Performance)

As a first step it might make sense to inform users on the search404 configuration page and in the README.md that configuring fast_404 in settings.php in this combination is still recommended?

Any maintainer oppinions here?

Thinking about this for a while, I think this might be a really heavy performance trap. If only some file-URLs or .map files are requested by the browser are handled by search404 that might have huge impact on the database due to the search queries?

The input field is limited to 128 characters. We also need to remove that until the core issue is solved.

I've created a separate issue to remove the #maxlength limit on the input fields, as mentioned in #9: 🐛 Settings form fields #maxlength limit=128 (from core) is too short Fixed

I guess, we can close this issue in favour of 📌 Add option to exclude all non whitelisted file paths (Performance) Needs review ?

As, we will simply disallow all file extensions there.

Well I think we should still add the paths that core used to set in fast_404 before. Having a predefined list of useful suggestions if the "exclude all" still makes sense and aligns with the committed core change referenced.

But we can set priority to normal here.

TBD:
0. Wait for 🐛 Settings form fields #maxlength limit=128 (from core) is too short Fixed to be merged, as otherwise the value will be cut off on next settings form save.
1. Check the list of file suffixes to align with fast_404 and typical file etensions
2. Write an update hook to add these values and inform the user, if the value is equal to the old default value
3. Ensure we have some tests for these extensions in place

Postponing this on 🐛 Settings form fields #maxlength limit=128 (from core) is too short Fixed
Still 1. and 2. can already be implemented and shouldn't be too hard

Unpostponing, as 🐛 Settings form fields #maxlength limit=128 (from core) is too short Fixed is fixed.

Linking semi-related issue/improvement

Please see / follow: ✨ Add minimal 4xx responses for invalid image derivative requests RTBC

Should all be moved into ignore now, with the new deny logic.

Back to needs work. Rereading the issue summary, this issue is mainly about the performance:

For that heavy performance impact, I'm categorizing this as bug.

Still, it will trigger a 404 search for such a file, which results in expensive database queries

Having these extensions defined under "search404_ignore_extensions" doesn't make sense performance wise. The extension will simply be removed from the search query, but the "expensive database query" will still apply, as "search404_ignore_extensions" doesn't abort the search.

I'd say, it would make more sense here to define it in "search404_deny_specific_file_extensions". But since "search404_deny_all_file_extensions" now also exists (see 📌 Add option to exclude all non whitelisted file paths (Performance) Needs review ), I suggest, we simply close this issue.

Thanks @Grevil, I agree with the point from #18

It might be worth adding it to

search404_deny_specific_file_extensions

but the key difference is, that there should be no error message, instead we need to return file not found (404).

That might be a further option or just a boolean for the deny-functionality... Things become quite complicated here.

I think a fix here would also fix a user experience (non performance) issue— missing assets can result in a Drupal message about searching for it!

"The page you requested does not exist. For your convenience, a search was performed using the query themes+example+css+example+css+map."

@mlncn I agree, so I opened https://www.drupal.org/project/search404/issues/3480706 ✨ Ignore typical system paths and file endings by default Active for that.