Batch mode does not allow other modules to alter query

Spent several hours debugging 🐛 _access tag can leading to fatal error Closed: won't fix that lead to creating 🐛 Query metadata get lost when $view->query->query() is called multiple times during page build Active .

I was going to propose a similar change to #12, but surprised people reporting it as problematic as my understanding was quite firm on that being the correct approach. It would be awesome if people can share details about what that approach breaks.

I completely share the concerns of #23 around performance , but mostly afraid of the failing tests . Quick patch test however succeeded, but that is only a quick test full test suite is running and I will report back here later.

I don't quite understand the purpose of modifying cache configuration of the view, so proposing a variation of #10 without changing cache settings , but with restoration of the original configuration as I believe this is crucial part that was missing in the original implementation - without it modified view is passed further for batch processing.

Remove unused use statement.

Test suite we have on a site has passed many times with patch from #25.

I use views_data_export:^1.2 and have more than 10K row and 5 column to export. three of this 5 column is views_field_view which fetch data from different views.
without any patch I can not export any thing. by using #25 patch problem solved.
but I have to use batch size = 100. any batch size more than that result in an error. by this batch size it takes me 30 to 40 minutes.

#25 looks reasonable.

This seems tricky to debug and reason about, so a test would be really good. This would also establish that it was fine to follow #25 in not unsetting the cache.

#25 fixed the issue for me.

#25 fixed the issue for me too. It wasn't clear if that causes a re-execution of the query or not. Can someone confirm?

#25 fixes the issue for me as well.

Those who are experiencing data inconsistency issue in batch as compared to standard mode.

I've identified the source of the batch export limit problem. It has to do with the sequence of query results.

In order to prevent changes in the query result sequence, add the unique fields to the sort.

I've tried patch #25 and worked well. The issue I had was that the view exposed filters weren't taken in countQuery() and therefore it always resulted into max number of rows (I have more than 100k in a table).

Acctually $view->build(); helped a lot which added the filters from the request.

I can confirm that patch #25 resolved the bug
Thank you @rosk0

#25 worked for me as well. Thanks!

Rebased #25 patch did not apply cleanly after changes in 8.x-1.x (use statement order changes only) and opened MR.

Just rerolled patch from comment #25

steven jones → made their first commit to this issue’s fork.

Fixing the title of this issue, because presumably we do all the query to be altered, just that there's something about the query metadata that's not quite right.

I 100% agree with #28 in that we should absolutely have a test for this.

I guess we need a module that implements a hook_query_alter based on some metadata that doesn't get currently passed correctly by the existing code.

FYI, the patch in #39 has lot more stuff than needed.

Attaching the new path which works with v1.5.0