When using batch_size in the source config, it creates an infinite loop.
It seems to import everything in a first batch then loops infinitely on batches containing the whole dataset.
Here is an example migration that can reproduce the problem :
id: folio_migrate_pays
label: Pays
migration_group: folio
source:
plugin: custom_sql_query
key: folio_catalogue
keys:
- id
track_changes: true
batch_size: 2
sql_query:
#language=mysql
'SELECT id, name FROM v_countries'
destination:
plugin: 'entity:taxonomy_term'
default_bundle: pays
process:
name: name
field_identifiant_catalogue: id
dependencies:
enforced:
module:
- folio_migrate
If I call drush migrate:import folio_migrate_pays --update, it hangs forever.
If I look at what it is doing, it seems it is calling CustomSQLQuery::fetchNextBatch() again and again forever.
In the code, it looks like $batchSize is never really used.
SqlBase uses it to add a range to the query. However, in our case I am not sure that's possible, since the query is not manipulated in PHP.
Active
1.3
Code
Issue #3347956: batch_size creates an infinite loop
I removed support for batch_size since it is broken.
We can always add it back later if we find a way to make it work.