I had the same issue that might be relevant and came across this issue. Adding an comment incase anybody has the same problem :)
I had a site with a lot of subdirectories and files (> 10GB)
The maximum records limit was set too low
I think the limit was exhausted on the root directory so it appeared as though the subdirectories were not being evaluated
Increasing the limit fixed my problem