Test-only changes reverts changes to test modules

The current logic for reverting/deleting files is based on a very simple regex and some git diff filtering options.
The code is here: https://git.drupalcode.org/project/drupal/-/blob/11.x/.gitlab-ci/pipelin...

      echo "1️⃣ Reverting non test changes"
      if [[ $(git diff refs/heads/${TARGET_BRANCH} --diff-filter=DM --name-only|grep -Ev "Test.php$"|grep -v .gitlab-ci|grep -v scripts/run-tests.sh) ]]; then
        git diff refs/heads/${TARGET_BRANCH} --diff-filter=DM --name-only|grep -Ev "Test.php$"|grep -v .gitlab-ci|grep -v scripts/run-tests.sh|while read file;do
          echo "↩️ Reverting $file";
          git checkout refs/heads/${TARGET_BRANCH} -- $file;
        done
      fi
      if [[ $(git diff refs/heads/${TARGET_BRANCH} --diff-filter=A --name-only|grep -Ev "Test.php$"|grep -v .gitlab-ci|grep -v scripts/run-tests.sh) ]]; then
        git diff refs/heads/${TARGET_BRANCH} --diff-filter=A --name-only|grep -Ev "Test.php$"|grep -v .gitlab-ci|grep -v scripts/run-tests.sh|while read file;do
          echo "🗑️️ Deleting $file";
          git rm $file;
        done
      fi

Can we change the regex to filter on paths matching */tests/* or something like that?

If I am not mistaken, another shortcoming is that if you do changes to an abstract test base class, that is reverted too. But you may want to test-only based on the base class changes.

We can surely change the regex. We knew from the beginning that it'd be tricky to cover 100% of the cases. The current implementation covers a good percentage but obviously not all.

We can open an MR and start experimenting with the "test-only" job. We can test changes by modifying additional files in the same MR that will eventually be reverted.

I did a first MR showing where the changes would need to happen in the gitlabci file and also changed some other files so we can start debugging whether those files should be part of the test only or not. Right now I didn't force any errors, I just made an example.

I don't think I can follow up on this task, at least this week, but I'm happy to review or guide if needed to the best of my knowledge.

That MR is just a starting point.

Needs further testing, but some review would be great.

.gitlab-ci/pipeline.yml
core/modules/book/book.module
core/modules/book/tests/src/Unit/Menu/BookLocalTasksTest.php
If this list contains more files than what you changed, then you need to rebase your branch.
1️⃣ Reverting non test changes
↩️ Reverting core/modules/book/book.module

But the issue is when a change is made to a module in the test folder right? Could we add a change to see?

As I mentioned on #7, I didn't have more time so couldn't follow up because I needed to jump on other things.

I managed to do changes as requested in #9 and it seems to be working well: https://git.drupalcode.org/issue/drupal-3395977/-/jobs/353569

Note that I reopened 🐛 Test-only job shouldn't require constant rebases to detect which files were changed. Active so some of the changes seen in the MR are due to that, and of course, all those dummy changes that will be removed :-)

This looks good! Once those changes are reverted can RTBC.

All the test changes are now reverted and only the actual fix is left in the MR. Ready for final review.

Think this is good!

Committed/pushed to 11.x and cherry-picked to 10.2.x, thanks!

Automatically closed - issue fixed for 2 weeks with no activity.