GitLab Merge Requests Unable to Generate Incremental Patch Files

Created on 19 March 2021, over 3 years ago
Updated 4 May 2023, about 1 year ago

Problem/Motivation

When working with Drupal, it is common to apply patches to core or contrib modules in order to fix a bug or test someone elseโ€™s code change before the change is committed and included in a release.

This is relatively straightforward when working with patch files, but when utilizing GitLabโ€™s merge request (MR) functionality, the problem is that the diff URL is not stable. Once multiple commits are pushed to an MR, it becomes complex to generate a patch file that does not include either a single commit or the entire changeset of the MR.

Steps to reproduce

Given an MR (https://git.drupalcode.org/project/commerce_migrate/-/merge_requests/3), it's possible to get a diff or patch file for the entire MR changeset by appending a .diff or .patch extension:

However, the following scenario can occur:

  1. Alice finds a bug. She creates an issue fork on drupal.org, pushes her branch 1234-alice, and creates a MR to fix it. It contains more than one commit.
  2. Alice pushes an additional commit to the MR.
  3. GitLab generates a diff for the MR as well as exposes a patch file at (https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31527.patch) so Alice can include it via whatever patch workflow she chooses.
  4. Clare reviews the MR but sees what she thinks is a potential improvement. She pushes a change to the MRโ€™s 1234-alice branch.
  5. Sarah finds this issue in the issue queue and wants to test the fix. She accesses the patch generated by GitLab. It contains the entirety of Alice and Clareโ€™s changes.
  6. Sarah determines that Clareโ€™s changes actually break the fix proposed in the MR. She wants to test just Aliceโ€™s original changes. GitLab is able to provide a patch file for a specific commit or the entire MR, but not a subset of the commits on an MR.

Proposed resolutions

  1. GitLab does have the ability to display a diff between two specified commits, but this view does not support generating a patch file via appending .patch to the URL. @moshe has created a GitLab issue to propose this functionality: Allow .patch to be appended to any Compare URL. There are some similar proposals as well including "Compare" View: Export compare data in .diff format.
  2. This could be positioned as a GitLab bug and pursued in that way.

Note: During the GitLab Acceleration Initiative โ†’ meeting on 9/8/21 it was discussed and generally agreed upon that we should avoid potential resolutions that resulted in โ€œspecial snowflaking the architectureโ€ such as generating patch files that get posted back to the D.O. issue as GitLab activity.

Workarounds

  1. If you'd like a static patch file for an MR at a given point in time, create a patches folder in your project next to your global composer.json, download the diff into this folder and then use composer-patches to apply it from there: "Example patch entry from file": "patches/drupal.ISSUE.SHA.brief-description.patch"
  2. Use composer-patches' fix for Patches from drupal.org merge request URLs are dangerous?: Add support for patch checksums. If the MR changes, the new patch won't be applied because the checksum doesn't match. This will have the effect of breaking building that run into this, which will avoid the stability and security problems. This is included in version 2. However, at the time of this writing, v2 hasn't been released yet.
๐Ÿ› Bug report
Status

Active

Version

3.0

Component

GitLab integration

Created by

๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom joachim

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • ๐Ÿ‡ฆ๐Ÿ‡บAustralia gordon Melbourne

    @eduardo-morales-alberti yes I think that this is the best idea.

    I am a Drupal dev and system implementor and use the composer-patches to implement fixes. So using uploaded patches is just so easy to do. Where as working with the a MR branch is so much more difficult and also applying this to a branch which is not the dev branch is just very difficult.

    Options I have considered is:-

    1. being able to download a patch generated from the gitlab user interface. However this doesn't seem possible.
    2. Pin the release to a commit, this is possible but it means you are locked to the version when the branch was created, and you can't apply multiple issues.

    If we created patches it would solve a lot more issues with the using of the patches in system deployments.

    1. Having a patch means you may be able to apply to a stable version.
    2. you can apply multiple patches to the same project/module
    3. Have the posibility to use patches with older version of a module.
    4. Makes it easier for people who do not use gitlab to look at patches and see what the changes are.
    5. Some people uploading patches or using existing patches are doing this as a patch of a larger project who have dedicated testers and are doing more intense testing that what most developers will do.

    FYI I still upload patches to d.o because I am generally working with clients to fix an issue, and uploading a patch means that it will get fixed upstream and the fix is not required to be maintained by my client.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada colan Toronto ๐Ÿ‡จ๐Ÿ‡ฆ

    Added section for workarounds to the IS:

    1. If you'd like a static patch file for an MR at a given point in time, download the diff, and then upload it as an attachment to a comment on the issue. This patch file can then be used by Composer in composer.json. The problem is that this option adds more comments to the issue, which don't directly help solve it (i.e. noise).
    2. Use composer-patches' fix for Patches from drupal.org merge request URLs are dangerous?: Add support for patch checksums. If the MR changes, the new patch won't be applied because the checksum doesn't match. This will have the effect of breaking building that run into this, which will avoid the stability and security problems. This is included in version 2. However, at the time of this writing, v2 hasn't been released yet.
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom joachim

    > If you'd like a static patch file for an MR at a given point in time, download the diff, and then upload it as an attachment to a comment on the issue.

    I download the diff and commit it to the project repo in a /patches folder, and then specify the patch in composer.json as a relative path.

  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany FeyP

    I download the diff and commit it to the project repo in a /patches folder, and then specify the patch in composer.json as a relative path.

    This is the way to go. I don't think we want to promote that people upload patches of various states of MRs just to apply them to composer. That would just add more noise to the issue and complicate finding the latest code to review/commit. Updated the IS.

  • ๐Ÿ‡ฉ๐Ÿ‡ฐDenmark ressa Copenhagen

    Thanks for sharing your simple and pragmatic solution @joachim, and @FeyP for adding it in the IS.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States nicxvan

    While I agree that adding the file to a comment can add noise, I think my point in #36 still stands. Having a way to track which points in time the community has tested is helpful.

    When there is a patch submitted for each change it's easier to do an interdiff if there was a regression. If you want to do that purely with people downloading the diff themselves, you have to compare the comment time with commit history and guess when they downloaded their version of the diff.

    It also helps in another way, for projects that are in maintenance mode sometimes a patch is only looked at again when it fails to apply in CI / CD. If I have a patch from an early comment in an issue that no longer applies and there are three patches between that and the newest I can check the comments more clearly with the progress of the patch and testing. If it's just an MR and everyone has been downloading the diff as they go correlating the comment history with the commit history is more difficult.

    Another thing that makes point in time patching helpful is if a new patch has regressions. Imagine the following scenario:

    Patch-1 (Patch I initially applied fixing my issue)
    Patch-2 (New progress)
    Patch-3 (Regression in patch)

    If I am using Patch 1 and it no longer applies, if there are point in time patches I can see patch 3 has a regression from other user comments and I can attempt to apply patch 2 that other community members have stated still works. If it's an MR only, I again have to try to guess which commit put the regression in and apply that change instead.

    I don't think generating a static patch per commit or push is helpful, but being able to click a button to generate a patch in time automatically would be a huge improvement ( assuming there is some check to ensure you're not generating duplicate patches which is unhelpful in all scenarios).

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom joachim

    > When there is a patch submitted for each change it's easier to do an interdiff if there was a regression. If you want to do that purely with people downloading the diff themselves, you have to compare the comment time with commit history and guess when they downloaded their version of the diff.

    I recommend using a filename for the patch in a project repo that contains all of: Drupal project name, issue number, and SHA of the branch tip at the point the patch was made. E.g. 'drupal.ISSUE.SHA.brief-description.patch.

    Patches that people upload are *probably* fresh, but it's a bit fiddly to compare the date of the comment the file is on with the dates in the git commit log. AND if the MR gets rebased -- as it should be -- don't the commits get new dates?

  • ๐Ÿ‡ฉ๐Ÿ‡ฐDenmark ressa Copenhagen

    It would be awesome if patches were offered as files in the format of drupal.ISSUE.SHA.issue-title-lower-cased-and-shortened.patch. That would probably require adjusting Gitlab code though, which may not be possible ...

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom joachim

    Yeah, my impressions is that github doesn't allow much customisation -- it's why we have this issue in the first place!

    I meant that I manually rename the patch file.

    Though... it's something that could be automated as a Composer command.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States nicxvan

    @joachim, that solution still works just on individual / team for finding that info. It still loses the community aspect of testing and knowing what point in time others have tested.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada colan Toronto ๐Ÿ‡จ๐Ÿ‡ฆ

    โ˜๏ธ

Production build 0.69.0 2024