GitLabCI does not allow MRs from forks opened by non-members to run in the parent project

Created on 17 May 2023, about 1 year ago
Updated 22 September 2023, 9 months ago

Problem/Motivation

In DrupalCI today, we trigger testing when a user uploads a patch, opens an MR, or hits the manual add-test button.

In GitLabCI, we have configured a template with the following workflow rules to trigger testing:

workflow:
  rules:
  # These 3 rules from https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml
    # Run on merge requests
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
    # Run on tags
    - if: $CI_COMMIT_TAG
    # Run when called from an upstream pipeline https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html?tab=Multi-project+pipeline#use-rules-to-control-downstream-pipeline-jobs
    - if: $CI_PIPELINE_SOURCE == 'pipeline'
    - if: $CI_PIPELINE_SOURCE == 'parent-child'
    # Run on commits to the default branch
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    # The last rule above blocks manual and scheduled pipelines on non-default branch. The rule below allows them:
    - if: $CI_PIPELINE_SOURCE == "schedule"
    # Run if triggered from Web using 'Run Pipelines'
    - if: $CI_PIPELINE_SOURCE == "web"
     # Run if triggered from WebIDE
    - if: $CI_PIPELINE_SOURCE == "webide"   

However, GitLab CI places limits on automated pipeline runs based on the permissions of the user running the pipeline:

https://git.drupalcode.org/help/ci/pipelines/merge_request_pipelines.md#...

Specifically:

Project members in the parent project can trigger a merge request pipeline for a merge request submitted from a fork project. This pipeline:

  • Is created and runs in the parent (target) project, not the fork (source) project.
  • Uses the CI/CD configuration present in the fork project's branch.
  • Uses the parent project's CI/CD settings, resources, and project CI/CD variables.
  • Uses the permissions of the parent project member that triggers the pipeline.

Because our collaboration model has collaborators working on a fork without granting them any permissions on the parent project, it means that despite the workflow trigger for running on an open MR - the pipeline will not run if the user opening the MR isn't a project member with sufficient permissions.

The only ways for an MR opened by a non-member to run is:

  • To run the pipeline in the fork itself, where the non-member of the parent project does have permissions
  • For a project member to 'run pipeline' on the merge request manually.
  • (maybe)For a bot user added as a project member to run the pipeline based on a webhook event.

Steps to reproduce

To show that the current configuration is using the merge request trigger correctly:

  • As a project member:
    • open a fork
    • make changes
    • and submit an MR back to the project

The pipelines should run in the parent correctly

To show that the current configuration does not allow non-members to trigger pipelines

  • As a non-member of the project:
    • open a fork
    • make changes
    • and submit an MR back to the project

Note that the MR is open but pipelines have not run.

Proposed resolution

Pending some more testing to validate the problem, the options will be:

Remaining tasks

  • Run the test scenarios
  • Check additional keyword options including triggers, and the 'downstream' and 'upstream' workflow rules, in case they can work around this.
  • Evaluate security tradeoffs of either resolution
  • Decide on one of the proposed resolutions
  • Implement
🐛 Bug report
Status

Fixed

Version

3.0

Component

GitLab integration

Created by

🇺🇸United States hestenet Portland, OR 🇺🇸

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @hestenet
  • 🇺🇸United States drumm NY, US

    Enable testing on project forks

    We currently don’t enable testing on forks for cost savings. Having a merge request signals that work is ready to review and test. GitLab CI on forks would be testing on commit, which would add up when people make incremental commits.

    It looks like scheduled pipelines are configured in the GitLab UI, outside of .gitlab-ci.yml, so I don’t think they should transfer to forks. That would have to be verified. If forks inherit their parent project schedule, that’d start multiplying the CI resources used.

  • 🇺🇸United States drumm NY, US

    https://git.drupalcode.org/help/ci/pipelines/merge_request_pipelines.md mentions a few security considerations for both resolutions. I hope most aren’t relevant for Drupal’s use case. We do have people storing important secrets in CI variables, like API keys for publishing to NPM.

    Added “Evaluate security tradeoffs of either resolution” to the issue summary.

  • 🇬🇧United Kingdom jonathan1055

    In the issue summary, under "The only ways for an MR opened by a non-member to run is:" the second bullet point says

    For a project member to 'run pipeline' on the merge request manually.

    I'm not sure this is right, at least this is not what I experience. I'm the contrib maintainer, pipelines for my own MRs run, but for MRs created by non-members yes the 'run pipeline' button exists for me, but clicking it does not trigger a pipeline to run.

    #3359327-12: ESLINT does not respect the 'ignore' files #12 and #13 gives some background to this.

  • 🇬🇧United Kingdom jonathan1055

    For info, @drumm has fixed the problem I reported in #4, and maintainers can now run pipelines created by non-members. Thank you.

  • 🇺🇸United States moshe weitzman Boston, MA

    I can verify that scheduled pipelines dont transfer to the forks.

    We currently don’t enable testing on forks for cost savings.

    I dont think we gitlab_templates allow pipelines to run on a commit outside of an MR or default branch. If it does, we can easily disallow that. I dont think anyone wants to increase the number of pipelines that are run.

    Evaluate security tradeoffs of either resolution

    Pipelines in forks is more secure, as secrets are not available to malicious code running MR pipelines. Thats also how Github/Travis/CircleCI work, so we would be in good company.

  • 🇺🇸United States drumm NY, US

    Only testing the default branch is surprising. Drupal.org projects often maintain more than one branch. Opened #3361330: Test branches other than default

  • 🇺🇸United States hestenet Portland, OR 🇺🇸

    A suggested solution from @fjgarlin:


    So the rule can be changed from:
    if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    To (untested):
    if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PROJECT_ROOT_NAMESPACE == "project"

  • Status changed to Fixed about 1 year ago
  • 🇺🇸United States hestenet Portland, OR 🇺🇸

    Committed: https://git.drupalcode.org/project/gitlab_templates/-/commit/65f25ebba78...

    Now we'll need to also incorporate the changes in: #3361330: Test branches other than default but we can do that over in that issue..

  • Status changed to Needs work about 1 year ago
  • 🇺🇸United States hestenet Portland, OR 🇺🇸

    Whoops - this issue isn't just for the branch testing config, it's also for finalizing the decision the solution to actually get MRs running from forks - reopening.

    However - I think the change committed above should prevent the forks from doing a bunch of unnecessary automatic testing. They should now only test on manual trigger or MR.

  • 🇺🇸United States drumm NY, US
  • @drumm opened merge request.
  • Status changed to Needs review about 1 year ago
  • 🇺🇸United States drumm NY, US

    The bulk update will look a lot like Enable GitLab CI, wiki & snippets for projects, except sandboxes Fixed

    • drumm committed ed44bb31 on 7.x-3.x
      Issue #3361106: GitLabCI does not allow MRs from forks opened by non-...
    • drumm committed ed44bb31 on owner-tools-updates
      Issue #3361106: GitLabCI does not allow MRs from forks opened by non-...
  • 🇬🇧United Kingdom jonathan1055

    Thanks for doing this. Next time a non-maintainer creates a MR I will report back to let you know if the pipeline ran immediately.

  • Assigned to drumm
  • Status changed to Fixed 10 months ago
  • 🇺🇸United States drumm NY, US

    Yes, this has been deployed, and the existing issue forks were bulk updated on Friday. The code used for the bulk update was

    $client = versioncontrol_gitlab_get_client()->getHttpClient();
    $result = (new EntityFieldQuery())->entityCondition('entity_type', 'drupalorg_issue_fork')
    ->execute();
    foreach (array_keys($result['drupalorg_issue_fork']) as $gitlab_project_id) {
      print $gitlab_project_id . ' ';
      $client->put('projects/' . $gitlab_project_id, ['Content-type' => 'application/x-www-form-urlencoded'], drupal_http_build_query([
        'jobs_enabled' => 'true',
        'shared_runners_enabled' => 'true',
      ]));
    }

    And we now have integrity checks ensuring this stays enabled, so everything is done here.

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • Status changed to Fixed 9 months ago
  • 🇬🇧United Kingdom jonathan1055

    As promised I would report back, a non-maintainer has created a MR and the pipeline ran without my intervention
    https://git.drupalcode.org/project/scheduler/-/merge_requests/100

Production build 0.69.0 2024