How to generate new GitLab's Personal access token for providing external access?

Note:
I raised this at the January 11th Gitlab initiative meeting.

The response from @hestnet was:

The proxy block should be the main technical change to make... I may need to put up a policy page somewhere.

May need to bring this up again at a future meeting, The 11th was in the middle of some serious Gitlab performance issues making it not the best time to push discussion on this.

My current use case for this is to use glab, the CLI tool for Gitlab. Its invaluable for things like browsing MRs/issues, checking that your gitlab ci yaml is valid, etc. It requires a token.

Thank you for the specific use cases, they have helped us understand the need.

With the upgrade to GitLab 16, personal access tokens must have an expiration date set, https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html. Drupal.org creates personal access tokens for people to use as HTTP passwords for pushing to Git without setting up SSH keys. This is currently broken since we had not been setting an expiration date. While adding information about expiration dates to the UI wouldn’t be a huge lift, it would be better long-term to scrap this integration in favor of opening up GitLab’s UI for this.

Moving to drupalorg module since this will be a code change for removing that UI.

@cmlara - re mirroring, https://git.drupalcode.org/project/[project name]/-/settings/repository was enabled to allow pull mirroring from other services and push mirroring to other services

Access tokens at http://git.drupalcode.org/-/profile/personal_access_tokens are now allowed.

Before closing this out, the documentation at https://www.drupal.org/drupalorg/docs/user-accounts/git-authentication-f... → and anywhere else mentioning HTTP password authentication with git.drupal.org should be updated.

@drumm Has PATs enabled for all users ?

I am still being redirected to Drupal.org profile page when accessing https://git.drupalcode.org/-/profile/personal_access_tokens . Thanks.

Hmm, i tried using the PAT to connect through PHPStorm for their integrations, but unfortunately that gives a 403.

The gitlab CLI tooling does seem to work (https://gitlab.com/gitlab-org/cli#installation). Not sure what PHPStorm does differently ;(

You need to tell some extra info to the CLI though since it gets a little confused with the whole git.drupal.org vs git.drupalcode.org

# What protocol to use when performing Git operations. Supported values: ssh, https.
git_protocol: ssh
# What editor glab should run when creating issues, merge requests, etc. This global config cannot be overridden by hostname.
editor:
# What browser glab should run when opening links. This global config cannot be overridden by hostname.
browser:
# Set your desired Markdown renderer style. Available options are [dark, light, notty]. To set a custom style, refer to https://git>glamour_style: dark
# Allow glab to automatically check for updates and notify you when there are new updates.
check_update: true
# Whether or not to display hyperlink escape characters when listing items like issues or merge requests. Set to TRUE to display hy>display_hyperlinks: false
# Default GitLab hostname to use.
host: git.drupal.org
# Set to true (1) to disable prompts, or false (0) to enable them.
no_prompt: false
# Configuration specific for GitLab instances.
hosts:
    gitlab.com:
        # What protocol to use to access the API endpoint. Supported values: http, https.
        api_protocol: https
        # Configure host for API endpoint. Defaults to the host itself.
        api_host: gitlab.com
        # Your GitLab access token. To get one, read https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
        token:
    git.drupalcode.org:
        # What protocol to use to access the API endpoint. Supported values: http, https.
        api_protocol: https
        # Configure host for API endpoint. Defaults to the host itself.
        api_host: git.drupalcode.org
        # Your GitLab access token. To get one, read https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
        token: xxxxxxxxxxxxxxxxxx
    git.drupal.org:
        # What protocol to use to access the API endpoint. Supported values: http, https.
        api_protocol: https
        # Configure host for API endpoint. Defaults to the host itself.
        api_host: git.drupalcode.org
        # Your GitLab access token. To get one, read https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
        token: xxxxxxxxxxxxxxxxxxx

I'll second having issues.

Phpstorm (for me on 2023.2 with GItlab plugin version 232.8660.205) uses the GraphQL endpoint, which uses the "Authorization: Bearer" header. When PHPStorm tries to query the current user I receive:

{
    "data": {
        "currentUser": null
    }
}

Note: I had to use the git.drupalcode.org endpoint as it seems Phpstorm doesn't handle the git.drupal.org throwing a 302 redirect to git.drupalcode.org in response to the post to the GraphQL endpoint and receives an error trying to deseralize the empty response.

Using another app (GitBlur on an iPhone) which uses the REST API and the PRIVATE-TOKEN header I'm getting intermittent 401's as it tries to query the API, especially at api/v4/version target. It seems to partially work as I'm seeing project listings in the app, though the app also crashes trying to access any project. I haven't fully ruled out the crashes being solely GitBlur fault, but combined with other API issues and the fact it worked against a lab target it makes me wonder if this is caused by the 401 as well. I could imagine GitBlur needing to know version and when it doesn't it makes the wrong requests causing some sort of fault internally and crashing, slightly poor error handling if that is the case, but also hard to expect it to work either.

I'm wondering if there some paths that might have some CDN or Reverse Proxy config still in the way that is blocking the API? Perhaps a path was previously blocked but wasn't fully unblocked or a path that didn't get any changes to pass through all the possible GitLab auth headers? I guess the second half of that is which server should be providing the the API, git.drupalcode.org or git.drupal.org?

I am still being redirected to Drupal.org profile page when accessing https://git.drupalcode.org/-/profile/personal_access_tokens

The URL did have a 301 permanent redirect in the past. If clearing your browser cache does not help, I could have missed something or there could be something else wrong.

git.drupalcode.org is the domain to use for all HTTP requests, including API requests.

git.drupal.org is only for clones via SSH, keeping the same git remote URIs for SSH as we had pre-GitLab. We keep GitLab and Drupal.org on different domains so that cross-site-scripting and other vulnerabilities are contained to one service. And git.drupalcode.org is fronted by a CDN, which is great for HTTP, but does not route other ports.

If there is any documentation suggesting using git.drupal.org for anything but SSH, please open an issue so we can look at updating it.

We do have a load balancer in front of GitLab which filters some paths and headers. We have some things we can't allow in GitLab, such as editing your user account or projects to get out of sync with Drupal.org. So we are allowlisting authenticated read & write API access on a case-by-case basis. Please open separate issue in the infrastructure project with specific requests, including their URL and HTTP method.

GET requests to /api/v4/version look safe to make while authenticated, those are now allowed. I don’t see any other specifics requested, so please collect those in followup issues, as mentioned above.

For PHPStorm, I am not spotting what looks like an official or preferred plugin. I do see requests for /api/v4/metadata with a user agent containing PhpStorm. GET requests to that are now also allowed to be authenticated.

GraphQL is tricky since all requests are POSTs to a single URL. We can and do allow unauthenticated read access. It’s the API that much of the Drupal.org issue queue integration uses. Protecting writes to the GraphQL in the same way would require inspecting the request body, which is not a capability of our load balancer, as far as I know.

Automatically closed - issue fixed for 2 weeks with no activity.

Just wanted to add a follow up.

I successfully connect the official GitLab Workflow VScode extension to https://git.drupalcode.org/ using an access token.
I was able to view open MRs and the pipeline results in VSCode. I did not test reviews, comments, pushes as I only chose basic permissions setting up the access token as a proof-of-concept.