Provide setting to capture credentials if site uses HTTP authentication like Shield

I have this issue, but rather than using Shield I have a site with basic authentication configured via Nginx.

This would be very helpful for decoupled sites with a static frontend and a locked-down backend.

Seems like it might be easy to add the settings and then use them below in the headers something like:$headers['Authorization'] = 'Basic ' . base64_encode($user . ":" . $pass); for internal links

  public function check(LinkCheckerLinkInterface $link) {
    $userAgent = $this->linkcheckerSetting->get('check.useragent');

    $headers = [];
    $headers['User-Agent'] = $userAgent;

    $uri = @parse_url($link->getUrl());

    // URL contains a fragment.
    if (in_array($link->getRequestMethod(), ['HEAD', 'GET']) && !empty($uri['fragment'])) {
      // We need the full content and not only the HEAD.
      $link->setRequestMethod('GET');
      // Request text content only (like Firefox/Chrome).
      $headers['Accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
    }
    elseif ($link->getRequestMethod() == 'GET') {
      // Range: Only request the first 1024 bytes from remote server. This is
      // required to prevent timeouts on URLs that are large downloads.
      $headers['Range'] = 'bytes=0-1024';
    }

    // Add in the headers.
    $options = [
      'headers' => $headers,
      'max_redirects' => 0,
      'http_errors' => FALSE,
      'allow_redirects' => FALSE,
      'synchronous' => FALSE,
    ];

    return $this->httpClient
      ->requestAsync($link->getRequestMethod(), $link->getUrl(), $options)
      ->then(function (ResponseInterface $response) use ($link, $uri) {
        if (!empty($uri['fragment'])) {
          $response = $response->withHeader('Fragment', $uri['fragment']);
        }
        $this->statusHandling($response, $link);
      },
      function (RequestException $e) use ($link) {
        $this->exceptionHandling($e, $link);
      }
    );
  }

Thanks to @stooit for the internal discussion on this.

Here's a start. I didn't add any tests.

Here are some screenshots for before and after:

Before, there are lots of 401 errors

After, there the 401 errors are gone

New fields

Form error checking

Logging

I'll reroll this for the 2.0.x branch so it can be used for Drupal 10 as well.

Actually, it is still applying (with offets/fuzz) but I'll reroll so it's a clean apply.

Kristens-MacBook-Pro:linkchecker kristenpol$ patch -p1 < 3334357-linkchecker-basic-auth.patch 
patching file config/install/linkchecker.settings.yml
patching file config/schema/linkchecker.schema.yml
patching file linkchecker.install
patching file src/Form/LinkCheckerAdminSettingsForm.php
Hunk #1 succeeded at 183 (offset -22 lines).
Hunk #2 succeeded at 459 with fuzz 2 (offset -141 lines).
Hunk #3 succeeded at 486 (offset -141 lines).
patching file src/LinkCheckerService.php
Hunk #1 succeeded at 153 (offset 11 lines).
Hunk #2 succeeded at 269 (offset 20 lines).

I rerolled it but there's an issue when the basic auth is in the settings because then the Clear link data... button isn't working so I'm debugging.

The behavior is different between D9 and D10.

D9: If you click the Clear link data... button, you don't need to fill in the basic auth password.

D10: If you click the Clear link data... button, you do need to fill in the basic auth password. It does show an error (which I didn't notice the first time since the behavior is different).

Since it does provide an error and it works when you put the basic auth password in, then it seems fine as is.

Note that we could change this so that the password is clear text given that the Shield module uses clear text for its password. If this was clear text, then we wouldn't have this problem because the password would show up in the form after you provide it.

Ran tests against 9.5 and 10.0 which have passed.

A nice enhancement to this would be to pull the settings from the Shield module if it's installed and configured.

After an internal discussion with @stooit, I'll update the password field to clear text to match the Shield module behavior.

Here's a 2.0.x patch that has the password in clear text and adds an option to use the Shield settings.

Whoops. Removed debugging in this patch.

If Shield is enabled, you'll see the new option.

Added a minor tweak to handle if the shield module is installed but disabled in the settings: && \Drupal::config('shield.settings')->get('shield_enable')

Here's a patch for the 8.x-1.1 branch.

The last submitted patch, 15: 3334357-linkchecker-basic-auth-2-0-x-15.patch, failed testing. View results →
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

The last submitted patch, 16: 3334357-linkchecker-basic-auth-2-0-x-16.patch, failed testing. View results →
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

The last submitted patch, 17: 3334357-linkchecker-basic-auth-2-0-x-17.patch, failed testing. View results →
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Hi everyone,

I've created simplest approach, skipping new fields, and configuration. It's passing basic authorization inline, like "login:pass@host.com" in Base Path field.

@jkamizato It might be simple, but it's not secure. Credentials should not be stored in plaintext, and that method is going to do so.

Back to Needs Work because patch didn't apply and tests failing. Also, the changes need to be converted to a merge request so that it uses GitLab CI.

Hi @solideogloria, thanks for your advice.

I've recreated this patch using key module dependecy ( https://www.drupal.org/project/key → )

Now, admin user can create a new key entry (user/pass) and reference it in linkchecker configuration.

Sorry, wrong patch file. I'm updating the correct file.

Changed status to "Needs Review"

That sounds very nice!

Could you create a merge request instead of a patch too? This is the only way to trigger the tests 🤓✌️

Update:
Fix "Undefined variable $authUsername" error

@eiriksm Merge Request created

PHPStan failing

 ------ ---------------------------------------------------------------------- 
  Line   src/Form/LinkCheckerAdminSettingsForm.php                             
 ------ ---------------------------------------------------------------------- 
  209    \Drupal calls should be avoided in classes, use dependency injection  
         instead                                                               
 ------ ---------------------------------------------------------------------- 
 ------ ---------------------------------------------------------------------- 
  Line   src/LinkCheckerService.php                                            
 ------ ---------------------------------------------------------------------- 
  161    \Drupal calls should be avoided in classes, use dependency injection  
         instead                                                               
  169    \Drupal calls should be avoided in classes, use dependency injection  
         instead                                                               
  170    \Drupal calls should be avoided in classes, use dependency injection  
         instead                                                               
  181    \Drupal calls should be avoided in classes, use dependency injection  
         instead                                                               
 ------ ---------------------------------------------------------------------- 

I know it needs to fix phpunit issue, but here the latest patch version with phpstan and phpcs fixes

The phpunit test was fixed and I'm attaching the patch.