Unable to validate s3fs configuration settings.

Created on 6 September 2023, over 1 year ago
Updated 20 September 2023, over 1 year ago

Problem/Motivation

Hello, I have s3fs:^3.3 installed. However when I try to validate, it shows: "Unable to validate your s3fs configuration settings. Please configure S3 File System from the admin/config/media/s3fs page or settings.php and try again."

I have the following configured in my settings.php

$settings['s3fs.access_key'] = 'access key';
$settings['s3fs.secret_key'] = 'secret key';
$settings['s3fs.use_s3_for_public'] = TRUE;
$settings['s3fs.use_s3_for_private'] = FALSE;

PHP version is 8.2.7
Drupal version is 9.5.10

I'm suspecting that s3fs is not using the keys in my settings.php. Is there any way to validate? Thanks.

πŸ’¬ Support request
Status

Closed: works as designed

Version

3.3

Component

Code

Created by

πŸ‡΅πŸ‡­Philippines josiahcavitana

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @josiahcavitana
  • Status changed to Postponed: needs info over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States cmlara

    There should be additional information included with the messages indicating what validation tests were failing, and in cases where the error was from the Bucket side it will generally include feedback from the bucket.

    I'm suspecting that s3fs is not using the keys in my settings.php.

    If this is the case its most likely cause is that the settings.php is not being read (when its an include file and not the root settings.php) or the config name is incorrect as the s3fs module prefers the s3fs.access_key and s3fs.secret_key over all other authentication types.

    Worst case bucket logging can help you see what errors were logged from the bucket.

    You could also try the patch from https://www.drupal.org/project/s3fs/issues/3274647#comment-14478798 πŸ’¬ Performance extremely slow Closed: cannot reproduce if you wanted to locally log some debug information, though this shouldn't provide much more information than the validation already does.

  • πŸ‡΅πŸ‡­Philippines josiahcavitana

    Hello, cmlara. Thanks for the response.

    Im getting this bucket error:

    Unable to validate your s3fs configuration settings. Please configure S3 File System from the admin/config/media/s3fs page or settings.php and try again.
    An unexpected error occurred. Error executing "PutObject" on "s3-url/s3fs-tests-results/write-test-060923-1305.txt"; AWS HTTP error: Client error: `PUT s3-url/s3fs-tests-results/write-test-060923-1305.txt` resulted in a `403 Forbidden` response: <?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess Denied0Q32M7 (truncated...) AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?> AccessDeniedAccess Denied0Q32M7EDSMG3GWVY

    But I have PutObject in my permissions:

    {
    "Action": [
    "s3:PutObject",
    "s3:PutObjectAcl",
    "s3:GetObject",
    "s3:GetObjectAcl",
    "s3:DeleteObject"
    ],
    "Resource": "arn:aws:s3:::cms/*",
    "Effect": "Allow"
    },

  • πŸ‡ΊπŸ‡ΈUnited States cmlara

    That log entry does confirm we obtained some credential, otherwise there would have been a different message. There is a chance that was an Instance Credential, though again I will say I don't believe that is likely as the access_key and secret_key take priority, the only way it wouldn't use them is if the settings.php is not being loaded in your environment, or is otherwise being overwritten

    I will note that the permissions you list look a little low compared to our declared minimums, though I'm not sure any of those would cause the error in this case.

    I'm assuming "cms" is actually whatever your bucket name is.

    I would suggest working through https://docs.aws.amazon.com/AmazonS3/latest/userguide/troubleshoot-403-e...
    Note: its not likely the public access blocked in this case, as that particular error message shown is for non-public files.

    I will point out that AWS runs on a "Deny if no allow policies match and deny if any deny policy" matches so it would be important to check the bucket access policies as well.

    Bucket Logging or the linked patch should let you see what user is being used.

  • πŸ‡΅πŸ‡­Philippines josiahcavitana

    I can confirm that the permissions/policies are working.
    I've tried using the dev S3 credentials in stg server and it worked. I was able to successfully validate my dev s3 bucket. My stg server runs on PHP 7, Drupal 8, and s3fs 3.0-alpha15.

    Tried the other way around, stg S3 credentials in dev server does not work.

    I'm not sure if this is a valid point but Im thinking, is it possible that the 403 issue is because of headers validation set by the module?
    Or configuration in the php version?

  • πŸ‡ΊπŸ‡ΈUnited States cmlara

    Is it possible that the 403 issue is because of headers validation set by the module?
    Or configuration in the php version?

    Nothing inside the s3fs modules.

    If your development machine for some reason does not support a version of TLS that matches the minimum requirements of the bucket I believe that could generate a 403 as well. I don't believe that to be very likely given its a recent version of PHP however I've been surprised in the past with what some OS images remove from the binaries.

    I believe Bucket Logging as suggested in #2 would show this as well.

  • πŸ‡΅πŸ‡­Philippines josiahcavitana

    I have implemented the bucket logging. Is there any thing that I need to check in particular?

    The logger just showed the same message -- Error executing "PutObject" . Access Denied.

  • πŸ‡΅πŸ‡­Philippines josiahcavitana

    Hello, cmlara. Thank you for taking the time answering my concerns. It's already fixed.

    Problem is due to IAM role. I added codes to assume role from my s3 bucket user.

  • Status changed to Fixed over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States cmlara

    Glad you were able to track it down!

  • Status changed to Closed: works as designed over 1 year ago
  • πŸ‡ΊπŸ‡¦Ukraine voleger Ukraine, Rivne

    Set to the proper status.

Production build 0.71.5 2024