Unable to get S3FS working with IAM

Is the s3fs access key and secret key both empty?

Are you sure your environment variables are making it through ? PHP-FPM for example by default strips environment variables unless you configure it not to. Running a phpinfo() is a good idea to validate, I believe the Drupal phpinfo() page sanitizes environment variables so need to run as a custom php file.

What did your credentials ini file look like and was it stored at the path configured and readable by the user that PHP is running under?

Hi,

We're looking out for configuration using "Option 1 (Use AWS defaultProvider)" as mentioned here: https://git.drupalcode.org/project/s3fs/-/blob/4.0.x/README.txt

I believe that should work without specifying the access key and secret key. We're able to access s3 bucket using our configured IAM role (tested via AWS CLI). The intention is to be able to access s3 bucket using the IAM role (instead of credentials).

Any pointers will be very helpful. Thanks!

@e3aa3b This is a community support method, please do not send direct messages through the contact form, instead please place the information in this issue both so that others can provide support and to retain the timeline of the issue. if you want a direct private connection we can discuss a professional services contract.

I believe that should work without specifying the access key and secret key.

Specifically you must NOT specify the access key and secret key, this is why I ask if they are set, either in settings.php or via the key module, if they are it would be expected for the Default Provider not to be called. This is one of the only limited items inside of the S3FS module that can impact operations, beyond that its generally outside of the S3FS module.

Any pointers will be very helpful. Thanks!

Please see #4 specifically:

• What is the results of phpinfo() inside the webserver? Are the variables present? Note this needs to be run inside the server, not from the CLI or Drush as the results can be different.
• What did your credentials ini file look like and was it stored at the path configured and readable by the user that PHP is running under? (This method should rule out environment variable issues)

Adding onto the above since you provide a number of environment variables by email
Was your AWS_WEB_IDENTITY_TOKEN_FILE readable by the user that PHP executes under? Its possible for this to be readable by the user running the AWS CLI and not for PHP, though I believe this should throw an exception I'm not sure this will percolate up or not inside a chained provider.

Thanks for the comment @cmlara

1. We're not specifying the access key and secret key anywhere

2. when I do phpinfo(), I notice following AWS specific variables present:
- AWS_DEFAULT_REGION
- AWS_REGION
- AWS_ROLE_ARN
- AWS_ROLE_SESSION_NAME
- AWS_WEB_IDENTITY_TOKEN_FILE

These are matching with the values in the environment. Are we expecting more which maybe missing here..?

3. Also the AWS_WEB_IDENTITY_TOKEN_FILE has full access across, so I believe should be readable by PHP.

Thanks for your feedback so far! Pls let me know if some more debugging maybe possible to crack it.

Apologies for the delay, I was not at my desk most of the past week.

At this point I would suggest you review your AWS S3 bucket access logs to determine what ARN is accessing, that its the correct ARN, and that it has the appropriate access permissions.

As long as the secret key and access key are both empty this is going to be unlikely to be an issue in s3fs.

As this issue has not received an update in over 2 weeks I am closing the issue out as cannot reproduce with the assumption that this was likely related to attempting to use a token that was not valid.

It amazes me how unintuitive IAM is. In case this helps someone else, I had to define a policy for my user like this:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "VisualEditor0",
			"Effect": "Allow",
			"Action": [
				"s3:PutObject",
				"s3:GetObjectAcl",
				"s3:GetObject",
				"s3:DeleteObjectVersion",
				"s3:DeleteObject",
				"s3:PutObjectAcl",
				"s3:GetObjectVersion"
			],
			"Resource": "arn:aws:s3:::[bucket-name]/*"
		},
		{
			"Sid": "VisualEditor1",
			"Effect": "Allow",
			"Action": [
				"s3:ListBucketVersions",
				"s3:ListBucket"
			],
			"Resource": "arn:aws:s3:::[bucket-name]"
		}
	]
}