The AWS Access Key Id does not exist in our records

Question

I created a new Access Key and configured that in the AWS CLI with aws configure. It created the .ini file in ~/.aws/config. When I run aws s3 ls it gives:

A client error (InvalidAccessKeyId) occurred when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.

AmazonS3FullAccess policy is also attached to the user. How to fix this?

is there a file `~/.aws/credentials` and does it contain the correct id and key? — Dusan Bajic, Aug 20 '16 at 07:29
I am facing the same issue mentioned here. I have got credentials file created and the credentials are verified for validity. Couldn't find anything wrong. Any help? — SanthoshSolomon, Jan 17 '20 at 15:53
I've faced this issue on a Lambda function uploading files to S3 and managed to solve by configuring S3 object using the endpoint instead of the AWS credentials. — pafede2, Apr 21 '21 at 13:55

score 58 · Answer 1 · answered Jan 25 '17 at 11:18

58

It might be happening that you have the old keys exported via env variables (bash_profile) and since the env variables have higher precedence over credential files it is giving the error "the access key id does not exists".

Remove the old keys from the bash_profile and you would be good to go.

Happened with me once earlier when I forgot I have credentials in bash_profile and gave me headache for quite some time :)

answered Jan 25 '17 at 11:18

Manish Joshi

2,512
1
18
29

1

Thanks for the idea. This was driving me batty. – Kevin Pauli Feb 20 '17 at 19:57
this should be the accepted answer. in my case, I was mounting host keys but there was already empty variables define an ENV for AWS keys. – Adiii Dec 27 '19 at 14:47
how to check bash_profile ? If wehave any keys there? please help – Sneha Apr 22 '20 at 05:02

score 50 · Answer 2 · answered Aug 16 '17 at 01:53

50

It looks like some values have been already set for the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

If it is like that, you could see some values when executing the below commands.

echo $AWS_SECRET_ACCESS_KEY
echo $AWS_ACCESS_KEY_ID

You need to reset these variables, if you are using aws configure

To reset, execute below commands.

unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY

answered Aug 16 '17 at 01:53

Ashik Mohammed

659
6
9

1

That's exactly my case: I should have set these variable long time ago and completely forgot about them. Thank you a lot! – Andrea Sciamanna Feb 03 '18 at 14:33

score 12 · Answer 3 · answered Apr 16 '20 at 11:17

12

Besides aws_access_key_id and aws_secret_access_key, I also added aws_session_token in credentials, it works for me

answered Apr 16 '20 at 11:17

Robin He

841
8
7

Can u provide an example for aws_session_token. – Crysis Mar 02 '21 at 05:00

score 5 · Answer 4 · answered Nov 24 '17 at 19:35

5

None of the up-voted answers work for me. Finally I pass the credentials inside the python script, using the client API.

import boto3
client = boto3.client(
's3',
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_KEY,
aws_session_token=SESSION_TOKEN)

Please notice that the aws_session_token argument is optional. Not recommended for public work, but make life easier for simple trial.

answered Nov 24 '17 at 19:35

Frank

595
7
5

7

If you're using EC2/Lambda/etc. you'll get [temporary security credentials](http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#UsingTemporarySecurityCredentials) which means you'll need to use `AWS_SESSION_TOKEN` environment variable and pass it to boto3. I'm pretty sure it's not optional, because omitting it immediately throws `InvalidAccessKeyId`. – Hendy Irawan Dec 10 '17 at 16:08
The solution works in my local jupyter notebook, and it has not been tested for a EC2/Lambda environment. Thanks for the experiments you have done, @HendyIrawan, do you test for both EC2 and Lambda(or other environments if any)? – Frank Dec 11 '17 at 18:40
Yep the session token must be used in this case. – Istvan Jan 30 '20 at 18:24
using okta and gimme-aws-creds. this is what I needed – frostymarvelous Mar 20 '20 at 12:10
Setting up the "aws_session_token" in credential file also solved my problem. – Cihangir Jan 14 '21 at 08:37

score 5 · Answer 5 · answered Nov 01 '20 at 17:39

5

I made the mistake of setting my variables with quotation marks like this:

AWS_ACCESS_KEY_ID="..."

answered Nov 01 '20 at 17:39

Rivers Cuomo

129
1
3
10

I have made the same mistake. Thanks! – Franklin Innocent F Dec 17 '20 at 09:30

score 3 · Answer 6 · answered Oct 08 '17 at 14:54

For me, I was relying on IAM EC2 roles to give access to our machines to specific resources.

I didn't even know there was a credentials file at ~/.aws/credentials, until I rotated/removed some of our accessKeys at the IAM console to tighten our security, and that suddenly made one of the scripts stop working on a single machine.

Deleting that credentials file fixed it for me.

score 3 · Answer 7 · answered Jan 15 '20 at 05:02

3

You may need to set the AWS_DEFAULT_REGION environment variable.

answered Jan 15 '20 at 05:02

Ela Dute

346
6
8

score 2 · Answer 8 · answered Nov 17 '17 at 07:07

2

you can configure profiles in the bash_profile file using

<profile_name>
aws_access_key_id = <access_key>
aws_secret_access_key = <acces_key_secret>

if you are using multiple profiles. then use:

aws s3 ls --profile <profile_name>

answered Nov 17 '17 at 07:07

Rahul Radhakrishnan

21
2

score 2 · Answer 9 · answered Nov 30 '20 at 09:04

2

You may have configured AWS credentials correctly, but using these credentials, you may be connecting to some specific S3 endpoint (as was the case with me).

Instead of using:

aws s3 ls

try using:

aws --endpoint-url=https://<your_s3_endpoint_url> s3 ls

Hope this helps those facing the similar problem.

answered Nov 30 '20 at 09:04

Abhinav Sureka

21
3

worked for me after setting up localstack – ishon19 Feb 25 '21 at 05:54

Dusan Bajic · Answer 10 · 2016-08-20T09:00:41.840

1

Looks like ~/.aws/credentials was not created. Try creating it manually with this content:

[default]
aws_access_key_id = sdfesdwedwedwrdf
aws_secret_access_key = wedfwedwerf3erfweaefdaefafefqaewfqewfqw

(on my test box, if I run aws command without having credentials file, the error is Unable to locate credentials. You can configure credentials by running "aws configure".) Can you try running these two commands from the same shell you are trying to run aws:

$ export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
$ export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

and then try aws command.

edited Aug 20 '16 at 09:00

answered Aug 20 '16 at 07:40

Dusan Bajic

8,851
3
29
35

3

@DusanBajic the accesskeyid "does not exist in our records" is an error I have encountered during testing when I deliberately modified the value of the key in a signed URL while trying to simulate various failures. It implies that a key was indeed used but that IAM claims it doesn't exist. Assuming no copy/paste error, this response suggests -- to me -- that since the credentials were newly created, then it may be the case that IAM is/was experiencing a transient replication delay that will sort itself out by waiting, rather than the config file being missing. – Michael - sqlbot Aug 20 '16 at 10:30
3

So it must be replication delay. When I changed the AWS_DEFAULT_REGION to us-east-1 it started working. – kaushikdr Aug 21 '16 at 09:11

score 1 · Answer 11 · answered Jan 25 '17 at 05:23

another thing that can cause this, even if everything is set up correctly, is running the command from a Makefile. for example, I had a rule:

awssetup:
        aws configure
        aws s3 sync s3://mybucket.whatever .

when I ran make awssetup I got the error: fatal error: An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: The AWS Access Key Id you provided does not exist in our records.. but running it from the command line worked.

score 1 · Answer 12 · answered Sep 12 '19 at 08:34

Adding one more answer since all the above cases didn't work for me.

In AWS console, check your credentials(My Security Credentials) and see if you have entered the right credentials.

Thanks to this discussion: https://forums.aws.amazon.com/message.jspa?messageID=771815

score 1 · Answer 13 · answered Jul 21 '20 at 09:19

To those of you who run aws s3 ls and getting this exception. Make sure You have permissions to all regions under the provided AWS Account. When running aws s3 ls you try to pull all the s3 buckets under the AWS Account. therefore, in case you don't have permissions to all regions, you'll get this exception - An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.

Follow Describing your Regions using the AWS CLI for more info.

score 1 · Answer 14 · edited Mar 07 '21 at 00:28

1

Kindly export the below variables from the credential file from the below directory.

path = .aws/
filename = credentials

export aws_access_key_id = AK###########GW
export aws_secret_access_key = g#############################J

edited Mar 07 '21 at 00:28

Nguyễn Văn Phong

11,572
15
21
43

answered Mar 06 '21 at 19:07

Sudd Unity

11
1

score 0 · Answer 15 · answered May 31 '18 at 20:50

0

I tries below steps and it worked: 1. cd ~ 2. cd .aws 3. vi credentials 4. delete aws_access_key_id = aws_secret_access_key = by placing cursor on that line and pressing dd (vi command to delete line).

Delete both the line and check gain.

answered May 31 '18 at 20:50

Shubhangi

31
2

is this on the server or the local pc? – Искрен Станиславов Jan 18 '19 at 11:51

score 0 · Answer 16 · edited Dec 20 '19 at 12:51

If you have an AWS Educate account and you get this problem:

An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records".

The solution is here:

Go to your C:/ drive and search for .aws folder inside your main folder in windows.
Inside that folder you get the "credentials" file and open it with notepad.
Paste the whole key credential from AWS account to the same notepad and save it.
Now you are ready to use you AWS Educate account.

score 0 · Answer 17 · answered Apr 21 '20 at 16:11

This could happen because there's an issue with your AWS Secret Access Key. After messing around with AWS Amplify, I ran into this issue. The quickest way is to create a new pair of AWS Access Key ID and AWS Secret Access Key and run aws configure again. I works for me. I hope this helps.

score 0 · Answer 18 · answered Oct 19 '20 at 17:40

Assuming you already checked Access Key ID and Secret... you might want to check file team-provider-info.json which can be found under amplify/ folder

"awscloudformation": {
      "AuthRoleName": "<role identifier>",
      "UnauthRoleArn": "arn:aws:iam::<specific to your account and role>",
      "AuthRoleArn": "arn:aws:iam::<specific to your account and role>",
      "Region": "us-east-1",
      "DeploymentBucketName": "<role identifier>",
      "UnauthRoleName": "<role identifier>",
      "StackName": "amplify-test-dev",
      "StackId": "arn:aws:cloudformation:<stack identifier>",
      "AmplifyAppId": "<id>"
    }

IAM role being referred here should be active in IAM console.

Attaque · Answer 19 · 2020-10-22T17:04:33.187

If you get this error in an Amplify project, check that "awsConfigFilePath" is not configured in amplify/.config/local-aws-info.json

In my case I had to remove it, so my environment looked like the following:

{
  // **INCORRECT**
  // This will not use your profile in ~/.aws/credentials, but instead the
  // specified config file path
  // "dev": {
  //  "configLevel": "project",
  //  "useProfile": false,
  //  "awsConfigFilePath": "/Users/dev1/.amplify/awscloudformation/cEclTB7ddy"
  // },
  // **CORRECT**
  "dev": {
    "configLevel": "project",
    "useProfile": true,
    "profileName": "default",
  }
}

score 0 · Answer 20 · answered Dec 19 '20 at 18:12

0

Maybe you need to active you api keys in the web console, I just saw that mine were inactive for some reason...

answered Dec 19 '20 at 18:12

Tomislav Mikulin

4,016
4
17
31

rubendmatos1985 · Answer 21 · 2021-02-16T09:45:37.490

I had the same problem in windows and using the module aws-sdk of javascript. I have changed my IAM credentials and the problem persisted even if i give the new credentials through the method update like this

s3.config.update({
    accessKeyId: 'ACCESS_KEY_ID',
    secretAccessKey: 'SECRET_ACCESS_KEY',
    region: 'REGION',
});

After a while i found that the module aws-sdk had created a file inside the folder User on windows with this path

C:\Users\User\.aws\credentials

. The credentials inside this file take precedence over the other data passed through the method update.

The solution for me was to write here

C:\Users\User\.aws\credentials

the new credentials and not with the method s3.config.update

score 0 · Answer 22 · edited Apr 03 '21 at 06:31

Thanks, everyone. This helped to solve.

Something somehow happened which changed the keys & I didn't realize since everything was working fine until I connected to S3 from a spark...then from the command line also error started coming even in AWS s3 ls

Steps to solve

Run AWS configure to check if keys are set up (verify from last 4 characters & just keep pressing enter)
AWS console --> Users --> click on the user --> go to security credentials--> check if the key is the same that is showing up in AWS configure
If both not the same, then generate a new key, download csv
run --> AWS configure, set up new keys
try AWS s3 ls now

Change keys at all places in my case it was configs in Cloudera.

score 0 · Answer 23 · answered May 08 '21 at 10:53

I couldn't figure out how to get the system to accept my Vocareum credentials so I took advantage of the fact that if you configure your instance to use IAM roles, the SDK automatically selects the IAM credentials for your application, eliminating the need to manually provide credentials.

Once a role with appropriate permissions was applied to the EC2 instance, I didn't need to provide any credentials.

The AWS Access Key Id does not exist in our records

23 Answers23

Linked