3

I'm trying to solve this mystery for few days now, but no joy. Basically, Terraform cannot assume role and failing with:

Initializing the backend...
2019/10/28 09:13:09 [DEBUG] New state was assigned lineage "136dca1a-b46b-1e64-0ef2-efd6799b4ebc"
2019/10/28 09:13:09 [INFO] Setting AWS metadata API timeout to 100ms
2019/10/28 09:13:09 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/10/28 09:13:09 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2019/10/28 09:13:09 [INFO] Attempting to AssumeRole arn:aws:iam::72xxxxxxxxxx:role/terraform-admin-np (SessionName: "terra_cnp", ExternalId: "", Policy: "")

Error: The role "arn:aws:iam::72xxxxxxxxxx:role/terraform-admin-np" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

In AWS:

I have role: terraform-admin-np with 2 AWS managed policy: AmazonS3FullAccess & AdministratorAccess and a trust relationship with this:

  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::72xxxxxxxxxx:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then I have an user with policy document attached:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TfFullAccessSts",
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole",
                "sts:DecodeAuthorizationMessage",
                "sts:AssumeRoleWithSAML",
                "sts:AssumeRoleWithWebIdentity"
            ],
            "Resource": "*"
        },
        {
            "Sid": "TfFullAccessAll",
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "*",
                "arn:aws:ec2:region:account:network-interface/*"
            ]
        }
    ]
}

and a S3 bucket: txxxxxxxxxxxxxxte with this policy document attached:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TFStateListBucket",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::72xxxxxxxxxx:root"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::txxxxxxxxxxxxxxte"
        },
        {
            "Sid": "TFStateGetPutObject",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::72xxxxxxxxxx:root"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::txxxxxxxxxxxxxxte/*"
        }
    ]
}

In Terraform:

The snippet from the provider.tf: 

###---- Default Backend and Provider config values -----------###
terraform {
  required_version = ">= 0.12"
  backend "s3" {
    encrypt  = true
  }
}

provider "aws" {
  region  = var.region
  version = "~> 2.20"
  profile = var.profile
  assume_role {
    role_arn     = var.role_arn
    session_name = var.session_name
  }
}

Snippet from tgw_cnp.tfvars backend config: 

## S3 backend config
key             = "backend/tgw_cnp_state"
bucket          = "txxxxxxxxxxxxxxte"
region          = "us-east-2"
profile         = "local-tgw"
role_arn        = "arn:aws:iam::72xxxxxxxxxx:role/terraform-admin-np"
session_name    = "terra_cnp"

and then running this way:

TF_LOG=debug terraform init -backend-config=tgw_cnp.tfvars

With this, I can assume role using AWS CLI without any issue:

# aws --profile local-tgw sts assume-role --role-arn "arn:aws:iam::72xxxxxxxxxx:role/terraform-admin-np" --role-session-name AWSCLI
{
    "Credentials": {
        "AccessKeyId": "AXXXXXXXXXXXXXXXXXXA",
        "SecretAccessKey": "UixxxxxxxxxxxxxxxxxxxxxxxxxxxxMt",
        "SessionToken": "FQoGZXIvYXdzEJb//////////wEaD......./5LFwNWf6riiNw9vtBQ==",
        "Expiration": "2019-10-28T13:39:41Z"
    },
    "AssumedRoleUser": {
        "AssumedRoleId": "AROA2P7ZON5TSWMOBQEBC:AWSCLI",
        "Arn": "arn:aws:sts::72xxxxxxxxxx:assumed-role/terraform-admin-np/AWSCLI"
    }
}

but terraform fails with the above error. Any idea what's I'm doing wrong?

MacUsers
  • 1,761
  • 3
  • 28
  • 48
  • Can you share the AWS CLI command (after removing sensitive data) that you used to assume role? – krishna_mee2004 Oct 28 '19 at 12:15
  • sure thing @krishna_mee2004 - I've actually added the output to the OP. – MacUsers Oct 28 '19 at 12:50
  • It looks like you are using `default-tgw` profile when using CLI. However in Terraform you have mentioned `local-tgw`. Can you try the CLI using local-tgw profile and see if you can assume the role? – krishna_mee2004 Oct 28 '19 at 13:23
  • that was my silly typo @krishna_mee2004 - the actual profile name called neither `local-tgw` or `defaut-tgw`. There is about of sensitive in the naming, so I just tried to mask it and made the mistake during that. I can assure the exactly same profile is used in TF and CLI. I'm fixed the typo in OP to minimize the confusion. – MacUsers Oct 28 '19 at 14:07
  • not sure if it indicates anything - the same (similar) TF code running just fine on a different account to do the same thing. This is the new plan (to setup a different environment) on a different AWS a/c and it's not working at all. That particular user, that trying to assume that role, has no MFA configured and with only has Programmatic access access. – MacUsers Oct 28 '19 at 14:12
  • 2
    Your setup should ideally work without any issues, it worked for me. One thing to check is: have you defined the keys of profile `local-tgw` in ~/.aws/credentials file? – krishna_mee2004 Oct 29 '19 at 11:29
  • I have had a silly mistake - the `region` in _tgw_cnp.tfvars_ was wrong; corrected that and it's working now. thanks for your time @krishna_mee2004 – MacUsers Oct 31 '19 at 07:47

1 Answers1

1

Okay, answering to my own question........ It worked now. I have had a silly mistake - the region in tgw_cnp.tfvars was wrong, which I was keep missing out. In AWS CLI as I didn't have to specify the region (it was getting it from the profile), so it worked without any issue but in TF I specified the region and the value was wrong, hence the failure. The suggestions in the error reporting was a bit misleading.

I can confirm the above config works fine. It's all good now.

MacUsers
  • 1,761
  • 3
  • 28
  • 48