-2

I have a Elasticsearch inside the VPC running in account A.

I want to deliver logs from Firehose in Account B to the Elasticsearch in Account A.

Is it possible?

When I try to create delivery stream from AWS CLI I am getting below exception,

$: /usr/local/bin/aws firehose create-delivery-stream --cli-input-json file://input.json --profile devops
An error occurred (InvalidArgumentException) when calling the CreateDeliveryStream operation: Verify that the IAM role has access to the ElasticSearch domain.

The same IAM role, and same input.json works when modified to the Elasticsearch in Account B. I have Transit gateway connectivity enabled between the AWS accounts and I can connect telnet to the Elasticsearch in Account A from EC2 instance in Account B.

Adding my complete terraform code(i got same exception in AWS CLI and also in Terraform): https://gist.github.com/karthikeayan/a67e93b4937a7958716dfecaa6ff7767

karthikeayan
  • 2,278
  • 3
  • 26
  • 48
  • Would this help: https://docs.aws.amazon.com/firehose/latest/dev/vpc.html ? – Val May 11 '20 at 12:51
  • well, the actual issue is not with connectivity, i already have vpc to vpc private communication enabled via transit gateway and i am able to telnet from Account B to Account A... i believe the aws cli commands checks if the given elasticsearch instance name present in the current vpc and raising exception when it couldn't find it... – karthikeayan May 11 '20 at 16:27
  • why downvote? can you please comment? – karthikeayan May 18 '20 at 13:16

2 Answers2

0

It looks like you haven't granted sufficient permissions to the role that is used when creating the stream (from the CLI example provided I'm guessing its a role named 'devops'). At minimum you will need firehose: CreateDeliveryStream.

I suggest adding the below permissions to your role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "firehose:PutRecord",
                "firehose:CreateDeliveryStream",
                "firehose:UpdateDestination"
            ],
            "Resource": "*"
        }
    ]
}
chris stamper
  • 12,726
  • 1
  • 21
  • 46
  • the iam role i am attaching to the firehose delivery stream gives access to put data into elasticsearch... didn't get you, why we need to add firehose access to itself... also with the same role when i change the elasticsearch domain name to the one which is in the local vpc, it works.. – karthikeayan May 12 '20 at 13:52
0

https://forums.aws.amazon.com/message.jspa?messageID=943731

I have been informed from AWS forum that this feature is currently not supported.

karthikeayan
  • 2,278
  • 3
  • 26
  • 48
  • I found it supports https://aws.amazon.com/premiumsupport/knowledge-center/kinesis-firehose-cross-account-streaming/ – PPShein Oct 26 '20 at 08:17
  • @PPShein my question is about private elasticsearch inside a vpc... the link you mentioned have this... Streaming data delivery works only if the Amazon ES cluster is publicly accessible and has Node-to-node encryption disabled. – karthikeayan Oct 28 '20 at 09:14