3

I have to setup CI in Microsoft Azure Devops to deploy and manage AWS EKS cluster resources. As a first step, found few kubernetes tasks to make a connection to kubernetes cluster (in my case, it is AWS EKS) but in the task "kubectlapply" task in Azure devops, I can only pass the kube config file or Azure subscription to reach the cluster.

In my case, I have the kube config file but I also need to pass the AWS user credentials that is authorized to access the AWS EKS cluster. But there is no such option in the task when adding the New "k8s end point" to provide the AWS credentials that can be used to access the EKS cluster. Because of that, I am seeing the below error while verifying the connection to EKS cluster.

During runtime, I can pass the AWS credentials via envrionment variables in the pipeline but can not add the kubeconfig file in the task and SAVE it.

Azure and AWS are big players in Cloud and there should be ways to connect to connect AWS resources from any CI platform. Does anyone faced this kind of issues and What is the best approach to connect to AWS first and EKS cluster for deployments in Azure Devops CI.

No user credentials found for cluster in KubeConfig content. Make sure that the credentials exist and try again.

enter image description here

intechops6
  • 755
  • 1
  • 9
  • 27

3 Answers3

1

Amazon EKS uses IAM to provide authentication to your Kubernetes cluster through the AWS IAM Authenticator for Kubernetes. You may update your config file referring to the following format:

apiVersion: v1
clusters:
- cluster:
    server: ${server}
    certificate-authority-data: ${cert}
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: aws
  name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      env:
      - name: "AWS_PROFILE"
        value: "dev"
      args:
        - "token"
        - "-i"
        - "mycluster"

Useful links:

Cece Dong - MSFT
  • 25,734
  • 1
  • 13
  • 30
  • the kube config has the aws-iam-authenticator like you said and eks is accessible using the aws user but in the Azure task there is no option available to enter the aws key and secret that can be used to access the k8s cluster. this is the problem for me. – intechops6 Apr 07 '20 at 23:24
  • Could you try to capture the fiddler trace? – Cece Dong - MSFT Apr 09 '20 at 10:06
  • no i can not even reach that point. while configuring the task and before saving the pipeline and running, i am seeing this error. – intechops6 Apr 10 '20 at 02:29
  • Capture the fiddler trace when you create service connection and verify it. – Cece Dong - MSFT Apr 10 '20 at 09:59
  • Dong - I do not have option to enable any debug while creating the service connection. enabling any trace option would be from command line. problem i have is to test the connection to k8s cluster using the kube config file. On linux agent, trying to enable the kubectl debug/trace using verbosity but could not get it to work. do you know how to enable trace in kubectl, "kubectl config view --kubeconfig=$HOME/kubeconfig/deployer/config --v=4" – intechops6 Apr 14 '20 at 21:11
  • 1. Download and install Fiddler：http://www.telerik.com/download/fiddler 2. Close browser process 3. Open fiddler, choose Tools -> Clean Wininet cache 4. Click Tools -> Fiddler Options -> Https -> check capture https, make sure Decrypt HTTPS traffic is checked. Then the fiddler will capture the trace when you create service connection and verify it. – Cece Dong - MSFT Apr 15 '20 at 02:37
0

I got the solution by using ServiceAccount following this post: How to deploy to AWS Kubernetes from Azure DevOps

Khoa
  • 823
  • 8
  • 17
0

For anyone who is still having this issue, i had to set this up for the startup i worked for and it was pretty simple.

After your cluster is created create the service account

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: build-robot
EOF

Then apply the cluster rolebinding

$ kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/name: build-robot
  name: build-robot
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
subjects:
  - kind: ServiceAccount
    name: build-robot
    namespace: default
EOF

Be careful with the above as it gives full access, checkout (https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more info for scoping the access.

From there head over to ADO and follow the steps using deploy-robot as the SA name

$ kubectl get serviceAccounts build-robot -n default -o='jsonpath={.secrets[*].name}'
xyz........

$ kubectl get secret xyz........ -n default -o json
...
...
...

Paste the output into the last box when adding the kubernetes resource into the environment and select Accept UnTrusted Certificates. Then click apply and validate and you should be good to go.

Christopher Bradley
  • 195
  • 2
  • 14