Can't display dashboard through API Traefik v2

Question

I'm a little new to Kubernetes and traefik. I have a Kubernetes cluster up and running with Calico, MetalLB, and traefik. I thought this would be a simple task of just getting the dashboard to display but for the life of me, I can't get it to work. Below are my YAML files

traefik2-ird.yaml

    apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - ingressroutes
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - ingressroutetcps
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - tlsoptions
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - traefikservices
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: traefik

traefik2-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: traefik

spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
    - protocol: TCP
      name: websecure
      port: 443
  selector:
    app: traefik
  type: LoadBalancer
status:
  loadBalancer: {}

traefik2-deployment.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: traefik
  name: traefik-ingress-controller
---
kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: traefik
  name: traefik
  labels:
    app: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
        - name: traefik
          image: traefik:v2.1
          args:
            - --log.level=DEBUG
            #- --api=true
            #- --api.insecure
            - --api.dashboard=true
            - --accesslog
            - --entrypoints.web.Address=:80
            - --entrypoints.websecure.Address=:443
            - --providers.kubernetescrd
            - --certificatesresolvers.default.acme.tlschallenge=true
            - --certificatesresolvers.default.acme.httpChallenge.entryPoint=web
            - --certificatesresolvers.default.acme.email=franklin.shearer@gmail.com
            - --certificatesresolvers.default.acme.storage=acme.json
            # - --certificatesResolvers.default.acme.dnsChallenge.provider=godaddy
            # - --certificatesResolvers.default.acme.dnsChallenge.delayBeforeCheck=5
            # Please note that this is the staging Let's Encrypt server.
            # Once you get things working, you should remove that whole line altogether.
            #- --certificatesresolvers.default.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
          #env:
            #- name: GODADDY_API_KEY
            #  valueFrom:
            #    secretKeyRef:
            #      key: 2s7Z15CMe9_8HN6j28ZM47RXdb5bbpXms
            #      name: godaddy-api-key
            #- name: GODADDY_API_SECRET
            #  valueFrom:
            #    secretKeyRef:
            #      key: X35yD64HoJ827Hd4d9k33L
            #      name: godaddy-api-secret
          ports:
            - name: web
              containerPort: 80
            - name: websecure
              containerPort: 443
            - name: admin
              containerPort: 8080

traefik2-dashboard-igr.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: traefik
  name: traefik-dashboard
spec:
  entryPoints:
    - web
  routes:
    #- match: Host(`traefik.cloud.djcminuz.com`)
    # The dashboard can be accessed on http://traefik.domain.com/dashboard/
    - match: Host(`traefik.cloud.djcminuz.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
      middlewares:
        - name: admin-auth
      kind: Rule
      services:
        - name: api@internal
          kind: TraefikService

---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: admin-auth
spec:
  basicAuth:
    secret: admin-authsecret

I have look and followed similar traefik + Kubernetes tutorials, and even searched traefik site, but I am not understanding what I'm doing wrong or even how to check the logs to see the errors. All help is greatly appreciated.

kubernetes log for traefik pod:

level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"kubernetesCRD\":{}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"accessLog\":{\"format\":\"common\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"drop\"}}},\"certificatesResolvers\":{\"default\":{\"acme\":{\"email\":\"franklin.shearer@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"httpChallenge\":{\"entryPoint\":\"web\"},\"tlsChallenge\":{}}}}}"
time="2020-03-01T18:46:46Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/v2.0/contributing/data-collection/\n"
time="2020-03-01T18:46:46Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
time="2020-03-01T18:46:46Z" level=debug msg="Start TCP Server" entryPointName=web
time="2020-03-01T18:46:46Z" level=debug msg="Start TCP Server" entryPointName=websecure
time="2020-03-01T18:46:46Z" level=info msg="Starting provider *crd.Provider {}"
time="2020-03-01T18:46:46Z" level=debug msg="Using label selector: \"\"" providerName=kubernetescrd
time="2020-03-01T18:46:46Z" level=info msg="label selector is: \"\"" providerName=kubernetescrd
time="2020-03-01T18:46:46Z" level=info msg="Creating in-cluster Provider client" providerName=kubernetescrd
time="2020-03-01T18:46:46Z" level=info msg="Starting provider *traefik.Provider {}"
time="2020-03-01T18:46:46Z" level=info msg="Starting provider *acme.Provider {\"email\":\"email address\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"httpChallenge\":{\"entryPoint\":\"web\"},\"tlsChallenge\":{},\"ResolverName\":\"default\",\"store\":{},\"ChallengeStore\":{}}"
time="2020-03-01T18:46:46Z" level=info msg="Testing certificate renew..." providerName=default.acme
time="2020-03-01T18:46:46Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"services\":{\"api\":{},\"dashboard\":{}}},\"tcp\":{},\"tls\":{}}" providerName=internal
time="2020-03-01T18:46:46Z" level=debug msg="Configuration received from provider default.acme: {\"http\":{},\"tls\":{}}" providerName=default.acme
time="2020-03-01T18:46:46Z" level=debug msg="No default certificate, generating one"
time="2020-03-01T18:46:46Z" level=debug msg="No default certificate, generating one"
time="2020-03-01T18:46:47Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{},\"tcp\":{},\"tls\":{}}" providerName=kubernetescrd
time="2020-03-01T18:46:47Z" level=debug msg="Skipping Kubernetes event kind *v1.Secret" providerName=kubernetescrd
time="2020-03-01T18:46:47Z" level=debug msg="Skipping Kubernetes event kind *v1.Secret" providerName=kubernetescrd
time="2020-03-01T18:46:48Z" level=debug msg="No default certificate, generating one"
time="2020-03-01T18:46:48Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2020-03-01T18:46:48Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2020-03-01T18:46:49Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{\"routers\":{\"kube-system-traefik-dashboard-f900464a898c1ec5833b\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"kube-system-admin-auth\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik.cloud.djcminuz.com`)\"}}},\"tcp\":{},\"tls\":{}}" providerName=kubernetescrd
time="2020-03-01T18:46:49Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{\"routers\":{\"kube-system-traefik-dashboard-f900464a898c1ec5833b\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"kube-system-admin-auth\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik.cloud.djcminuz.com`)\"}},\"middlewares\":{\"default-admin-auth\":{\"basicAuth\":{\"users\":[\"admin:$apr1$/m9V9Oaa$gFnjDk3bLgJV/S/Itcu1X/\"]}}}},\"tcp\":{},\"tls\":{}}" providerName=kubernetescrd
time="2020-03-01T18:46:49Z" level=debug msg="Middleware name not found in config (ResponseModifier)" entryPointName=web routerName=kube-system-traefik-dashboard-f900464a898c1ec5833b@kubernetescrd middlewareName=kube-system-admin-auth@kubernetescrd middlewareType=undefined
time="2020-03-01T18:46:49Z" level=debug msg="Added outgoing tracing middleware api@internal" routerName=kube-system-traefik-dashboard-f900464a898c1ec5833b@kubernetescrd middlewareType=TracingForwarder middlewareName=tracing entryPointName=web
time="2020-03-01T18:46:49Z" level=error msg="middleware \"kube-system-admin-auth@kubernetescrd\" does not exist" routerName=kube-system-traefik-dashboard-f900464a898c1ec5833b@kubernetescrd entryPointName=web
time="2020-03-01T18:46:49Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=web middlewareName=traefik-internal-recovery
time="2020-03-01T18:46:49Z" level=debug msg="No default certificate, generating one"
time="2020-03-01T18:46:50Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd

How does nodeport, differ from metallib, and does nodeport support load balancing multiple clusters? — Frank S, Feb 29 '20 at 16:59
No nodeport will.not do load balancing across multiple kubernetes clusters..it will do TCP layer load balancing within a single kubernetes cluster — Arghya Sadhu, Feb 29 '20 at 17:02
@Arghya Sadhu, I edited the original post to add the logs, which I figured out how to view. — Frank S, Mar 01 '20 at 18:59
Please, can you share the output for `kubectl get svc -n traefik`. — Mark Watney, Mar 02 '20 at 10:09
@mWatney this is the result of the above command No resources found in traefik namespace. — Frank S, Mar 02 '20 at 18:17

score 0 · Accepted Answer · answered Mar 03 '20 at 08:32

I see all your resources are namespaced except for your service. I reproduced a similar scenario in my infrastructure and I could confirm that you need to attach your service to the same namespace your deployment is.

If you check kubectl api-resources you can see that services are namespaced:

$ kubectl api-resources 
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
services                          svc                                         true         Service

So , change your service to the following:

apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: traefik
spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
    - protocol: TCP
      name: websecure
      port: 443
  selector:
    app: traefik
  type: LoadBalancer
status:
  loadBalancer: {}

It's possible to point to a deployment from another namespace in a service using FQDN.

Thanks, @mWatney I had solved it by putting it in the default namespace, but your info added to the tools I'm learning about. — Frank S, Mar 04 '20 at 01:24

Can't display dashboard through API Traefik v2

1 Answers1