Skip to main content

Troubleshooting Operator Issues

Describe Resources

You can use kubectl describe on the custom resources for the ngrok operator to check their statuses if they are not behaving properly or are not accessible. This is a good first step

Check Kubernetes Events

If something is not right with the operator you can check the events for the operator namespace.

kubectl get events -n ngrok-operator --sort-by='.lastTimestamp'

Example Output:

LAST SEEN   TYPE     REASON              OBJECT                                                     MESSAGE
3m24s       Normal   Scheduled           pod/ngrok-operator-manager-657c5b69d7-f7vvg                Successfully assigned ngrok-operator/ngrok-operator-manager-657c5b69d7-f7vvg to k3d-ngrok-operator-server-0
3m25s       Normal   ScalingReplicaSet   deployment/ngrok-operator-manager                          Scaled up replica set ngrok-operator-manager-657c5b69d7 to 1
3m25s       Normal   SuccessfulCreate    replicaset/ngrok-operator-manager-657c5b69d7               Created pod: ngrok-operator-manager-657c5b69d7-f7vvg
3m24s       Normal   Started             pod/ngrok-operator-manager-657c5b69d7-f7vvg                Started container ngrok-operator
3m24s       Normal   Created             pod/ngrok-operator-manager-657c5b69d7-f7vvg                Created container ngrok-operator
3m24s       Normal   Pulled              pod/ngrok-operator-manager-657c5b69d7-f7vvg                Container image "docker.io/ngrok/ngrok-operator:0.15.0" already present on machine
3m15s       Normal   ScalingReplicaSet   deployment/ngrok-operator-manager                          Scaled down replica set ngrok-operator-manager-74b76d8f6d to 0 from 1
3m15s       Normal   SuccessfulDelete    replicaset/ngrok-operator-manager-74b76d8f6d               Deleted pod: ngrok-operator-manager-74b76d8f6d-2679x
3m15s       Normal   Killing             pod/ngrok-operator-manager-74b76d8f6d-2679x                Stopping container ngrok-operator
2m55s       Normal   Updating            boundendpoint/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc   Updating BoundEndpoint/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc
2m55s       Normal   Updated             boundendpoint/ngrok-012050e0-8f06-560c-953e-c9d675d41305   Updated BoundEndpoint/ngrok-012050e0-8f06-560c-953e-c9d675d41305
2m55s       Normal   Updated             boundendpoint/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc   Updated BoundEndpoint/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc
2m55s       Normal   Status              boundendpoint/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc   Successfully reconciled status
2m55s       Normal   Updated             boundendpoint/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc   Updated Services
2m55s       Normal   Updated             service/ngrok-22680758-eb09-576e-a7ac-dc3728458dfc         Updated Upstream Service
2m55s       Normal   Updating            boundendpoint/ngrok-012050e0-8f06-560c-953e-c9d675d41305   Updating BoundEndpoint/ngrok-012050e0-8f06-560c-953e-c9d675d41305
2m55s       Normal   LeaderElection      lease/ngrok-operator-leader                                ngrok-operator-manager-657c5b69d7-f7vvg_72fb437b-38ad-444c-a9d8-d37349f2f677 became leader
2m55s       Normal   Status              boundendpoint/ngrok-012050e0-8f06-560c-953e-c9d675d41305   Successfully reconciled status
2m55s       Normal   Updated             boundendpoint/ngrok-012050e0-8f06-560c-953e-c9d675d41305   Updated Services
2m55s       Normal   Updated             service/ngrok-012050e0-8f06-560c-953e-c9d675d41305         Updated Upstream Service
2m55s       Normal   Updating            kubernetesoperator/ngrok-operator                          Updating KubernetesOperator/ngrok-operator
2m55s       Normal   Status              kubernetesoperator/ngrok-operator                          Successfully reconciled status
2m55s       Normal   Updated             kubernetesoperator/ngrok-operator                          Updated KubernetesOperator/ngrok-operator
tip

Kubernetes events usually expire after an hour, so you might want to restart the operator to re-create the events while troubleshooting if the events happened more than an hour ago.

Check Operator Errors

To quickly check the operator logs for only errors, you can use a command such as the following. You can replace the name/namespace if you installed the operator somewhere other than the default location.

k logs -n ngrok-operator deploy/ngrok-operator-manager | grep "ERROR"

Resources not getting deleted

If you marked an ngrok custom resource for deletion but it is not getting deleted, there may be unresolved finalizers on it. You can refer to the finalizers page for more information about how the operator uses finalizers and how to clean up resources with dangling finalizers.

Cloud endpoints not getting created from Ingress / Gateway API

If you're applying Ingress and/or Gateway API resources and do not see any CloudEndpoint or AgentEndpoint custom resources created by the operator, here are a few steps to identify the reason.

Ingress uses ngrok IngressClass?

The ngrok IngressClass is required to use Ingress resources. Check that your Ingress uses the ngrok IngressClass like the following example.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-ingress
  namespace: default
spec:
  ingressClassName: ngrok
  rules:
    - host: example-ingress.ngrok.io
      http:
        paths:
          - backend:
              service:
                name: example-service
                port:
                  number: 80
            path: /
            pathType: Prefix

Gateway API uses the ngrok GatewayClass?

The ngrok GatewayClass is required to use Gateway API resources. If you do not already have it installed, you can create it with:

kubectl apply -f -<<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: ngrok
spec:
  controllerName: ngrok.com/gateway-controller
EOF

Then, make sure your Gateway uses the GatewayClass like the following example:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example-gateway
  namespace: default
spec:
  gatewayClassName: ngrok
  listeners:
    - allowedRoutes:
        namespaces:
          from: Same
      hostname: example-gateway.ngrok.io
      name: example
      port: 443
      protocol: HTTPS

Check Manager Pod Logs

The "manager" pods are responsible for performing the translation from Ingress and Gateway API into native ngrok resources. Check the manager pods for any errors to see if there is a problem with your configuration.

kubectl logs -n ngrok-operator deploy/ngrok-operator-manager | grep "ERROR"

Endpoints not showing up on the dashboard

If you have CloudEndpoint and/or AgentEndpoint resources that are not working, and you can't tell what's wrong with them using kubectl describe, then check the logs for any errors processing them.

Check for AgentEndpoint Error Logs

AgentEndpoint resources are handled by the "agent" pods. Check these for any errors to see if there was an issue processing your AgentEndpoint. Common errors are an invalid traffic policy configuration that may prevent your endpoint from being created.

kubectl logs -n ngrok-operator deploy/ngrok-operator-agent | grep "ERROR"

Check for CloudEndpoint Error Logs

CloudEndpoint resources are handled by the "manager" pods. Check these for any errors to see if there was an issue processing your CloudEndpoint. Common errors are an invalid traffic policy configuration that may prevent your endpoint from being created.

kubectl logs -n ngrok-operator deploy/ngrok-operator-manager | grep "ERROR"

Resource Limits

If you've added several resources such as domains and endpoints, you may encounter a limit for the resource you are trying to create depending on your payment plan. If you are on the pay-as-you-go plan, then you will not encounter this issue and you will be able to create resources freely and only be billed for the resources you create. Check with the pricing and limits page for information about each plan and any limits associated with it.

Still Need Help?

If you aren't able to identify and resolve your issue, head over to the support page to reach out to ngrok support. You can also head over to the operator GitHub repo and see if anyone else encountered the same problem or file a new issue.