> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Lab Solution Application Failure

> Kubernetes lab troubleshooting a two tier web and MySQL app across namespaces, diagnosing and fixing DNS, service ports, selectors, credentials, and NodePort failures.

In this lab we walk through troubleshooting a simple two‑tier application (web + MySQL). The web UI must display a green success page when the application stack is healthy. Each namespace (alpha, beta, gamma, delta, epsilon, zeta) contains a different failure scenario. For every case we:

* Inspect the application resources (Deployment / Pod / Service).
* Reproduce the failure (curl the NodePort where applicable).
* Identify the root cause and apply a fix.
* Verify the web application returns the green success page.

<Callout icon="lightbulb" color="#1CB2FE">
  Tip: To avoid repeating `-n <namespace>` for every kubectl command, set the default namespace for your current context with `kubectl config set-context --current --namespace=<namespace>`.
</Callout>

***

## Quick reference — Useful kubectl commands

Use these as you go through each namespace:

```bash theme={null}
# set the default namespace for current context
kubectl config set-context --current --namespace=alpha

# inspect resources
kubectl get pods
kubectl get svc
kubectl describe svc <service-name>
kubectl describe deployment <deployment-name>
kubectl logs <pod-name>

# edit/replace resources
kubectl edit svc <service-name>
kubectl edit deployment <deployment-name>
kubectl replace --force -f /tmp/kubectl-edit-XXXXX.yaml
```

You may also find these links helpful:

* [Kubernetes Concepts — Services](https://kubernetes.io/docs/concepts/services-networking/service/)
* [kubectl Cheat Sheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/)

***

## Troubleshooting walkthrough by namespace

Below we keep the original sequence and the exact investigation/fix examples so you can reproduce the steps.

### 1) Namespace: alpha — DNS / Service name mismatch

Symptom: Web UI loads but shows a red (failed) page. The web app logs show it cannot resolve the MySQL service name.

Observed error:

```bash theme={null}
Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=root; DB_Password=paswrd;
2003: Can't connect to MySQL server on 'mysql-service:3306' (-2 Name does not resolve)
From webapp-mysql-84fbfc644f-nhfsv!
```

Investigation (select excerpts):

```bash theme={null}
kubectl get pods -n alpha
# webapp-mysql-84fbfc644f-nhfsv   1/1 Running
kubectl get svc -n alpha
# mysql         ClusterIP   10.43.208.47   <none>    3306/TCP
kubectl describe deployment webapp-mysql -n alpha
# ... Environment: DB_Host: mysql-service ...
```

Root cause: The web app expects `mysql-service` but the Service is named `mysql`. In‑cluster DNS requires the Service name to match the hostname the application uses.

Fix applied:

* Delete or replace the Service so its name matches the DB host used by the app. Example:

```bash theme={null}
kubectl delete svc mysql -n alpha
kubectl create -f /tmp/kubectl-edit-3970124164.yaml -n alpha
# service/mysql-service created
kubectl get svc -n alpha
# mysql-service  ClusterIP   10.43.208.47   <none>   3306/TCP
```

Verification:

```bash theme={null}
curl http://localhost:30081
# Page shows green success and DB-connected message:
# Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=root; DB_Password=paswrd;
# From webapp-mysql-84fbfc644f-nhfsv!
```

***

### 2) Namespace: beta — Service `targetPort` wrong (3306 vs 8080)

Symptom: Web app reports connection refused when trying to reach MySQL.

Observed:

```bash theme={null}
2003: Can't connect to MySQL server on 'mysql-service:3306' (111 Connection refused)
```

Investigation:

```bash theme={null}
kubectl config set-context --current --namespace=beta
kubectl get pods
kubectl get svc
kubectl describe svc mysql-service
# Port: 3306/TCP
# TargetPort: 8080/TCP
# Endpoints: 10.42.0.12:8080
kubectl get pods -o wide
# mysql pod IP = 10.42.0.12
```

Root cause: The Service forwards port 3306 to targetPort 8080 on the pod, but MySQL listens on 3306. Traffic hits the wrong container port and is refused.

Fix applied:

* Edit the Service so `targetPort: 3306` (match the MySQL container port):

```yaml theme={null}
# kubectl edit svc mysql-service -n beta
ports:
- port: 3306
  protocol: TCP
  targetPort: 3306
```

Verification:

```bash theme={null}
kubectl describe svc mysql-service -n beta
# Endpoints: 10.42.0.12:3306
# Web app UI now shows the green success page
```

***

### 3) Namespace: gamma — Service selector did not match Pod labels (no endpoints)

Symptom: Web page times out and the Service has no endpoints.

Investigation:

```bash theme={null}
kubectl config set-context --current --namespace=gamma
kubectl get pods
kubectl get svc
kubectl describe svc mysql-service
# Selector: name=mysql
# Endpoints: <none>
kubectl get pods -o wide
# mysql pod IP: 10.42.0.13
kubectl describe pod mysql
# check pod labels that the service selector should match
```

Root cause: The Service selector did not match the pod labels, so Kubernetes did not populate endpoints for the Service. Without endpoints, the Service cannot forward traffic to pods.

Fix applied:

* Edit the Service selector to match the pod labels, or update the pod labels to match the Service selector. Example (edit Service):

```yaml theme={null}
# kubectl edit svc mysql-service -n gamma
selector:
  name: mysql
```

Verification:

```bash theme={null}
kubectl describe svc mysql-service -n gamma
# Endpoints: 10.42.0.13:3306
# Web application loads and shows success.
```

***

### 4) Namespace: delta — Incorrect DB user configured in web app

Symptom: Web app reports access denied for user `sql-user`.

Observed:

```bash theme={null}
Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=sql-user; DB_Password=paswrd;
1045 (28000): Access denied for user 'sql-user'@'10.42.0.16' (using password: YES)
From webapp-mysql-7fbcc4fb8f-dfmnw!
```

Investigation:

```bash theme={null}
kubectl config set-context --current --namespace=delta
kubectl get pods
kubectl describe deployment webapp-mysql
# env: DB_Host: mysql-service
# env: DB_User: sql-user
# env: DB_Password: paswrd
```

Root cause: The application is configured to authenticate as `sql-user`, which does not exist or lacks required privileges on the MySQL server. The intended credential is `root`.

Fix applied:

* Edit the Deployment to set `DB_User: root` and trigger a rollout so the web pod restarts with correct env vars:

```bash theme={null}
kubectl edit deployment webapp-mysql -n delta
# change environment DB_User to "root"
```

Verification:

* After the Deployment updates and a new pod becomes Ready, refresh the web page — it shows green success.

***

### 5) Namespace: epsilon — MySQL root password mismatch

Symptom: Even after DB user is set to `root`, the web app still gets access denied. Inspecting the MySQL pod shows its `MYSQL_ROOT_PASSWORD` differs from the web app's `DB_Password`.

Investigation:

```bash theme={null}
kubectl config set-context --current --namespace=epsilon
kubectl get pods
kubectl describe pod mysql
# Environment:
#   MYSQL_ROOT_PASSWORD: passwoooorrddd  <-- not the expected "paswrd"
kubectl describe deployment webapp-mysql
# env: DB_User: root
#      DB_Password: paswrd
```

Root cause: The MySQL root password configured in the MySQL Pod does not match the password used by the web application.

Fix applied:

* Ensure both the MySQL pod (or its managing controller) and the web deployment use the same credential. Update the MySQL configuration to `MYSQL_ROOT_PASSWORD=paswrd` and recreate/redeploy the MySQL Pod so the new password takes effect.

Example sequence:

```bash theme={null}
kubectl edit pod mysql -n epsilon
# update the env MYSQL_ROOT_PASSWORD to "paswrd"
# If editing the Pod directly is not supported, update the controller and replace the pod:
kubectl replace --force -f /tmp/kubectl-edit-XXXXX.yaml -n epsilon
```

Verification:

* After MySQL restarts with the matching root password and the web app uses `DB_User=root` and `DB_Password=paswrd`, the web UI displays the green success page.

<Callout icon="warning" color="#FF6B6B">
  Do not store credentials in plain text for production. Use Kubernetes Secrets to manage database credentials and ensure deployments reference the same Secret to avoid drift.
</Callout>

***

### 6) Namespace: zeta — NodePort incorrect and credentials mismatch

Symptom: Initially you may see "Bad Gateway" or inability to reach the web app. After fixing NodePort, the web app reports DB access denied.

Investigation & fixes:

1. NodePort mismatch
   * Found web-service NodePort was `30088` while the environment expects `30081`.
   * Fix: Edit the Service to set `nodePort: 30081`.

```bash theme={null}
kubectl config set-context --current --namespace=zeta
kubectl get svc -n zeta
kubectl edit svc web-service -n zeta
kubectl get svc -n zeta
# web-service NodePort 8080:30081/TCP
```

2. DB user and password mismatch
   * Web Deployment used `DB_User: sql-user`. Change it to `root`.
   * MySQL pod had a different `MYSQL_ROOT_PASSWORD`. Update MySQL to use `MYSQL_ROOT_PASSWORD=paswrd` and recreate/restart the pod so the password is applied.

Example changes:

```bash theme={null}
# Update web app deployment
kubectl edit deployment webapp-mysql -n zeta
# Update mysql pod env (or the controller) then replace:
kubectl edit pod mysql -n zeta
# set MYSQL_ROOT_PASSWORD: paswrd
kubectl replace --force -f /tmp/kubectl-edit-XXXXX.yaml -n zeta
```

Verification:

* After correcting the NodePort and synchronizing DB username/password, the web application displays the green success page.

***

## Summary table — Namespace issues & fixes

| Namespace | Symptom | Root cause | Fix |
| -: | - | - | - |
| alpha | Web can't resolve DB host | Service name `mysql` ≠ app DB\_Host `mysql-service` | Rename/recreate Service as `mysql-service` |
| beta | Connection refused to DB | Service targetPort set to 8080 but MySQL listens on 3306 | Set `targetPort: 3306` |
| gamma | Service has no endpoints | Service selector doesn't match Pod labels | Update Service selector or Pod labels so they match |
| delta | Access denied for DB user | App uses `sql-user` (nonexistent) | Set `DB_User: root` (or create the user in DB) |
| epsilon | Access denied (password mismatch) | MySQL root password differs from app `DB_Password` | Sync `MYSQL_ROOT_PASSWORD` with `DB_Password` (use Secrets) |
| zeta | Bad gateway then DB access denied | NodePort wrong; DB user/password mismatch | Set correct `nodePort`, set `DB_User: root`, sync MySQL root password |

***

## Troubleshooting checklist (order to follow)

1. DNS / Service name: does the Service name match the DB host used by the app?
2. Service ports: does Service `port` and `targetPort` map to the container port MySQL listens on (3306)?
3. Service selector: does the selector match the Pod labels so endpoints are populated?
4. App credentials: do the app’s DB\_USER/DB\_PASSWORD match credentials on the MySQL server?
5. Service exposure: if you access the web UI via NodePort, is the NodePort value correct?
6. Logs: check web app and DB logs for authentication/connection errors; use `kubectl logs` and `kubectl describe` to gather context.

***

Following the steps above helps you systematically isolate DNS/name issues, connectivity/port mismatches, selector/endpoint errors, NodePort exposure problems, and credential mismatches — the common causes in this lab.

<CardGroup>
  <Card title="Watch Video" icon="video" cta="Learn more" href="https://learn.kodekloud.com/user/courses/cka-certification-course-certified-kubernetes-administrator/module/aff99955-65fd-443e-950f-3b25d3311bc2/lesson/7cde3120-4b35-493b-bea1-1e48c7759110" />
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.