api7-demo/web/docs/service-discovery.md

Service Discovery

Learn how API7 Gateway dynamically discovers backend services using Kubernetes service discovery.

Overview

Service Discovery enables API7 Gateway to automatically discover and track backend Pod endpoints instead of using static upstream node configuration. This provides a dynamic, scalable, and resilient architecture.

Why Use Service Discovery?

Traditional Static Configuration

upstream:
  nodes:
    - host: 10.0.1.23
      port: 8080
      weight: 100
    - host: 10.0.1.24
      port: 8080
      weight: 100

Problems:

• ❌ Manual updates required when Pods scale
• ❌ Stale Pod IPs after deployments
• ❌ No automatic health checks
• ❌ Downtime during updates

With Kubernetes Service Discovery

upstream:
  discovery_type: kubernetes
  service_name: my-service
  namespace_id: default

Benefits:

• ✅ Automatic Pod discovery
• ✅ Real-time updates during scaling
• ✅ Health checks built-in
• ✅ Zero downtime deployments

How It Works

graph LR
    A[API7 Gateway] -->|Query Endpoints| B[Kubernetes API]
    B -->|Return Pod IPs| A
    A -->|Route Traffic| C[Pod 1]
    A -->|Route Traffic| D[Pod 2]
    A -->|Route Traffic| E[Pod 3]

    F[HPA/Deploy] -->|Scale/Update| B
    B -->|Notify| A

1. API7 Gateway watches Kubernetes Endpoints API
2. Kubernetes API returns list of healthy Pod IPs
3. Gateway automatically adds/removes Pods from upstream
4. Traffic is load-balanced across discovered Pods

Configuration

Enable in Helm Chart

Edit values.yaml:

api7:
  serviceDiscovery:
    enabled: true  # Enable Kubernetes service discovery
    namespace: ""  # Leave empty to use release namespace

ADC Configuration

The Helm chart automatically generates ADC configuration with service discovery:

services:
  - name: my-service
    upstream:
      name: my-upstream
      type: roundrobin
      discovery_type: kubernetes
      service_name: my-service  # Kubernetes Service name
      namespace_id: default     # Kubernetes namespace

RBAC Requirements

Service Discovery requires permissions to watch Kubernetes resources. These are automatically configured in rbac-adc.yaml:

rules:
  # Allow reading services and endpoints for service discovery
  - apiGroups: [""]
    resources: ["services", "endpoints"]
    verbs: ["get", "list", "watch"]

Use Cases

1. Auto-Scaling Applications

When your application scales:

# Scale deployment
kubectl scale deployment my-app --replicas=10

# API7 Gateway automatically discovers new Pods
# No configuration changes needed!

2. Rolling Updates

During deployments:

# Start rolling update
kubectl rollout restart deployment my-app

# Gateway automatically:
# 1. Discovers new Pods
# 2. Adds them to upstream
# 3. Removes old Pods
# 4. Zero downtime!

3. Multi-Service Architecture

Discover multiple backend services:

services:
  - name: web-frontend
    upstream:
      discovery_type: kubernetes
      service_name: web-frontend
      namespace_id: production

  - name: api-backend
    upstream:
      discovery_type: kubernetes
      service_name: api-backend
      namespace_id: production

  - name: admin-api
    upstream:
      discovery_type: kubernetes
      service_name: admin-api
      namespace_id: admin

Advanced Configuration

Cross-Namespace Discovery

Discover services in different namespaces:

upstream:
  discovery_type: kubernetes
  service_name: shared-service
  namespace_id: shared-services  # Different namespace

!!! warning "RBAC Permissions" Ensure the ServiceAccount has permissions to watch Endpoints in the target namespace.

Load Balancing Algorithms

Combine service discovery with load balancing:

upstream:
  discovery_type: kubernetes
  service_name: my-service
  namespace_id: default
  type: roundrobin  # or chash, least_conn, ewma

Available algorithms:

• roundrobin: Distribute requests evenly (default)
• chash: Consistent hashing (sticky sessions)
• least_conn: Send to Pod with fewest connections
• ewma: Exponentially weighted moving average

Health Checks

API7 Gateway respects Kubernetes readiness probes:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: app
    readinessProbe:
      httpGet:
        path: /health
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 3

Only Pods with passing readiness checks receive traffic.

Monitoring

Check Discovered Endpoints

View upstream nodes in API7 Dashboard:

# Port-forward to dashboard
kubectl port-forward -n api7ee svc/api7ee3-0-1759339083-dashboard 7443:7443

# Open browser
open https://localhost:7443

Navigate to: Gateway Groups → Upstreams → View Nodes

Debug Service Discovery

Check ADC sync logs:

# View ADC sync job logs
kubectl logs -n api7ee job/api7ee-demo-api7ee-demo-k8s-adc-sync

# Look for service discovery messages
# [INFO] Discovered 3 endpoints for service my-service

Troubleshooting

No Endpoints Discovered

Symptoms: Gateway shows 0 upstream nodes

Causes:

1. Service doesn't exist
2. No Pods match Service selector
3. All Pods are not ready

Solution:

# Check Service exists
kubectl get service my-service -n default

# Check Endpoints
kubectl get endpoints my-service -n default

# Check Pod readiness
kubectl get pods -l app=my-service -n default

RBAC Permission Denied

Error: forbidden: User "system:serviceaccount:..." cannot list endpoints

Solution: Verify RBAC permissions:

# Check Role binding
kubectl get rolebinding -n api7ee | grep adc

# Describe permissions
kubectl describe role api7ee-demo-api7ee-demo-k8s-adc -n api7ee

Stale Endpoints

Symptoms: Traffic sent to terminated Pods

Causes: Kubernetes API not updating fast enough

Solution: Check Kubernetes API health:

# Verify API server is responsive
kubectl get --raw /healthz

# Check endpoint update timestamps
kubectl get endpoints my-service -o yaml | grep -i time

Best Practices

1. Use Service Discovery for All Dynamic Workloads

✅ Do:

upstream:
  discovery_type: kubernetes
  service_name: my-dynamic-app

❌ Don't:

upstream:
  nodes:
    - host: pod-ip  # Brittle, breaks on restart

2. Configure Readiness Probes

Always configure readiness probes on Pods:

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5

3. Use Namespaces for Isolation

Separate environments by namespace:

# Production
namespace_id: production

# Staging
namespace_id: staging

# Development
namespace_id: development

4. Monitor Discovery Performance

Set up alerts for:

• Number of discovered endpoints
• Service discovery latency
• RBAC permission errors

5. Test Failover Scenarios

Regularly test:

• Pod termination
• Rolling updates
• Scale up/down events

Comparison: Static vs Service Discovery

Feature	Static Nodes	Service Discovery
Setup	Manual	Automatic
Scaling	Manual updates	Automatic
Health checks	Manual	Built-in
Deployments	Downtime risk	Zero downtime
Maintenance	High	Low
Flexibility	Low	High
Production-ready	❌	✅

Next Steps

• Configure Secret Management
• Set up Ingress Routing
• Monitor Gateway Metrics

References

• API7 Service Discovery Documentation
• Kubernetes Endpoints API
• Kubernetes Service Discovery