Skip to content

Docker Setup and Healthchecks

Services Overview

  • PostgreSQL with TimescaleDB + pgvector (port 5432) - Time-series database and vector store
  • Container: redhound-postgres
  • Redis (port 6379) - Caching layer
  • Container: redhound-redis
  • Redhound application (port 8000) - Main framework
  • Container: redhound-app
  • Built from Dockerfile; image tagged redhound:latest
  • Prometheus (port 9090) - Metrics collection and UI
  • Container: redhound-prometheus
  • Alertmanager (port 9093) - Alert routing and notifications (Slack webhooks)
  • Container: redhound-alertmanager
  • Built from docker/alertmanager/Dockerfile; image tagged redhound-alertmanager:local
  • Grafana (port 3000) - Metrics visualization and dashboards
  • Container: redhound-grafana

Grafana Service

Configuration

  • Container: redhound-grafana
  • Port: 3000 (configurable via GRAFANA_PORT environment variable)
  • Access URL: http://localhost:3000

Environment Variables

  • GF_SECURITY_ADMIN_USER: Grafana admin username (defaults to admin if not set)
  • GF_SECURITY_ADMIN_PASSWORD: Grafana admin password (defaults to admin for local development if not set)
  • Note: Grafana uses GF_ prefix for all configuration environment variables

Production Security

Default credentials (admin / admin) are for local development only. Always set GF_SECURITY_ADMIN_PASSWORD to a strong, unique password in .env before deploying to any non-local environment.

Accessing Grafana UI

  1. Start Grafana service: docker-compose up -d grafana (or docker-compose up -d for all services)
  2. Open browser: Navigate to http://localhost:3000
  3. Login:
  4. Local development: Default credentials are admin / admin (when GF_SECURITY_ADMIN_PASSWORD is not set in .env)
  5. Production/non-local: Use credentials from .env file (GF_SECURITY_ADMIN_USER / GF_SECURITY_ADMIN_PASSWORD)
  6. Navigate to dashboards: Click "Dashboards" → "Redhound" folder to see pre-provisioned dashboards
  7. Verify metrics: Ensure REDHOUND_METRICS_ENABLED=true for metrics to appear in dashboards

Pre-Provisioned Components

  • Data Source: Prometheus data source automatically configured (no manual setup required)
  • Dashboards: Three starter dashboards pre-provisioned:
  • Redhound Overview (comprehensive system metrics)
  • Redhound Vendor/Operation Latency (detailed latency breakdowns)
  • Redhound Cache Performance (cache-specific metrics)
  • All dashboards are organized in the "Redhound" folder in Grafana UI

Running the stack

  • First run or after code changes: make up-build (builds images and starts all services).
  • Start only (image already built): make up.
  • Stop: make down; logs: make logs; container status: make docker-ps.
  • The app is exposed on the host at http://localhost:8000 (override with APP_PORT in .env). Use curl http://localhost:8000/health to verify.

Persistent Storage

  • Grafana data (dashboards, users, preferences) stored in grafana_data volume
  • Data persists across container restarts
  • Provisioning configs mounted read-only from docker/grafana/provisioning/

Application Service Configuration

The application service uses a configurable entrypoint script that allows customization of uvicorn startup parameters via environment variables.

Environment Variables for Application Service

  • APP_MODULE: Application module path (default: redhound.api.app:app)
  • APP_HOST: Bind host (default: 0.0.0.0)
  • APP_PORT: Bind port (default: 8000)
  • UVICORN_WORKERS: Number of worker processes (default: 1)
  • UVICORN_RELOAD: Enable auto-reload for development (default: false)
  • EXTRA_UVICORN_ARGS: Additional uvicorn arguments (space-separated)

Default Startup Command

By default, the application starts with: 

uvicorn redhound.api.app:app --host 0.0.0.0 --port 8000 --workers 1

Resource Limits

  • CPU: 2.0 limit, 0.5 reservation
  • Memory: 4G limit, 512M reservation

Image Contents and ML Dependencies

The application image is built from pyproject.toml and uv.lock; all Python dependencies are installed at build time. The image includes ML dependencies required by the deterministic sentiment analyst:

  • PyTorch (torch) — runtime for transformer models
  • Hugging Face Transformers (transformers) — DeBERTa-v3 model for sentiment analysis

No separate Dockerfile or docker-compose changes are required for the sentiment analyst; it runs with the same app service. Model loading uses additional memory; the default 4G memory limit is sufficient for a single DeBERTa-v3 model. Under heavy load or multiple workers, monitor memory and adjust the limit if needed.

Logging Configuration

Docker logging driver matches application log rotation: - Driver: json-file - Max size: 10m (matches REDHOUND_LOG_MAX_BYTES) - Max files: 5 (matches REDHOUND_LOG_BACKUP_COUNT)

Healthcheck Configuration (docker-compose)

  • App healthcheck: Uses the healthcheck script at /usr/local/bin/healthcheck.
  • Timing: interval 30s, timeout 2s, retries 3, start_period 40s.
  • Defined in docker-compose.yml under the app service.
  • The healthcheck script validates:
  • /health endpoint (required - must return 200 for container to be healthy)
  • /metrics endpoint (optional - 503 is acceptable if metrics are disabled)
  • Expectations:
  • Returns 200 when required dependencies are healthy.
  • Returns 503 when any required dependency (PostgreSQL, Redis) is unhealthy.

Healthcheck Configuration (Dockerfile)

  • Runtime HEALTHCHECK mirrors compose settings: interval 30s, timeout 2s, retries 3, start_period 40s.
  • Uses the healthcheck script at /usr/local/bin/healthcheck which checks both /health and /metrics endpoints.
  • The healthcheck script (docker/healthcheck.sh) validates:
  • /health endpoint (required - must return 200 for container to be healthy)
  • /metrics endpoint (optional - 503 is acceptable if metrics are disabled)
  • Runtime app: uvicorn redhound.api.app:app --host 0.0.0.0 --port 8000 (unified health + metrics).

Entrypoint Script

The application uses a deterministic entrypoint script (docker/entrypoint.sh) that allows configurable uvicorn startup via environment variables:

Environment Variables

  • APP_MODULE: Application module path (default: redhound.api.app:app)
  • APP_HOST: Bind host (default: 0.0.0.0)
  • APP_PORT: Bind port (default: 8000)
  • UVICORN_WORKERS: Number of worker processes (default: 1)
  • UVICORN_RELOAD: Enable auto-reload for development (default: false)
  • EXTRA_UVICORN_ARGS: Additional uvicorn arguments (space-separated)

Default Behavior

By default, the entrypoint starts: 

uvicorn redhound.api.app:app --host 0.0.0.0 --port 8000 --workers 1

Customization Examples

# Use multiple workers
docker run -e UVICORN_WORKERS=4 redhound-app

# Enable reload for development
docker run -e UVICORN_RELOAD=true redhound-app

# Custom module and port
docker run -e APP_MODULE=redhound.api.app:app -e APP_PORT=9000 redhound-app

# Add extra arguments
docker run -e EXTRA_UVICORN_ARGS="--log-level debug" redhound-app

Healthcheck Script

The healthcheck script (docker/healthcheck.sh) performs the following checks:

  1. Required check: /health endpoint must return 200 for the container to be considered healthy
  2. Optional check: /metrics endpoint (503 is acceptable if metrics are disabled)

The script uses curl with a 2-second timeout for each endpoint. The healthcheck respects the APP_HOST and APP_PORT environment variables (defaults: 127.0.0.1:8000).

Verifying Health

  • Container status: make docker-ps or docker-compose ps (shows healthy/unhealthy).
  • From host: curl -i http://localhost:8000/health (app port is published by compose).
  • From inside container: docker exec redhound-app curl -i http://localhost:8000/health.
  • Startup period: during start_period the container may show starting; it should transition to healthy once dependencies are reachable and the endpoint responds 200.

Troubleshooting Healthcheck Failures

  • Check logs: docker-compose logs app | tail -n 100.
  • Inspect dependencies:
  • PostgreSQL: docker-compose ps postgres; logs with docker-compose logs postgres.
  • Redis: docker-compose ps redis; logs with docker-compose logs redis.
  • Manual probes:
  • Postgres connectivity from app: docker exec redhound-app pg_isready -h postgres -p 5432 -U ${POSTGRES_USER:-redhound} (if client installed).
  • Redis connectivity from app: docker exec redhound-app redis-cli -h redis -p 6379 ping (if client installed).
  • Verify config:
  • Ensure REDHOUND_HEALTHCHECK_ENABLED=true (default) and endpoint path matches healthcheck URL.
  • Confirm timeouts are not set too low for your environment (REDHOUND_HEALTHCHECK_TIMEOUT_DEFAULT, per-dependency overrides).
  • If health remains unhealthy:
  • Curl the endpoint and inspect errors in the JSON response to see which dependency failed.
  • Ensure uvicorn is running and listening on port 8000 inside the container.