Skip to content

Data & Database Overview

The system uses a single PostgreSQL instance with TimescaleDB and pgvector for all persistent data, and Redis for caching. The application accesses data through a repository pattern and higher-level services that orchestrate repositories and optional cache.

Data Flow

flowchart LR
    subgraph Application
        Orch[Orchestrator / API]
        Svc[Services]
    end

    subgraph Services
        ADS[AgentDatabaseService]
        DDS[DebateDatabaseService]
        APS[AgentPerformance]
        SPS[StockProfileService]
        AC[AgentCache]
    end

    subgraph Repositories["backend.database.repositories"]
        AAR[AgentAnalysisRepository]
        AMR[AgentMemoryRepository]
        DBR[Debate Repositories]
        SPR[StockProfileRepository]
        SR[SignalRepository]
        CR[CostRepository]
    end

    subgraph Storage
        subgraph PG["PostgreSQL (TimescaleDB + pgvector)"]
            T1[agent_analyses\nagent_memory\nsignals]
            T2[debate_sessions\ndebate_messages\ndebate_votes]
            T3[stock_profile\nfundamental_metrics]
            T4[sessions\ncosts]
        end
        Redis[(Redis\nCache)]
    end

    Orch --> Svc
    ADS --> AAR
    ADS --> AMR
    ADS --> AC
    DDS --> DBR
    APS --> AAR
    SPS --> SPR
    SPS --> AC

    AAR --> PG
    AMR --> PG
    DBR --> PG
    SPR --> PG
    SR --> PG
    CR --> PG
    AC --> Redis

Flow summary: The orchestrator or API calls services (e.g. AgentDatabaseService, DebateDatabaseService, StockProfileService). Services use repositories for persistence and, where configured, AgentCache or other cache clients for Redis. Repositories read and write to PostgreSQL; cache clients read and write to Redis.

Storage Layout

Store Purpose Key contents
PostgreSQL (TimescaleDB) Relational and time-series data Agent analyses, agent memory (with embeddings), debate sessions/messages/votes, signals, sessions, costs
PostgreSQL (pgvector) Vector similarity search agent_memory.embedding (1536-dim); HNSW index for nearest-neighbor queries
Redis Application cache Latest agent analyses per symbol/type (AgentCache), stock profile data; TTL-based expiry

All database models and repository APIs live under backend.database (models in backend.database.models, repositories in backend.database.repositories). Session and cost data use the same PostgreSQL instance.

Topic Document
Agent persistence, cache, debate lifecycle Agent-Database Integration
Vector columns, HNSW, similarity search pgvector Setup
Connection settings, Redis keys Configuration Reference — Database
Migrations, maintenance, connectivity DevOps Operations — Database
Repository pattern and usage Architecture — Data access and repository pattern