How rs-010806/evolve-mem works
Project Type and Purpose
• EVOLVE-MEM is a research‐oriented, full‐stack Python system for self‐evolving hierarchical memory in agentic AI.
• It ingests raw experiences, embeds them, organizes them into a three‐tier memory hierarchy, serves queries at different abstraction levels, evaluates performance with SOTA metrics, and self‐improves over time.
File Structure
• Top‐level modules correspond to each major tier and function:
• dynamic_memory.py → Tier 1: embedding & storage
• hierarchical_manager.py → Tier 2: clustering & summarization
• self_improvement.py → Tier 3: performance monitoring & reorganization
• llm_backend.py → Gemini LLM integration
• memory_system.py → Main façade tying all tiers together
• evaluation.py & test_system_performance.py → Metrics & tests
• dataset_loader.py → Data ingestion
• run_comprehensive_pipeline.py → Orchestrates full research pipeline
• config.py, utils.py → Shared config & helper functions
• requirements.txt & .gitignore are standard. There are also directories for datasets, logs, and results.
Key Architectural Patterns
• Layered (three‐tier) memory hierarchy
• Pipeline/orchestration pattern for ingestion → organization → query → evaluation → improvement
• External service integration (Gemini LLM)
• Persistent storage (ChromaDB + JSON dumps)
Main Components
a. Data Ingestion Layer
• Dataset Loader (LoCoMo or custom)
• SentenceTransformer → Embedding
• In‐memory dictionary + ChromaDB (vector store)
b. Tier 1: Dynamic Memory Network
• Raw experience store (embeddings + metadata)
• Fast semantic search
c. Tier 2: Hierarchical Memory Manager
• Level 0 → raw notes
• Level 1 → KMeans clusters → LLM summaries
• Level 2 → Meta‐clusters → LLM abstractions
• JSON persistence for hierarchy
d. Tier 3: Self‐Improvement Engine
• Monitors accuracy, retrieval balance, latency
• Triggers re‐clustering, parameter tuning
e. Query & Retrieval Engine
• Query router (classifies by complexity)
• Multi‐level retrieval paths
• LLM answer extraction & post‐processing
• Logs retrieval path & similarity scores
f. Evaluation & Metrics
• Metric computations (Accuracy, BLEU-1, ROUGE, METEOR, SBERT)
• Reporting (results/, logs/)
g. LLM Service
• Gemini API (summarization, abstraction, reasoning)
Relationships and Data Flow
• Tier 1 → Tier 2 (clustering & summarization loops)
• Tier 2 → Tier 3 (self‐improvement feedback loop)
• External LLM used by Tier 2 & Query Engine
• Query Layer ↔ All memory tiers → returns answers
• Evaluation receives answers + ground truth → reports → feeds Self‐Improvement
• Pipeline orchestrator (run_comprehensive_pipeline.py) triggers all stages end‐to‐end
Architectural Patterns & Principles
• Separation of Concerns: distinct modules for memory storage, management, self‐improvement, LLM integration, evaluation
• Feedback Loop: evaluation → self‐improvement → memory reorganization
• Persistent State: vector DB + JSON dumps ensure reproducibility
• [ ] Highlight feedback loops (evaluation → self‐improvement → memory manager)