How hpcclab/action_engine works
Project Type & Purpose
• This is neither a typical web app nor a compiler; it’s an end-to-end workflow-generation pipeline/tool that ingests a natural language query and emits an executable, platform-agnostic FaaS workflow. It also contains extensive evaluation and data-preparation sub-systems.
Core Modules & Directory Structure
• actione_engine_reverse/: “reverse chain” LLM pipeline for alternate evaluation
• main_rae.py — entry point
• utils/ — config, LLM wrappers, module runners, data schemas
• db/ & db_old/: CSV/JSON + FAISS vector-store construction for API metadata
• utils/ (root): shared compiler, dependency analyzer, LLM abstractions, schema definitions
• eval/: ground-truth vs. predicted comparison logic, confidence intervals, plotting, Jupyter notebooks
• output_file/: compiled Argo YAML
• main.py + requirements.txt + Dockerfile: CLI orchestration, containerization
High-Level System Components
A. User Interface (CLI / REST)
B. Text-to-Workflow Engine
1. Configuration Loader (utils/config.py)
2. API Info Database (db/*.csv/.json + FAISS index)
3. LLM Orchestration (utils/llms.py + actione_engine_reverse/utils/llms.py)
4. Function Selector & Dependency Analyzer (utils/func_identifier.py, data_dependency.py, modules.py)
5. DAG Constructor (schemas/workflow.py, wf_optimizer.py)
6. Workflow Compiler (utils/argo_compiler.py → Argo YAML)
C. Execution Interface (API Endpoint generator, output_file/argo_workflow.yaml)
D. Evaluation Subsystem
1. Ground-truth loader (eval/data/gt)
2. Prediction loader (eval/data/predict)
3. Metrics calculators (eval/culculate.py, confidence_interval.py)
4. Plotting & Reports (eval/plot_*.py, eval/plots)
Component Relationships & Data Flow
• User → issues natural-language query (CLI or REST)
• → Config Loader reads parameters
• → API DB + FAISS returns candidate API functions
• → LLM Orchestrator calls external LLM service (GPT-4o, Qwen, etc.) with prompts (few-shot, zero-shot, tool-augmented)
• → Function Selector chooses functions, identifies data dependencies
• → DAG Constructor builds a directed graph of function calls
• → Workflow Compiler translates DAG into platform-independent YAML (Argo / other FaaS orchestrators)
• → Output: compiled workflow (output_file/) and optionally RESTful API endpoint definitions
• Evaluation taps ground-truth vs. predictions, computes F1, order accuracy, and renders plots
Architectural Patterns & Principles
• Pipeline Pattern: clear stage separation (selection → dependency → DAG → compile)
• Plugin/Modular Design: utils/ and actione_engine_reverse/utils/ define interchangeable LLM backends and compilation targets
• Event-Driven/Eager Dataflow: DAG-based workflow representation
• Separation of Concerns: distinct directories for data preparation (db/), inference (actione_engine_reverse/), evaluation (eval/)
Technologies & External Services
• Python 3.x, Jupyter notebooks for exploration
• FAISS vector database for semantic API retrieval
• External LLM endpoints (OpenAI GPT-4o, local quantized Qwen-GPTQ)
• Argo Workflows (YAML) for FaaS orchestration
• Docker for containerization
• Matplotlib / PDF for plots