# xisen-w/startup-success-forecasting-framework architecture

> Architecture diagram of the GitHub repository xisen-w/startup-success-forecasting-framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xisen-w/startup-success-forecasting-framework
- Repository: https://github.com/xisen-w/startup-success-forecasting-framework

## Overview

##### Project Type and Purpose
- The project is a specialized forecasting framework for evaluating startup viability.
- It integrates traditional machine learning (e.g., random forest classifiers, neural networks) with advanced LLM agents (founder, market, product, integration) and real-time external data.
- The README highlights both a simple and an advanced mode of operation along with an emerging web interface.

##### File Structure for Architectural Cues
- Directories like EDA (notebooks for data exploration), algorithms (modules for segmentation, predictions, and preprocessing), models (trained ML models and reports), and agents (various specialized agents) indicate a modular design.
- Files such as overallPipeline.py, baseline_framework.py, and ssff_framework.py suggest multiple pipelines or variations in the processing/evaluation workflow.
- The existence of utility modules (utils/api_wrapper.py, utils/config.py) implies external service integration and configuration management.
- The project also contains build and environment configuration scripts (requirements.txt, .env setup in README, GitHub workflows) that support smooth deployment and CI/CD processes.

a. Main Components:
- Data Ingestion and Preprocessing:
o EDA Notebooks for data exploration.
o Algorithms for data preprocessing and segmentation.
o Utilities that wrap external API access (i.e., SERPAPI, OpenAI).
- Core Machine Learning Pipeline:
o Model training components shown in "models" (random forest, neural network).
o Overall pipeline orchestration files (overallPipeline.py, ssff_framework.py).
- Agent Modules:
o Different “agents” (founder, market, product, integration, vc_scout) that encapsulate specialized evaluation logic.
- User/Interface Layer:
o The web interface (as noted in the README) and main entry point (main.py / app.py) that allow users to run evaluations either in simple or advanced mode.
- Experimentation and Validation:
o Experiment directory and notebooks for testing and evaluation.

b. Relationships and Data Flow:
- Data flowing into preprocessing and transformation modules (algorithms).
- Feed the processed data into the ML models, which generate predictions.
- Agents interfacing with the ML pipeline to provide evaluations:
- For example, the founder_agent refines insights based on founder profiles.
- The market_agent and vc_scout_agent interact with real-time market or external data.
- Finally, the orchestration layer aggregates these outputs and sends them to the web interface for user interaction.

c. Highlighted Architectural Patterns and Technologies:
- Modularity: Clear separation into drivers for data ingestion, ML models, and agent-based evaluations.
- Pipeline Architecture: A sequential flow from data ingestion/preprocessing through ML modeling to decision making.
- Integration design: Wrapping of external APIs using utilities and configuration files.
- Technologies: Python 3.7+, Jupyter (for EDA), machine learning libraries (joblib, keras), and external services (OpenAI, SERPAPI).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External and Utilities Layer
    ED["External Data"]:::external
    subgraph "Utilities & External API Integration"
        UT1["API Wrapper"]:::utilities
        UT2["Config Manager"]:::utilities
    end
    ED --> UT1
    ED --> UT2

    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        EDA1["Data Exploration"]:::ingestion
        EDA2["Data Processing"]:::ingestion
        EDA3["Preprocessing and EDA"]:::ingestion
        EDA4["Data Review"]:::ingestion
    end

    %% Data Segmentation & Preprocessing Algorithms
    subgraph "Data Segmentation & Preprocessing Algorithms"
        ALGO1["LLM Segmentation"]:::algorithms
        ALGO2["Segmentation Prediction"]:::algorithms
        ALGO3["Founder Idea Fit"]:::algorithms
        ALGO4["Neural Networks"]:::algorithms
        ALGO5["Random Forest"]:::algorithms
    end

    %% Core ML Pipeline
    subgraph "Core ML Pipeline"
        subgraph "ML Models"
            ML1["Decision Tree Model"]:::ml
            ML2["Neural Network Model"]:::ml
            ML3["Random Forest Classifier"]:::ml
        end
        subgraph "Orchestration"
            ML4["Overall Pipeline"]:::ml
            ML5["SSFF Framework"]:::ml
            ML6["Baseline Framework"]:::ml
        end
    end

    %% Agent Modules
    subgraph "Agent Modules"
        AGENT1["Founder Agent"]:::agents
        AGENT2["Market Agent"]:::agents
        AGENT3["Product Agent"]:::agents
        AGENT4["Integration Agent"]:::agents
        AGENT5["VC Scout Agent"]:::agents
        AGENT6["Base Agent"]:::agents
    end

    %% Experimentation & Validation
    subgraph "Experimentation & Validation"
        EXP1["Experiment Runner"]:::validation
        EXP2["Experiments Notebook"]:::validation
    end

    %% User Interface
    UI1["Application Entry"]:::ui

    %% Data Flow Connections
    UT1 --> EDA1
    UT2 --> EDA1
    EDA4 --> ALGO1
    ALGO5 --> ML4
    ML4 --> AGENT1
    ML4 --> AGENT2
    ML4 --> AGENT3
    ML4 --> AGENT4
    ML4 --> AGENT5
    ML4 --> AGENT6
    AGENT1 --> ML4
    AGENT2 --> ML4
    AGENT3 --> ML4
    AGENT4 --> ML4
    AGENT5 --> ML4
    AGENT6 --> ML4
    ML4 --> UI1
    ML4 --> EXP1
    ML4 --> EXP2

    %% Click Events for Data Ingestion & Preprocessing
    click EDA1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/EDA/data_exploration.ipynb"
    click EDA2 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/EDA/data_processing.ipynb"
    click EDA3 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/EDA/Preprocessing_and_EDA.ipynb"
    click EDA4 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/EDA/data_review.ipynb"

    %% Click Events for Data Segmentation & Preprocessing Algorithms
    click ALGO1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/algorithms/LLM_Segmentation.py"
    click ALGO2 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/algorithms/Segmentation_Prediction.ipynb"
    click ALGO3 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/algorithms/founder_idea_fit.ipynb"
    click ALGO4 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/algorithms/neuralNetworks.py"
    click ALGO5 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/algorithms/randomForest.py"

    %% Click Events for Core ML Pipeline
    click ML1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/models/decision_tree_model.joblib"
    click ML2 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/models/neural_network.h5"
    click ML3 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/models/random_forest_classifier.joblib"
    click ML4 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/overallPipeline.py"
    click ML5 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/ssff_framework.py"
    click ML6 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/baseline_framework.py"

    %% Click Events for Agent Modules
    click AGENT1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/agents/founder_agent.py"
    click AGENT2 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/agents/market_agent.py"
    click AGENT3 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/agents/product_agent.py"
    click AGENT4 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/agents/integration_agent.py"
    click AGENT5 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/agents/vc_scout_agent.py"
    click AGENT6 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/agents/base_agent.py"

    %% Click Events for Utility & External API Integration
    click UT1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/utils/api_wrapper.py"
    click UT2 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/utils/config.py"

    %% Click Events for Experimentation & Validation
    click EXP1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/experiments/experiment.py"
    click EXP2 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/experiments/experiments.ipynb"

    %% Click Event for User Interface / Application Entry
    click UI1 "https://github.com/xisen-w/startup-success-forecasting-framework/blob/main/app.py"

    %% Style Classes
    classDef external fill:#ddd,stroke:#333,stroke-width:2px;
    classDef utilities fill:#cff,stroke:#333,stroke-width:2px;
    classDef ingestion fill:#f9f,stroke:#333,stroke-width:2px;
    classDef algorithms fill:#ccf,stroke:#333,stroke-width:2px;
    classDef ml fill:#cfc,stroke:#333,stroke-width:2px;
    classDef agents fill:#fcf,stroke:#333,stroke-width:2px;
    classDef validation fill:#ffc,stroke:#333,stroke-width:2px;
    classDef ui fill:#fcc,stroke:#333,stroke-width:2px;
```
