How symphony-professional-services/chat-factory works
Main Components
• Data Generation Engine – Represented primarily by main.py.
• It uses configuration settings from config.py.
• It reads taxonomy definitions from taxonomy.json.
• It constructs inputs (e.g., advisor, client, topics, categories) and then calls a language model (or placeholder) to generate conversation data.
• Synthetic Data Storage – Represented by the synthetic_data directory.
• Contains subdirectories for each advisor and corresponding JSON files (one per client) where generated conversations are stored.
• Metrics and Analytics Modules – Encompassing metrics.py and display_metrics.py.
• These components analyze conversation logs (log files and metrics_log.csv) to provide insights and visualizations of usage and conversation metrics.
• Utility and Automation Scripts:
• run.sh (or run_and_log.sh) acts as a wrapper to automate the data generation process and logging.
• process_synthetic_data.py is used for further post-processing of the generated data.
Out Relationships and Interactions
• The starting point is execution via run.sh or run_and_log.sh. This script triggers the data generation by calling main.py.
• In main.py, the following interactions occur:
• It reads configuration (config.py) and taxonomy (taxonomy.json).
• It uses randomness to select advisors and clients.
• It constructs and logs generated synthetic chat data.
• It persists the output JSON files into synthetic_data (organized by advisor names).
• Separately, the metrics.py (and associated display_metrics.py) module picks up the logged data (metrics_log.csv, synthetic_chat_generator.log, and run_and_log.log) to compute analytics.
• process_synthetic_data.py provides an alternative script path for specialized data processing needs.
Architectural Patterns and Design Principles
• Modularity: The project is clearly divided into separate modules for generation, storage, and analytics.
• Separation of Concerns: Configuration, data generation, logging, and processing each have their own dedicated scripts and files.
• Extensibility: With external taxonomy and configuration files (taxonomy.json and config.py), the system is designed to be easily extended to include more advisors, topics, or client profiles.
• From the Synthetic Chat Generator, branch out to:
• A “Configuration & Taxonomy” block (config.py and taxonomy.json) indicating the data used as input.
• Optionally, include a “Logging” component that shows logging of both data generation and metric collection errors/information.
• That configuration and taxonomy files are the “knowledge base” for the synthetic conversation generation process.
In Summary
• The input components (config.py, taxonomy.json).
• The central synthetic data generator (main.py) and its control flow from run.sh.
• The storage mechanism for generated data (synthetic_data directory and JSON files) with clear mapping to advisor and client structure.
• The analytics modules (metrics.py, display_metrics.py) that consume logs and metrics for insights.