How ameliawong1996/from_news_to_forecast works
Project type and purpose
• This is a machine-learning research system that integrates unstructured news data with numerical time series data to improve forecasting accuracy using Large Language Models (LLMs) and generative agents.
• Key use cases: electricity load forecasting, exchange‐rate prediction, Bitcoin price prediction, traffic volume forecasting.
File‐structure analysis
• Top‐level directories:
• Agent/ – contains the generative‐agent code, data, and notebooks for iterative event reasoning.
• data/ – raw and paired time series & news data.
• code files at root: llm-finetune, model_download, validation, evaluation notebooks.
• utils/, templates/ – supporting code and template prompts for LLM interactions.
• Patterns:
• Data ingestion & preprocessing (data/raw_*, Agent/Data_all).
• Agent‐based news selection & reasoning (Agent/code, AgentBuilding.ipynb).
• Fine‐tuning pipeline (llm-finetune.py, finetune-lls4ts.sh).
• Validation & evaluation (validation.py, Evaluation.ipynb, accuracy-ts.py).
Architecture components
a. Data Sources (on left):
– Numerical time series CSVs (energy, exchange, bitcoin, traffic)
– Raw news JSONs (AU-news, finance, etc.)
– Supplementary data (weather, calendar, economic indicators)
b. Data Processing Layer:
– Raw data loader & cleaner (scripts in Agent/Data_all and data/raw_*)
– News preprocessing (tokenization, filtering)
– Time-series pairing logic (paired_time_series_news_training_data)
c. Generative Agent Component:
– LLM‐based news selector (Agent/code/gpt_news_evaluation.py, select_news_logic)
– Reflection loop for iterative reasoning (AgentBuilding.ipynb)
d. Fine-tuning Pipeline:
– Model download (model_download.py)
– Deepspeed‐driven training (llm-finetune.py, finetune-lls4ts.sh)
– LoRA adapters and hyperparameter configuration
e. Inference & Validation:
– Validation script (validation.py)
– Prompt templates (templates/ts_test.json)
– Metrics calculation (Evaluation.ipynb, accuracy-ts.py)
f. Outputs & Visualization:
– Forecast JSON results
– Plots and comparison charts (plot_comparision.py)
– Notebooks summarizing results
Relationships & data flow
• Process blocks feeding cleaned & paired datasets into both the Agent Component and directly into Fine-tuning.
• Agent outputs (selected news events) feed back into the paired dataset.
• Paired dataset flows into the Fine-tuning block.
• Trained model flows into Inference & Validation.
• Validation outputs feed into Evaluation & Visualization.
Key architectural patterns & design principles
• Modular pipeline: clear separation of data ingestion, agent reasoning, model training, and evaluation.
• Iterative feedback loop: Agent’s reflection loop refines news selection before re-training.
• Reusable components: prompt templates, utility functions, LoRA adapters.
• Scalability: deepspeed for distributed training, configurable data loaders.