How manish-kotra/algorithmic-trading works
Main Components (the “modules”
a. Data Collection Layer:
• External Data Sources: Represent external services and websites/web pages where data is obtained, such as bitcoininfocharts, finviz, and Google News.
• Webscraping Notebooks: Include components like “webscrapper_bitcoinfocharts.ipynb”, “webscrapper_finviz.ipynb”, and “webscrapper_googlenews.ipynb” which collect raw data.
b. Data Preprocessing & Feature Engineering Layer:
• Data Cleaning/Preprocessing: Represent notebooks or scripts such as “bitcoin_feature_engineering.ipynb” and possibly “detrendPrice.py”, which prepare and normalize data.
• Data Consolidation: Show how processed datasets (for example, “bitcoin_techindicators_filtered_normalized.csv” and “main.csv”) flow to the next stage.
c. Sentiment Analysis Layer:
• VADER Sentiment Analysis: A module for “news_sentiment_analysis_vader.ipynb” that processes news headlines using predefined sentiment rules.
• BERT Sentiment Analysis: A module for “news_sentiment_analysis_bert.ipynb” that leverages transformer models, outputting sentiment metrics.
• Data Storage: CSVs from sentiment analysis (like “bitcoin_news_data_sentiment_vader.csv” and “bitcoin_news_data_sentiment_finbert.csv”) should be depicted as outputs of these processes.
d. Exploratory and Correlation Analysis:
• EDA and Visualization: Include “bitcoin_tsa.ipynb” and “news_btc_correlation.ipynb” which facilitate exploratory data analysis and establish relationships between market data and sentiment.
• Results and Artifacts: Highlight generated insights such as “grid_search_results.pkl” (for model tuning) and result figures (from the figures folder).
e. Model Tuning and Strategy Evaluation:
Relationships and Data Flow
• That processed data (via feature engineering) and sentiment results are merged or passed together into model training and evaluation steps.
Key Architectural Patterns & Design Principles
• Data Pipeline: Emphasize the stepwise flow of data – collection, cleaning/processing, feature engineering, analytics, and final evaluations.
• Modularity: Illustrate how each notebook or script represents a distinct functional component. This separation of concerns aids in maintenance and further extension.
• Reproducibility & Environment: Mention the “environment.yml” which ensures reproducible builds; this underpins the technical stack (Python, conda environment).