How abcasas/capstones_cidaen_22_24 works
Main Components
• Data Source & Ingestion: Several modules have “data” folders containing CSV files and backups (for example, in “mc3-m01”, “mc3-m02”, etc.). This indicates that the project begins with raw data acquisition.
• Data Preparation / Storage: The “mc3-m02-Acquisition, preparation, and storage of data” module shows that data ingestion is complemented by steps to prepare and store data.
• Exploratory Data Analysis (EDA): The “mc3-m03-Exploratory Data Analysis” folder encapsulates notebooks for analyzing and visualizing data.
• Machine Learning & Deep Learning: Modules such as “mc3-m04-Fundamentals of Machine Learning” and “mc3-m06-Deep Learning” represent the model building and training components.
• Specialized Analyses: Modules on “Time series” and “Text Mining, Networks, and Recommendation Systems” point to domain-specific analytical components.
• Big Data Architectures: The “mc3-m12-Big Data Architectures and Processes” module (with Docker configurations and notebooks) suggests integration with big data workflows and environments.
• Deployment & Services: The “mc3-m14-Creation and Deployment of Services” includes deployment scripts, Docker files, and service application components (app.py, appspec, etc.), highlighting the transformation of analytics into deployable services.
• Support/Infrastructure Components: Dockerfiles, docker-compose, and .devcontainer configurations are used for containerization and development environment consistency.
Relationships and Interactions
• Modular and Independent Capstones: Although each module stands on its own (as evident from separate folders and capstone notebooks), later modules such as deployment and big data architectures can be seen as integrating or building upon insights from earlier stages.
• Integration Points: Highlight the role of containerization and orchestration (Docker, docker-compose) in facilitating consistent environments and deployment strategies.
Architectural Patterns and Design Principles
• Data Pipeline Pattern: The project naturally follows a pipeline pattern—from acquisition, cleaning, exploration, and modeling to deployment.
• Modularity: Each module is self-contained, which makes it easier to update individual capstone projects as needed, demonstrating a highly modular design.
• Containerization: In the “Time series”, “Big Data”, and “Service Deployment” modules, the use of Docker (Dockerfile and docker-compose.yml) supports a modern, containerized deployment approach.
• Clear Separation of Concerns: The structure clearly separates data preparation, analysis, model training, and service deployment, which encourages maintainability and scalability.
Relevant Technologies and Frameworks
• Jupyter Notebooks: Most of the logic and analysis is implemented through notebooks (Capstone I–VIII, etc.).
• Data Handling Tools: CSV files indicate the use of standard data manipulation libraries (likely pandas in Python).
• Docker: Presence of Dockerfiles and docker-compose indicates containerization is part of some modules (particularly in “mc3-m07-Time series”, “mc3-m12-Big Data Architectures”, and “mc3-m14-Creation and Deployment of Services”).
• Python and Related Libraries: Although not explicitly listed, the naming and layout strongly suggest Python as the primary programming language for analysis, data science, and modeling.
• Deployment Scripts: The “scripts” folders and appspec files are part of automating service deployment.