# dennislamcv1/ibmdataengineering architecture

> Architecture diagram of the GitHub repository dennislamcv1/ibmdataengineering, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dennislamcv1/ibmdataengineering
- Repository: https://github.com/dennislamcv1/ibmdataengineering

## Overview

This repository is a collection of learning materials for the IBM Data Engineering Professional Certificate, not a runtime application. A learner studies programming, databases, and data-engineering topics, then applies them in labs and projects and builds a portfolio; the README also describes certificate and career outcomes. The graph groups the visible course directories into curriculum areas. It represents documented learning capabilities rather than executable calls or a required course sequence; relationships are based on the README, since most source content was not sampled.

Main entry points: [`Python for Data Science and AI`](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Python%20for%20Data%20Science%20and%20AI) (Python), [`Hands-on Introduction to Linux Commands and Shell Scripting`](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Hands-on%20Introduction%20to%20Linux%20Commands%20and%20Shell%20Scripting) (Linux and Shell), [`Databases and SQL for Data Science with Python`](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Databases%20and%20SQL%20for%20Data%20Science%20with%20Python) (SQL), [`Introduction to Relational Databases`](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Introduction%20to%20Relational%20Databases) (Relational Databases)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_foundations["Foundations"]
  node_python["Python"]
  node_linux["Linux and Shell"]
end

subgraph group_data_platforms["Data Platforms"]
  node_sql["SQL"]
  node_relational["Relational Databases"]
  node_nosql["NoSQL Databases"]
  node_warehouse["Data Warehousing"]
end

subgraph group_engineering_analytics["Engineering and Analytics"]
  node_etl["ETL Pipelines"]
  node_bigdata["Big Data and Spark"]
  node_spark_ml["Spark Machine Learning"]
  node_bi["BI Dashboards"]
end

subgraph group_projects_career["Projects and Career"]
  node_python_project["Python ETL Project"]
  node_capstone["Data Engineering Capstone"]
  node_spark_project["Spark Projects"]
  node_career["Career Resources"]
  node_certificate(("Certificate Outcomes"))
end

node_learner(("Learner"))

node_learner -->|"studies"| node_python
node_learner -->|"studies"| node_sql
node_learner -->|"studies"| node_relational
node_learner -->|"studies"| node_nosql
node_learner -->|"studies"| node_linux
node_learner -->|"studies"| node_warehouse
node_learner -->|"studies"| node_etl
node_learner -->|"studies"| node_bigdata
node_learner -->|"studies"| node_spark_ml
node_learner -->|"studies"| node_bi
node_learner -->|"applies skills"| node_python_project
node_learner -->|"builds projects"| node_capstone
node_learner -->|"builds projects"| node_spark_project
node_learner -->|"uses resources"| node_career
node_learner -->|"earns"| node_certificate

click node_python "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Python%20for%20Data%20Science%20and%20AI"
click node_linux "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Hands-on%20Introduction%20to%20Linux%20Commands%20and%20Shell%20Scripting"
click node_sql "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Databases%20and%20SQL%20for%20Data%20Science%20with%20Python"
click node_relational "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Introduction%20to%20Relational%20Databases"
click node_nosql "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Introduction%20to%20NoSQL%20Databases"
click node_warehouse "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Warehouse%20Fundamentals"
click node_etl "https://github.com/dennislamcv1/ibmdataengineering/tree/main/ETL%20and%20Data%20Pipelines%20with%20Shell%2C%20Airflow%20and%20Kafka"
click node_bigdata "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Big%20Data%2C%20Hadoop%2C%20and%20Spark%20Basics"
click node_spark_ml "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Machine%20Learning%20with%20Apache%20Spark"
click node_bi "https://github.com/dennislamcv1/ibmdataengineering/tree/main/BI%20Dashboards%20with%20IBM%20Cognos%20Analytics%20and%20Google%20Looker"
click node_python_project "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Python%20Project%20for%20Data%20Engineering"
click node_capstone "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Engineering%20Capstone%20Project"
click node_spark_project "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Engineering%20and%20Machine%20Learning%20using%20Spark"
click node_career "https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Engineering%20Career%20Guide"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_python,node_linux toneBlue
class node_sql,node_relational,node_nosql,node_warehouse toneAmber
class node_etl,node_bigdata,node_spark_ml,node_bi toneMint
class node_python_project,node_capstone,node_spark_project,node_career,node_certificate toneRose
class node_learner toneIndigo
```

## Components

### Foundations

- [Python](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Python%20for%20Data%20Science%20and%20AI): `Python for Data Science and AI`
- [Linux and Shell](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Hands-on%20Introduction%20to%20Linux%20Commands%20and%20Shell%20Scripting): `Hands-on Introduction to Linux Commands and Shell Scripting`

### Data Platforms

- [SQL](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Databases%20and%20SQL%20for%20Data%20Science%20with%20Python): `Databases and SQL for Data Science with Python`
- [Relational Databases](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Introduction%20to%20Relational%20Databases): `Introduction to Relational Databases`
- [NoSQL Databases](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Introduction%20to%20NoSQL%20Databases): `Introduction to NoSQL Databases`
- [Data Warehousing](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Warehouse%20Fundamentals): `Data Warehouse Fundamentals`

### Engineering and Analytics

- [ETL Pipelines](https://github.com/dennislamcv1/ibmdataengineering/tree/main/ETL%20and%20Data%20Pipelines%20with%20Shell%2C%20Airflow%20and%20Kafka): `ETL and Data Pipelines with Shell, Airflow and Kafka`
- [Big Data and Spark](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Big%20Data%2C%20Hadoop%2C%20and%20Spark%20Basics): `Big Data, Hadoop, and Spark Basics`
- [Spark Machine Learning](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Machine%20Learning%20with%20Apache%20Spark): `Machine Learning with Apache Spark`
- [BI Dashboards](https://github.com/dennislamcv1/ibmdataengineering/tree/main/BI%20Dashboards%20with%20IBM%20Cognos%20Analytics%20and%20Google%20Looker): `BI Dashboards with IBM Cognos Analytics and Google Looker`

### Projects and Career

- [Python ETL Project](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Python%20Project%20for%20Data%20Engineering): `Python Project for Data Engineering`
- [Data Engineering Capstone](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Engineering%20Capstone%20Project): `Data Engineering Capstone Project`
- [Spark Projects](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Engineering%20and%20Machine%20Learning%20using%20Spark): `Data Engineering and Machine Learning using Spark`
- [Career Resources](https://github.com/dennislamcv1/ibmdataengineering/tree/main/Data%20Engineering%20Career%20Guide): `Data Engineering Career Guide`
- **Certificate Outcomes**

### Other components

- **Learner**

## Connections

- Learner → Python: studies (evidence: `README.md`)
- Learner → SQL: studies (evidence: `README.md`)
- Learner → Relational Databases: studies (evidence: `README.md`)
- Learner → NoSQL Databases: studies (evidence: `README.md`)
- Learner → Linux and Shell: studies (evidence: `README.md`)
- Learner → Data Warehousing: studies (evidence: `README.md`)
- Learner → ETL Pipelines: studies (evidence: `README.md`)
- Learner → Big Data and Spark: studies (evidence: `README.md`)
- Learner → Spark Machine Learning: studies (evidence: `README.md`)
- Learner → BI Dashboards: studies (evidence: `README.md`)
- Learner → Python ETL Project: applies skills (evidence: `README.md`)
- Learner → Data Engineering Capstone: builds projects (evidence: `README.md`)
- Learner → Spark Projects: builds projects (evidence: `README.md`)
- Learner → Career Resources: uses resources (evidence: `README.md`)
- Learner → Certificate Outcomes: earns (evidence: `README.md`)
