# kanewilliams/political_gang_discourse architecture

> Architecture diagram of the GitHub repository kanewilliams/political_gang_discourse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kanewilliams/political_gang_discourse
- Repository: https://github.com/kanewilliams/political_gang_discourse

## Overview

##### Main Components
- Data Ingestion:
- Corpus Files: The project includes a .7z archive (NZ Parliament Transcripts Gang Corpus.7z) containing the scraped transcripts, along with supporting text files (Corpus_Information.txt and webscrape-IO_sitemap.txt) and the essay PDF that provides background.
- Data Preprocessing:
- Scripts & Notebooks: The preprocess.ipynb notebook and process_txt.py script are responsible for parsing, cleaning, and structuring the raw text from the transcripts.
- Analysis Modules:
- Analysis Techniques: The ReadMe mentions various analysis approaches (keyness, concordance, collocation, and word frequency). Each of these would be represented as separate modules or stages in the analysis pipeline.
- Output & Presentation:

##### Relationships and Interactions
- The preprocessed data then flows into the analysis stage where different analytical processes are executed in parallel or sequentially:
- Keyness Analysis
- Concordance Analysis
- Collocation Analysis
- Word Frequency Analysis
- The outputs of these analyses are then consolidated to support the final conclusions and are presented in the final essay document along with supplementary visualizations.

##### Architectural Patterns & Design Principles
- Modular Design: Each major stage (data ingestion, pre-processing, analysis, and output) should be clearly separated into distinct modules.
- Extensibility: The design should allow for adding new analysis modules or modifying pre-processing steps if additional text features are needed in future iterations.
- Documentation: Include clear labels and annotations to reflect the underlying research methodology.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Subgraph
    subgraph "Data Ingestion"
        A1["NZ Parliament Transcripts Gang Corpus.7z"]:::input
        A2["Corpus_Information.txt"]:::input
        A3["webscrape-IO_sitemap.txt"]:::input
    end

    %% Data Preprocessing Subgraph
    subgraph "Data Preprocessing"
        B1["Notebook: preprocess.ipynb"]:::process
        B2["Script: process_txt.py"]:::process
    end

    %% Aggregated Preprocessed Data Node
    B3["Preprocessed Data"]:::process

    %% Analysis Modules Subgraph
    subgraph "Analysis Modules"
        C1["Keyness Analysis"]:::analysis
        C2["Concordance Analysis"]:::analysis
        C3["Collocation Analysis"]:::analysis
        C4["Word Frequency Analysis"]:::analysis
        C5["Analysis Documentation (README.md)"]:::analysis
    end

    %% Output & Presentation Subgraph
    subgraph "Output & Presentation"
        D1["Final Essay: Corpus-Analysis-Essay_NZ-Parliament-and-Gangs.pdf"]:::output
        D2["Visualizations & Tables"]:::output
    end

    %% Data Flow Connections
    A1 -->|"ingests"| B1
    A2 -->|"ingests"| B1
    A3 -->|"ingests"| B1
    B1 -->|"explores"| B2
    B2 -->|"processes"| B3
    B3 -->|"feeds"| C1
    C1 -->|"triggers"| C2
    C2 -->|"triggers"| C3
    C3 -->|"triggers"| C4
    B3 -->|"reference"| C5
    C4 -->|"aggregates"| D1
    C5 -->|"supports"| D1
    D1 -->|"complements"| D2

    %% Click Events
    click A1 "https://github.com/kanewilliams/political_gang_discourse/blob/main/NZ Parliament Transcripts Gang Corpus.7z"
    click A2 "https://github.com/kanewilliams/political_gang_discourse/blob/main/Corpus_Information.txt"
    click A3 "https://github.com/kanewilliams/political_gang_discourse/blob/main/webscrape-IO_sitemap.txt"
    click B1 "https://github.com/kanewilliams/political_gang_discourse/blob/main/preprocess.ipynb"
    click B2 "https://github.com/kanewilliams/political_gang_discourse/blob/main/process_txt.py"
    click C5 "https://github.com/kanewilliams/political_gang_discourse/blob/main/README.md"
    click D1 "https://github.com/kanewilliams/political_gang_discourse/blob/main/Corpus-Analysis-Essay_NZ-Parliament-and-Gangs.pdf"

    %% Styles
    classDef input fill:#FFCCCC,stroke:#990000,stroke-width:2px;
    classDef process fill:#CCE5FF,stroke:#003366,stroke-width:2px;
    classDef analysis fill:#CCFFCC,stroke:#006600,stroke-width:2px;
    classDef output fill:#FFE5CC,stroke:#CC6600,stroke-width:2px;
```
