# rohanpatel981/consolidation-pipeline-design architecture

> Architecture diagram of the GitHub repository rohanpatel981/consolidation-pipeline-design, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rohanpatel981/consolidation-pipeline-design
- Repository: https://github.com/rohanpatel981/consolidation-pipeline-design

## Overview

This repository is an architecture write-up for a multi-tenant financial consolidation system, not the system’s source code. The documented user workflow is direct browser upload to S3, followed by asynchronous file processing, consolidation, and status or business operations through the platform service. The pipeline stages and major accounting capabilities are represented from the README; detailed implementation calls cannot be verified from source excerpts.

Main entry points: [`docs/architecture.md`](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md) (File processing), [`docs/pipeline.md`](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md) (Final consolidation), [`docs/adr/0001-sax-streaming-over-dom.md`](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/adr/0001-sax-streaming-over-dom.md) (Streaming extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Client"]
  node_browser(("Browser UI"))
end

subgraph group_services["Backend services"]
  node_core["Shared core<br/>[architecture.md]"]
  node_platform["Platform service<br/>[architecture.md]"]
  node_fileproc["File processing<br/>[architecture.md]"]
end

subgraph group_pipeline["Consolidation pipeline"]
  node_upload["Trial-balance upload<br/>[scaling.md]"]
  node_extract["Streaming extraction"]
  node_precon["Pre-consolidation data<br/>[pipeline.md]"]
  node_modulework["Nine-module processing<br/>[pipeline.md]"]
  node_retained["Retained earnings<br/>[pipeline.md]"]
  node_fctr["Currency translation<br/>[pipeline.md]"]
  node_final["Final consolidation<br/>[pipeline.md]"]
  node_accounting["Accounting operations<br/>[architecture.md]"]
end

subgraph group_infrastructure["Infrastructure"]
  node_s3[("AWS S3")]
  node_mongo[("MongoDB")]
  node_mysql[("MySQL")]
  node_redis[("Redis")]
  node_rabbit["RabbitMQ"]
end

node_browser -->|"uploads directly"| node_s3
node_browser -->|"uses REST"| node_platform
node_platform -.->|"depends on"| node_core
node_fileproc -.->|"depends on"| node_core
node_platform -->|"dispatches via RabbitMQ"| node_fileproc
node_fileproc -->|"reads files"| node_s3
node_fileproc -->|"streams rows"| node_extract
node_extract -->|"produces data"| node_precon
node_precon -->|"feeds processing"| node_modulework
node_modulework -->|"increments counters"| node_redis
node_redis -->|"coordinates watcher"| node_fileproc
node_fileproc -->|"starts after coordination"| node_retained
node_fileproc -->|"starts after coordination"| node_fctr
node_retained -->|"contributes to"| node_final
node_fctr -->|"contributes to"| node_final
node_platform -->|"reads and writes"| node_mongo
node_platform -->|"reads and writes"| node_mysql
node_platform -->|"uses counters and locks"| node_redis
node_platform -->|"publishes and consumes"| node_rabbit
node_fileproc -->|"reads and writes"| node_mongo
node_fileproc -->|"publishes and consumes"| node_rabbit
node_final -->|"stores consolidation data"| node_mongo
node_accounting -->|"managed through"| node_platform

click node_core "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md"
click node_platform "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md"
click node_fileproc "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md"
click node_upload "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/scaling.md"
click node_extract "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/adr/0001-sax-streaming-over-dom.md"
click node_precon "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md"
click node_modulework "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md"
click node_retained "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md"
click node_fctr "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md"
click node_final "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md"
click node_accounting "https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser toneBlue
class node_core,node_platform,node_fileproc toneAmber
class node_upload,node_extract,node_precon,node_modulework,node_retained,node_fctr,node_final,node_accounting toneMint
class node_s3,node_mongo,node_mysql,node_redis,node_rabbit toneRose
```

## Components

### Client

- **Browser UI**

### Backend services

- [Shared core](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md): `docs/architecture.md`
- [Platform service](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md): `docs/architecture.md`
- [File processing](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md): `docs/architecture.md`

### Consolidation pipeline

- [Trial-balance upload](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/scaling.md): `docs/scaling.md`
- [Streaming extraction](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/adr/0001-sax-streaming-over-dom.md): `docs/adr/0001-sax-streaming-over-dom.md`
- [Pre-consolidation data](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md): `docs/pipeline.md`
- [Nine-module processing](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md): `docs/pipeline.md`
- [Retained earnings](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md): `docs/pipeline.md`
- [Currency translation](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md): `docs/pipeline.md`
- [Final consolidation](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/pipeline.md): `docs/pipeline.md`
- [Accounting operations](https://github.com/rohanpatel981/consolidation-pipeline-design/blob/main/docs/architecture.md): `docs/architecture.md`

### Infrastructure

- **AWS S3**
- **MongoDB**
- **MySQL**
- **Redis**
- **RabbitMQ**

## Connections

- Browser UI → AWS S3: uploads directly
- Browser UI → Platform service: uses REST
- Platform service → Shared core: depends on
- File processing → Shared core: depends on
- Platform service → File processing: dispatches via RabbitMQ
- File processing → AWS S3: reads files
- File processing → Streaming extraction: streams rows
- Streaming extraction → Pre-consolidation data: produces data
- Pre-consolidation data → Nine-module processing: feeds processing
- Nine-module processing → Redis: increments counters
- Redis → File processing: coordinates watcher
- File processing → Retained earnings: starts after coordination
- File processing → Currency translation: starts after coordination
- Retained earnings → Final consolidation: contributes to
- Currency translation → Final consolidation: contributes to
- Platform service → MongoDB: reads and writes
- Platform service → MySQL: reads and writes
- Platform service → Redis: uses counters and locks
- Platform service → RabbitMQ: publishes and consumes
- File processing → MongoDB: reads and writes
- File processing → RabbitMQ: publishes and consumes
- Final consolidation → MongoDB: stores consolidation data
- Accounting operations → Platform service: managed through