# mahmoudessam820/pulsar_estate architecture

> Architecture diagram of the GitHub repository mahmoudessam820/pulsar_estate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mahmoudessam820/pulsar_estate
- Repository: https://github.com/mahmoudessam820/pulsar_estate

## Overview

Pulsar Estate is a UAE real-estate intelligence backend: users request market insights, which are searched, crawled, analyzed by a configured AI provider, trust-scored, and stored. The README describes an asynchronous Celery/RabbitMQ worker path, semantic-cache fast path, usage/quota enforcement, and completion events; sampled code also shows a direct API pipeline route and provider assembly. Other documented product branches include authentication, billing, integrations, and scheduled queries. Some routes and worker wiring were not sampled, so their connections are shown only where the README or source index supports them.

Main entry points: [`app/core/pipeline/pipeline_service.py`](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/pipeline_service.py) (Pipeline Service), [`app/core/pipeline/factory.py`](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/factory.py) (Pipeline Factory), [`app/api/routes/pipeline.py`](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/pipeline.py) (Pipeline API), [`app/core/pipeline/tasks.py`](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/tasks.py) (Celery Workers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and Access"]
  node_api["FastAPI Routes"]
  node_auth["Auth Services"]
  node_integration_api["Integrations API<br/>[integrations.py]"]
  node_pipeline_api["Pipeline API<br/>[pipeline.py]"]
  node_insights_api["Insights API<br/>[insights.py]"]
end

subgraph group_pipeline["Insight Pipeline"]
  node_pipeline_factory["Pipeline Factory<br/>[factory.py]"]
  node_pipeline_service["Pipeline Service"]
  node_search["Search Provider"]
  node_crawler["Crawler Provider<br/>[crawl4ai.py]"]
  node_ai["AI Providers"]
  node_embedding["Embedding Providers"]
  node_worker["Celery Workers<br/>[tasks.py]"]
  node_queue["RabbitMQ Queue"]
end

subgraph group_trust["Trust and Storage"]
  node_trust["Trust Engine"]
  node_insight_repo["Insight Repository<br/>[insight_repo.py]"]
  node_data[("PostgreSQL / pgvector")]
  node_cache["Semantic Cache"]
end

subgraph group_operations["Usage and Scheduling"]
  node_events["Events / Notifications"]
  node_entitlements["Entitlements and Usage<br/>[entitlements.py]"]
  node_scheduler["Scheduled Queries"]
  node_scheduler_impl["APScheduler"]
end

subgraph group_account["Accounts and Billing"]
  node_billing["Subscriptions and Payments<br/>[subscriptions.py]"]
  node_stripe["Stripe<br/>[stripe.py]"]
end

node_user(("User / Frontend"))

node_user -->|"requests"| node_api
node_api -->|"authenticates"| node_auth
node_user -->|"submits query"| node_pipeline_api
node_pipeline_api -->|"checks access"| node_entitlements
node_pipeline_api -->|"builds pipeline"| node_pipeline_factory
node_pipeline_factory -->|"assembles"| node_pipeline_service
node_pipeline_factory -->|"configures"| node_search
node_pipeline_factory -->|"configures"| node_crawler
node_pipeline_factory -->|"selects provider"| node_ai
node_pipeline_factory -->|"configures"| node_insight_repo
node_pipeline_service -->|"searches"| node_search
node_pipeline_service -->|"crawls results"| node_crawler
node_pipeline_service -->|"analyzes documents"| node_ai
node_pipeline_service -->|"scores insight"| node_trust
node_pipeline_service -->|"saves insight"| node_insight_repo
node_insight_repo -->|"persists insight"| node_data
node_insights_api -.->|"checks cache"| node_cache
node_cache -.->|"returns cached insight"| node_user
node_pipeline_api -.->|"dispatches task"| node_queue
node_queue -.->|"delivers task"| node_worker
node_worker -->|"runs pipeline"| node_pipeline_factory
node_worker -->|"rechecks quota"| node_entitlements
node_worker -->|"publishes completion"| node_events
node_pipeline_service -.->|"creates embeddings"| node_embedding
node_user -.->|"schedules query"| node_scheduler
node_scheduler -.->|"runs recurring queries"| node_scheduler_impl
node_scheduler -->|"stores schedule"| node_data
node_user -.->|"manages subscription"| node_billing
node_billing -->|"uses gateway"| node_stripe
node_integration_api -.->|"configures integrations"| node_ai

click node_api "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/api/routes"
click node_auth "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/auth"
click node_integration_api "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/integrations.py"
click node_pipeline_api "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/pipeline.py"
click node_insights_api "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/insights.py"
click node_pipeline_factory "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/factory.py"
click node_pipeline_service "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/pipeline_service.py"
click node_search "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/providers/search"
click node_crawler "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/providers/crawler/crawl4ai.py"
click node_ai "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/providers/ai"
click node_embedding "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/providers/embedding"
click node_trust "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/trust"
click node_insight_repo "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/data/repositories/insight_repo.py"
click node_data "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/data"
click node_cache "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/data"
click node_worker "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/tasks.py"
click node_events "https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/events"
click node_entitlements "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/monetization/entitlements.py"
click node_scheduler "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/scheduler/scheduled_query_service.py"
click node_scheduler_impl "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/scheduler/apscheduler_impl.py"
click node_billing "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/billing/subscriptions.py"
click node_stripe "https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/payments/stripe.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_auth,node_integration_api,node_pipeline_api,node_insights_api,node_user toneBlue
class node_pipeline_factory,node_pipeline_service,node_search,node_crawler,node_ai,node_embedding,node_worker,node_queue toneAmber
class node_trust,node_insight_repo,node_data,node_cache toneMint
class node_events,node_entitlements,node_scheduler,node_scheduler_impl toneRose
class node_billing,node_stripe toneIndigo
```

## Components

### API and Access

- [FastAPI Routes](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/api/routes): `app/api/routes`
- [Auth Services](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/auth): `app/auth`
- [Integrations API](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/integrations.py): `app/api/routes/integrations.py`
- [Pipeline API](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/pipeline.py): `app/api/routes/pipeline.py`
- [Insights API](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/api/routes/insights.py): `app/api/routes/insights.py`

### Insight Pipeline

- [Pipeline Factory](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/factory.py): `app/core/pipeline/factory.py`
- [Pipeline Service](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/pipeline_service.py): `app/core/pipeline/pipeline_service.py`
- [Search Provider](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/providers/search): `app/providers/search`
- [Crawler Provider](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/providers/crawler/crawl4ai.py): `app/providers/crawler/crawl4ai.py`
- [AI Providers](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/providers/ai): `app/providers/ai`
- [Embedding Providers](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/providers/embedding): `app/providers/embedding`
- [Celery Workers](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/pipeline/tasks.py): `app/core/pipeline/tasks.py`
- **RabbitMQ Queue**

### Trust and Storage

- [Trust Engine](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/trust): `app/trust`
- [Insight Repository](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/data/repositories/insight_repo.py): `app/data/repositories/insight_repo.py`
- [PostgreSQL / pgvector](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/data): `app/data`
- [Semantic Cache](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/data): `app/data`

### Usage and Scheduling

- [Events / Notifications](https://github.com/mahmoudessam820/pulsar_estate/tree/main/app/events): `app/events`
- [Entitlements and Usage](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/monetization/entitlements.py): `app/monetization/entitlements.py`
- [Scheduled Queries](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/core/scheduler/scheduled_query_service.py): `app/core/scheduler/scheduled_query_service.py`
- [APScheduler](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/scheduler/apscheduler_impl.py): `app/scheduler/apscheduler_impl.py`

### Accounts and Billing

- [Subscriptions and Payments](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/billing/subscriptions.py): `app/billing/subscriptions.py`
- [Stripe](https://github.com/mahmoudessam820/pulsar_estate/blob/main/app/payments/stripe.py): `app/payments/stripe.py`

### Other components

- **User / Frontend**

## Connections

- User / Frontend → FastAPI Routes: requests
- FastAPI Routes → Auth Services: authenticates (evidence: `app/api/routes/auth.py`)
- User / Frontend → Pipeline API: submits query
- Pipeline API → Entitlements and Usage: checks access (evidence: `app/api/routes/pipeline.py`)
- Pipeline API → Pipeline Factory: builds pipeline (evidence: `app/api/routes/pipeline.py`)
- Pipeline Factory → Pipeline Service: assembles (evidence: `app/core/pipeline/factory.py`)
- Pipeline Factory → Search Provider: configures (evidence: `app/core/pipeline/factory.py`)
- Pipeline Factory → Crawler Provider: configures (evidence: `app/core/pipeline/factory.py`)
- Pipeline Factory → AI Providers: selects provider (evidence: `app/core/pipeline/factory.py`)
- Pipeline Factory → Insight Repository: configures (evidence: `app/core/pipeline/factory.py`)
- Pipeline Service → Search Provider: searches (evidence: `app/core/pipeline/pipeline_service.py`)
- Pipeline Service → Crawler Provider: crawls results (evidence: `app/core/pipeline/pipeline_service.py`)
- Pipeline Service → AI Providers: analyzes documents (evidence: `app/core/pipeline/pipeline_service.py`)
- Pipeline Service → Trust Engine: scores insight (evidence: `app/core/pipeline/pipeline_service.py`)
- Pipeline Service → Insight Repository: saves insight (evidence: `app/core/pipeline/pipeline_service.py`)
- Insight Repository → PostgreSQL / pgvector: persists insight (evidence: `app/core/pipeline/factory.py`)
- Insights API → Semantic Cache: checks cache
- Semantic Cache → User / Frontend: returns cached insight
- Pipeline API → RabbitMQ Queue: dispatches task
- RabbitMQ Queue → Celery Workers: delivers task
- Celery Workers → Pipeline Factory: runs pipeline (evidence: `app/core/pipeline/tasks.py`)
- Celery Workers → Entitlements and Usage: rechecks quota (evidence: `app/core/pipeline/tasks.py`)
- Celery Workers → Events / Notifications: publishes completion (evidence: `app/core/pipeline/tasks.py`)
- Pipeline Service → Embedding Providers: creates embeddings
- User / Frontend → Scheduled Queries: schedules query
- Scheduled Queries → APScheduler: runs recurring queries
- Scheduled Queries → PostgreSQL / pgvector: stores schedule (evidence: `app/core/scheduler/scheduled_query_service.py`)
- User / Frontend → Subscriptions and Payments: manages subscription
- Subscriptions and Payments → Stripe: uses gateway (evidence: `app/billing/subscriptions.py`)
- Integrations API → AI Providers: configures integrations
