# elifesciences/search architecture

> Architecture diagram of the GitHub repository elifesciences/search, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/elifesciences/search
- Repository: https://github.com/elifesciences/search

## Overview

##### Main Components

a) Core Components:
- API Layer (SearchController)
- Elasticsearch/OpenSearch Service
- Queue System (SQS)
- Indexer System
- Content API (external system)

b) Data Processing Components:
- Model Indexers (for different content types)
- Query Builder
- Response Handlers
- Key-Value Store

##### Architecture Pattern
The system follows a service-oriented architecture with clear separation of concerns:

**[External Content API] → [Queue System (SQS)] → [Indexer Service] → [Elasticsearch/OpenSearch]**
- ↓
**[HTTP Clients] → [Search API Layer] → [Query Builder] → [Elasticsearch/OpenSearch]**

##### Detailed Component Breakdown

a) Search API Layer:

- Search API        |

- SearchController    |
- ApiValidator        |
- ContentType         |

b) Elasticsearch Integration:

- Elasticsearch Handlers  |

- Query Builder        |
- Response Serializer  |
- Mapping Management   |

c) Indexing System:

- Indexing System     |

- Model Indexers       |
- Change Set Handler   |
- Queue Processor      |

##### Content Type Handlers
- Research Articles
- Blog Articles
- Collections
- Interviews
- Labs Posts
- Podcast Episodes
- Reviewed Preprints

- → Synchronous calls
- ⇢ Asynchronous operations
- ⇄ Two-way communication

##### Environment Containers
- Docker containers
- External services

##### Additional Elements

Annotations for:
- Configuration management
- Logging and monitoring points
- Test coverage areas
- Cache layers

##### Index management flow

This architecture should emphasize the system's ability to handle different content types, its scalability through queuing, and its separation of concerns between indexing and searching operations.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph External
        ContentAPI[Content API]
    end

    subgraph "API Layer"
        SearchController["Search Controller"]:::apiLayer
        ApiValidator["API Validator"]:::apiLayer
        ContentType["Content Type Handler"]:::apiLayer
    end

    subgraph "Elasticsearch Integration"
        QueryBuilder["Query Builder"]:::elastic
        ResponseHandler["Response Handler"]:::elastic
        MappingMgmt["Mapping Management"]:::elastic
    end

    subgraph "Queue System"
        SQS["SQS Queue"]:::queue
        QueueCommands["Queue Commands"]:::queue
        ImportCommand["Import Command"]:::queue
        QueueWatch["Queue Watch"]:::queue
    end

    subgraph "Indexing System"
        Indexer["Main Indexer"]:::indexer
        ChangeSetHandler["Change Set Handler"]:::indexer
    end

    subgraph "Key-Value Store"
        KVStore["Key-Value Store Interface"]:::kvstore
        ESKVStore["Elasticsearch KV Store"]:::kvstore
    end

    subgraph "Model Indexers"
        direction TB
        ResearchArticle["Research Article Indexer"]:::modelIndexer
        BlogArticle["Blog Article Indexer"]:::modelIndexer
        Collection["Collection Indexer"]:::modelIndexer
        Interview["Interview Indexer"]:::modelIndexer
        LabsPost["Labs Post Indexer"]:::modelIndexer
        PodcastEpisode["Podcast Episode Indexer"]:::modelIndexer
        ReviewedPreprint["Reviewed Preprint Indexer"]:::modelIndexer
    end

    subgraph "Storage"
        Elasticsearch[(Elasticsearch/OpenSearch)]:::storage
    end

    %% Relationships
    ContentAPI --> SQS
    SQS --> QueueWatch
    QueueWatch --> ImportCommand
    ImportCommand --> Indexer
    Indexer --> ChangeSetHandler
    ChangeSetHandler --> Elasticsearch

    SearchController --> ApiValidator
    SearchController --> ContentType
    SearchController --> QueryBuilder
    QueryBuilder --> Elasticsearch
    Elasticsearch --> ResponseHandler
    ResponseHandler --> SearchController

    KVStore --> ESKVStore
    ESKVStore --> Elasticsearch

    Indexer --> ResearchArticle & BlogArticle & Collection & Interview & LabsPost & PodcastEpisode & ReviewedPreprint
    
    %% Styles
    classDef apiLayer fill:#2374ab
    classDef elastic fill:#048ba8
    classDef queue fill:#2a9d8f
    classDef indexer fill:#e9c46a
    classDef kvstore fill:#f4a261
    classDef modelIndexer fill:#e76f51
    classDef storage fill:#264653
    
    %% Click Events
    click SearchController "https://github.com/elifesciences/search/blob/master/src/Search/Api/SearchController.php"
    click ApiValidator "https://github.com/elifesciences/search/blob/master/src/Search/Api/ApiValidator.php"
    click ContentType "https://github.com/elifesciences/search/blob/master/src/Search/Api/ContentType.php"
    click QueryBuilder "https://github.com/elifesciences/search/blob/master/src/Search/Api/Elasticsearch/ElasticQueryBuilder.php"
    click ResponseHandler "https://github.com/elifesciences/search/tree/master/src/Search/Api/Elasticsearch/Response/"
    click MappingMgmt "https://github.com/elifesciences/search/blob/master/src/Search/Api/Elasticsearch/Command/resources/mappings.yaml"
    click Indexer "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/Indexer.php"
    click ChangeSetHandler "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ChangeSet.php"
    click QueueCommands "https://github.com/elifesciences/search/tree/master/src/Search/Queue/Command/"
    click ImportCommand "https://github.com/elifesciences/search/blob/master/src/Search/Queue/Command/ImportCommand.php"
    click QueueWatch "https://github.com/elifesciences/search/blob/master/src/Search/Queue/Command/QueueWatchCommand.php"
    click KVStore "https://github.com/elifesciences/search/blob/master/src/Search/KeyValueStore/KeyValueStore.php"
    click ESKVStore "https://github.com/elifesciences/search/blob/master/src/Search/KeyValueStore/ElasticsearchKeyValueStore.php"
    click ResearchArticle "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/ResearchArticleIndexer.php"
    click BlogArticle "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/BlogArticleIndexer.php"
    click Collection "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/CollectionIndexer.php"
    click Interview "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/InterviewIndexer.php"
    click LabsPost "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/LabsPostIndexer.php"
    click PodcastEpisode "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/PodcastEpisodeIndexer.php"
    click ReviewedPreprint "https://github.com/elifesciences/search/blob/master/src/Search/Indexer/ModelIndexer/ReviewedPreprintIndexer.php"
```
