# huyhoang17/tensorflow-serving-docker architecture

> Architecture diagram of the GitHub repository huyhoang17/tensorflow-serving-docker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huyhoang17/tensorflow-serving-docker
- Repository: https://github.com/huyhoang17/tensorflow-serving-docker

## Overview

##### Project Type and Purpose
- The project is designed to containerize and deploy TensorFlow Serving for an MNIST model.
- Its purpose is to allow clients to send inference requests (with base64-encoded images) either via HTTP or gRPC, and the service responds with predictions.
- The README instructs users to build and run the Docker containers using docker-compose, while the file tree confirms the presence of relevant Docker configuration files and serving scripts.

##### File Structure
- There are several key files:
- Dockerfile and docker-compose.yml: Define the containerized environment.
- serve.py: Likely the entry point script that might wrap the TensorFlow Serving logic.
- models/ and frozen/: Contain the MNIST models in various formats.
- protos/: Contains protobuf definitions required for gRPC communications with TensorFlow Serving. This suggests a strong gRPC integration component.
- utils.py: Provides helper functions, likely including code for generating requests or processing inputs.
- The directories and files indicate that the system is composed of multiple layers: data/model serving, communication (HTTP/gRPC endpoints), and container orchestration.

##### README for Additional Insights
- The README specifies the use of Docker Compose commands to build and run the service.
- It clearly states:
- The accepted input is a base64-encoded image.
- The HTTP request endpoint: host:8501/v1/models/mnist-serving:predict.
- gRPC details are abstracted with pointers to the utilities in utils.py and the proto definitions.
- This confirms that the system is configured for dual-protocol communication with TensorFlow Serving.

a. Main Components:
- Client Interface: External clients sending HTTP or gRPC requests.
- Docker Compose Environment: Manages multiple container components.
- TensorFlow Serving Container: The core service container, possibly initiated by serve.py, which loads the MNIST models (from the models or frozen directories).
- Model Storage: The directories (models/frozen) that contain the actual model files.
- gRPC Interface: Represented by the set of protobuf definitions in the “protos” folder and related helper functions in utils.py.
- HTTP Interface: The exposed API endpoint for REST-like inference requests.

b. Relationships and Interactions:
- Clients send inference requests to the TensorFlow Serving service via HTTP or gRPC.
- Docker Compose orchestrates and connects these service containers.
- The TensorFlow Serving component loads models from the models and frozen folders.
- In case of gRPC requests, the predefined protobuf interfaces (stored in the protos directory) are used to encode/decode messages.

c. Key Architectural Patterns/Design Principles:
- Containerization: The entire system is containerized using Docker and orchestrated by Docker Compose.
- Service Separation: Clear distinction between serving the model and handling inputs, leveraging microservice principles where possible.
- API Layer Abstraction: Support for both HTTP and gRPC ensures flexibility for integration and efficient data handling.

d. Relevant Technologies/Frameworks:
- Docker & Docker Compose: For container orchestration.
- TensorFlow Serving: For model inference.
- gRPC & Protocol Buffers: For the gRPC service communication.
- Likely Python (given the .py files such as serve.py and utils.py).

##### Elements
- Introduce additional visual cues:
- Optionally, denote the Dockerfile, docker-compose configuration, and any configuration files (like serving.config and serving_docker.config) as supporting artifacts to the build/deployment process.

- The TensorFlow Serving process.
- Mounted volumes for models (pointing to models/ and frozen/).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Clients
    ClientHTTP["Client (HTTP)"]:::client
    ClientgRPC["Client (gRPC)"]:::client

    %% Docker Environment Subgraph
    subgraph "Docker Environment"
        DockerCompose["Docker Compose Environment"]:::orchestration
        TFContainer["TensorFlow Serving Container"]:::container
        DockerCompose -->|"Orchestrates"| TFContainer
    end

    %% Other Components
    ModelStorage["Model Storage (models/ & frozen/)"]:::storage
    gRPCInterface["gRPC Interface (protos/)"]:::container
    Utility["Utility Scripts (utils.py)"]:::utility

    %% Client to TensorFlow Serving connections
    ClientHTTP -->|"base64-image@HTTP:8501/v1/models/mnist-serving:predict"| TFContainer
    ClientgRPC -->|"gRPCRequest(usingProtobufs)"| TFContainer

    %% TensorFlow Serving interactions
    TFContainer -->|"LoadsModelsFrom"| ModelStorage
    TFContainer -->|"Utilizes"| gRPCInterface
    TFContainer -->|"MayCall"| Utility

    %% Click Events for Component Mapping
    click DockerCompose "https://github.com/huyhoang17/tensorflow-serving-docker/blob/master/docker-compose.yml"
    click DockerCompose "https://github.com/huyhoang17/tensorflow-serving-docker/tree/master/Dockerfile"
    click DockerCompose "https://github.com/huyhoang17/tensorflow-serving-docker/blob/master/serving.config"
    click DockerCompose "https://github.com/huyhoang17/tensorflow-serving-docker/blob/master/serving_docker.config"
    
    click TFContainer "https://github.com/huyhoang17/tensorflow-serving-docker/blob/master/serve.py"
    
    click ModelStorage "https://github.com/huyhoang17/tensorflow-serving-docker/tree/master/models/"
    click ModelStorage "https://github.com/huyhoang17/tensorflow-serving-docker/tree/master/frozen/"
    
    click gRPCInterface "https://github.com/huyhoang17/tensorflow-serving-docker/tree/master/protos/"
    
    click Utility "https://github.com/huyhoang17/tensorflow-serving-docker/blob/master/utils.py"

    %% Styles
    classDef container fill:#FFD700,stroke:#B8860B,stroke-width:2px;
    classDef orchestration fill:#FFB6C1,stroke:#FF69B4,stroke-width:2px;
    classDef storage fill:#DDA0DD,stroke:#800080,stroke-width:2px;
    classDef client fill:#E0FFFF,stroke:#008B8B,stroke-width:2px;
    classDef utility fill:#F5DEB3,stroke:#DEB887,stroke-width:2px;
```
