How huyhoang17/tensorflow-serving-docker works
Project Type and Purpose
• The project is designed to containerize and deploy TensorFlow Serving for an MNIST model.
• Its purpose is to allow clients to send inference requests (with base64-encoded images) either via HTTP or gRPC, and the service responds with predictions.
• The README instructs users to build and run the Docker containers using docker-compose, while the file tree confirms the presence of relevant Docker configuration files and serving scripts.
File Structure
• There are several key files:
• Dockerfile and docker-compose.yml: Define the containerized environment.
• serve.py: Likely the entry point script that might wrap the TensorFlow Serving logic.
• models/ and frozen/: Contain the MNIST models in various formats.
• protos/: Contains protobuf definitions required for gRPC communications with TensorFlow Serving. This suggests a strong gRPC integration component.
• utils.py: Provides helper functions, likely including code for generating requests or processing inputs.
• The directories and files indicate that the system is composed of multiple layers: data/model serving, communication (HTTP/gRPC endpoints), and container orchestration.
README for Additional Insights
• The README specifies the use of Docker Compose commands to build and run the service.
• It clearly states:
• The accepted input is a base64-encoded image.
• The HTTP request endpoint: host:8501/v1/models/mnist-serving:predict.
• gRPC details are abstracted with pointers to the utilities in utils.py and the proto definitions.
• This confirms that the system is configured for dual-protocol communication with TensorFlow Serving.
a. Main Components:
• Client Interface: External clients sending HTTP or gRPC requests.
• Docker Compose Environment: Manages multiple container components.
• TensorFlow Serving Container: The core service container, possibly initiated by serve.py, which loads the MNIST models (from the models or frozen directories).
• Model Storage: The directories (models/frozen) that contain the actual model files.
• gRPC Interface: Represented by the set of protobuf definitions in the “protos” folder and related helper functions in utils.py.
• HTTP Interface: The exposed API endpoint for REST-like inference requests.
b. Relationships and Interactions:
• Clients send inference requests to the TensorFlow Serving service via HTTP or gRPC.
• Docker Compose orchestrates and connects these service containers.
• The TensorFlow Serving component loads models from the models and frozen folders.
• In case of gRPC requests, the predefined protobuf interfaces (stored in the protos directory) are used to encode/decode messages.
c. Key Architectural Patterns/Design Principles:
• Containerization: The entire system is containerized using Docker and orchestrated by Docker Compose.
• Service Separation: Clear distinction between serving the model and handling inputs, leveraging microservice principles where possible.
• API Layer Abstraction: Support for both HTTP and gRPC ensures flexibility for integration and efficient data handling.
d. Relevant Technologies/Frameworks:
• Docker & Docker Compose: For container orchestration.
• TensorFlow Serving: For model inference.
• gRPC & Protocol Buffers: For the gRPC service communication.
• Likely Python (given the .py files such as serve.py and utils.py).
Elements
• Introduce additional visual cues:
• Optionally, denote the Dockerfile, docker-compose configuration, and any configuration files (like serving.config and serving_docker.config) as supporting artifacts to the build/deployment process.
• The TensorFlow Serving process.
• Mounted volumes for models (pointing to models/ and frozen/).