How iamlekh/foodlensllm works
Main Components
a. Web Interface Layer (Frontend) – The project uses Streamlit (handled in app.py) to create an interactive interface where users upload images and view output recipes.
b. Image Processing Module – Within the application logic (likely a function inside app.py), an image captioning process is executed using the pre-trained Hugging Face model (Salesforce/blip-image-captioning-large), extracting text descriptions from the images.
c. Recipe Generation Module – Another function (e.g., generate_recipe) processes the text description from the image using a language model. This could be interacting with OpenAI’s API to generate recipes. It formats the recipe output with instructions, nutritional breakdown, and additional details.
d. External API Integration – The system depends on external services:
• Hugging Face API for image captioning.
• OpenAI API for language processing and recipe generation.
e. Configuration & Environment – While not shown directly in the file tree, a .env file is indicated in the README (where API keys are stored), which is critical for security and configuration.
Relationships and Data Flow
• Users interact with the Streamlit web interface to upload an image.
• The upload triggers the image processing module, calling the Hugging Face model to generate an ingredients or caption text.
• The generated text is then passed to the recipe generation module, which communicates with OpenAI’s API to get a formatted recipe.
• The results (recipe, nutritional information, etc.) are then displayed back to the user through the Streamlit interface.
• Data flows from the user interface -> image processing (ML module) -> recipe generation (API integration) -> results back to UI.
Architectural Patterns and Design Principles
• Separation of concerns: UI handling (via Streamlit) is separate from the backend processing (machine learning and API integrations).
• Modularity: Individual functions for image captioning and recipe generation help in isolating concerns.
• External integration: Clear dependency on third-party APIs (Hugging Face and OpenAI) for core functionality.
Relevant Technologies and Dependencies
• Programming Language: Python.
• Web Framework: Streamlit.
• Machine Learning Models: Pre-trained Hugging Face image captioning model and OpenAI language model.
• Configuration management through environment variables (.env file).