How gitnasr/storyvault works
Project Identification & Overview
• Project Type: A standalone Python script designed to monitor, download, and archive Instagram stories.
• Purpose: Automate Instagram story collection by periodically querying Instagram’s API, processing the images (with duplicate detection), and storing metadata and media locally.
• Key Functionalities: Story discovery, image processing with hashing, scheduled execution, multi-threaded downloading, and local database management.
File Structure
• Top-Level Files:
• stories.py: The main application script that coordinates execution.
• README.md, LICENSE, .gitignore, etc.: Standard repository files.
• Directory “utils” with modules:
• api.py: Manages Instagram API connection details and endpoints.
• cookies.py: Handles authentication through stored cookies.
• database.py: Implements local storage (likely SQLite) strategies for caching story metadata.
• images.py: Contains functions for downloading images, performing hash-based duplicate detection, and organizing files.
• This well-organized separation of concerns indicates a modular design where the primary script (stories.py) orchestrates activities by calling specialized utility functions.
Extracting Insights from the README
• The README details the periodic task schedule (every 30 minutes), the phased process (discovery ➔ processing ➔ storage), and the organized output structure (images sorted into directories and a SQLite database for metadata).
• It also provides an algorithm overview outlining key stages: fetching stories, batching API requests, image processing, and local database management.
• The technology stack includes external packages like requests, schedule, termcolor, and colorama.
a. Main Components to Include:
• Main Script (stories.py): Acts as the control center executing the scheduled tasks.
• Instagram API Module (utils/api.py): Handles external API communication.
• Authentication Module (utils/cookies.py): Manages Instagram authentication using cookies.
• Database Module (utils/database.py): Manages the local SQLite database for persisting story metadata.
• Image Processing Module (utils/images.py): Manages image downloading, duplicate detection via hashing, and file storage.
• External Libraries & Scheduler: Including the schedule library for periodic execution and others (requests, termcolor, etc.) used throughout the flow.
• Local Storage: Represented as the SQLite database (stories.db) and the “images/” directory hierarchy.
b. Relationships & Interactions:
• stories.py initiates the process by using the schedule library to periodically trigger the story archiving routine.
• It invokes functions from utils/api.py to fetch new story data.
• utils/cookies.py provides authentication credentials necessary for API access.
• Once data is fetched, the script calls utils/database.py to check against the local cache (to prevent duplicates) and record metadata.
• For media content, utils/images.py handles downloading the images, computes image hashes for duplicate detection, and saves images in a structured directory (username/story_id_timestamp.jpg).
• The components interact sequentially forming a pipeline from API fetch ➔ processing ➔ storage.
c. Architectural Patterns & Design Principles:
• Separation of Concerns: Different modules handle specific tasks (API, cookies, database, images).
• Layered Architecture: The main script acts as a coordinating layer, with lower layers handling specialized logic.
• Modularity: Each utility module is a reusable component which can be independently tested or replaced.
• Scheduling & Concurrency: Utilizing the schedule library and multithreaded downloading for efficiency.
d. Relevant Technologies, Frameworks, and Libraries:
• Python (as the implementation language)
• requests (for HTTP API calls)
• schedule (for periodic execution)
• termcolor and colorama (for enhanced console logging)
• SQLite (for local metadata persistence)
Specific Elements
• Use clear, descriptive labels for each component (e.g., “Main Controller (stories.py)”, “API Manager (utils/api.py)”, “Authentication Handler (utils/cookies.py)”, “Database Manager (utils/database.py)”, “Image Processor (utils/images.py)”).
stories.py → utils/api.py (fetch stories)
stories.py → utils/cookies.py (authenticate session)
stories.py → utils/database.py (check and store metadata)
stories.py → utils/images.py (process and save images)
• Rectangles for software modules.
• Callouts for the schedule mechanism and multithreading aspects.