How agnosto/fansly-scraper works
Main Components
a. CLI/TUI Interface (User Interaction Layer)
• Represented by the “cmd/fansly-scraper” entry point and the “ui” package.
• The TUI provides interactive and non-interactive modes with commands for scraping, download actions, and live monitoring.
b. Core Business Logic
• The “core” package contains the main functionality like managing accounts, applying filters, handling livestreams, and stream logic.
• This is essentially the engine that coordinates between user inputs and the actual scraping functionalities.
c. Download Manager
• The “download” package focuses on handling specific downloading of media (m3u8 streams, messages, purchases, stories, and timeline).
• It is designed to work in concert with the core functionality to fetch and save content.
d. Content & Post Processing
• The “posts” package deals with the manipulation and organization of individual posts, albums, messages, and stories.
• The “interactions” package handles timeline-specific interactions.
e. Configuration and Authentication
• The “config” directory and “config.md” file contain the setup for customizing the application behavior.
• The “auth” package deals with any authentication or authorization needs.
f. Service & Monitoring
• The “service” package implements live monitoring functionalities for creator streams.
• There are separate implementations for Unix and Windows (e.g., monitor_unix.go and monitor_windows.go).
g. Supporting and Utility Modules
• “logger” for logging.
• “updater” for updating the binary.
• “utils” for common helper functions such as hashing and web-related utilities.
• “headers” could be used to manage network headers or API request structures.
Mapping the Relationships and Flow
• Start at the Entry Point:
• The application is initiated via the “cmd/fansly-scraper/main.go”.
• Depending on user-provided CLI flags (e.g., username, download options, live monitor), control flows towards either interactive UI or non-interactive commands.
• User Interface Layer:
• Input from the user is processed by components in the “ui” package, which visually convey progress (e.g., download progress, monitoring updates) and invoke the core functionalities.
• Core Logic and API Calls:
• User commands trigger functions in the “core” package that coordinate tasks like filtering content or managing livestream sessions.
• The core logic communicates with the “interactions” package if timeline or API-specific tasks are required.
• Download and Post Handling:
• They also interact with the “core” for any necessary data transformations or filtering before saving the data.
• Service Components:
• The “service” package manages ongoing tasks such as live monitoring. It integrates with the UI to display real-time information.
• Operating system-specific implementations ensure the monitoring service works on both Unix and Windows systems.
• Support and Utility Functions:
• Configuration is read from the “config” package (likely coordinated with example-config.toml and config.md instructions).
• The “auth” module ensures that proper authentication is in place when accessing Fansly content.
• Logging through the “logger” and helper functions from “utils” are used throughout.
• The “updater” enables the application to check and update its binary.
Architectural Patterns and Design Principles
• Modular Design:
• The project is highly modular, with each package handling a specific responsibility (e.g., UI, core logic, downloads, configuration). This separation promotes clarity and ease of maintenance.
• Layered Architecture:
• There is a clear separation between the presentation layer (CLI/TUI), business logic (core, posts, download), and platform/service layer (service, updater).
• Configuration-Driven Behavior:
• The application behavior is controlled via configuration files, with an “example-config.toml” and a dedicated “config” package.
• Cross-Platform Considerations:
• Separate service monitoring logic for Unix and Windows indicates thoughtful cross-platform design.
Final Recommendations
• Every component is labelled (include package names such as “core”, “download”, “ui”, etc.).
• Integration points with external services/tools (ffmpeg, mt) and operating system-specific components in the Service module.
• Configuration and environmental factors that impact behavior (reference to example-config.toml and actual runtime config initialization).