How ansh-sarkar/mftp works
<explanation>
1. Project Type & Purpose
- This is a backend monitoring‐and‐notification tool, not a web UI or full‐stack app.
- It consists of two standalone services (“microservices” in Docker containers):
• mftp: polls the CDC noticeboard and forwards new notices (via SMTP, Gmail API, or NTFY)
• mftp‑doctor: periodically health‐checks mftp and its logs, sending alerts on failure
- Both services can be run under cron or as systemd services, and are configured via environment variables and external config directories.
2. File Structure Analysis
- Top‐level docker‑compose.yml ties together two services: mftp and mftp‑doctor, each with its own Dockerfile and requirements.txt under mftp/ and mftp‑doctor/.
- service/ contains installation scripts and systemd unit files for running as a native service.
- mftp/ directory holds the core Python modules:
• endpoints.py, notice.py, company.py, shortlist.py – parsing CDC notices
• db.py – persisting state (e.g., which notices have been sent)
• mail.py, ntfy.py – transport implementations
• mftp.py – orchestrator CLI entry point
- mftp‑doctor/ folder contains a simpler script (mftp‑doctor.py) and config to monitor mftp’s health.
3. Key Components for the Diagram
a. Scheduler Layer
• Cron daemon or systemd timer → triggers container runs (or keeps them always‑on)
b. mftp Service (Container)
• CDC Noticeboard (external HTTP) → endpoints.py → parsing modules → db.py → mail/ntfy modules → Email/NTFY services (SMTP/Gmail API/NTFY HTTP)
c. mftp‑doctor Service (Container)
• Monitors mftp service health & log files → on failures uses same mail/ntfy modules → alerts administrator
d. Configuration Volumes
• Two config directories mounted into each container: one for mftp, one for doctor, supplying env vars and credentials
e. External Services
• CDC website (HTTP)
• Email provider (SMTP or Gmail API)
• NTFY server (HTTP push notifications)
4. Interactions & Data Flows
- Scheduler ⟶ mftp container (via Docker Compose or systemd)
- mftp container ⟶ CDC Noticeboard HTTP GET
- mftp container ⟶ internal parsing & DB write
- mftp container ⟶ Email/NTFY service (SMTP, Gmail REST, or NTFY HTTP)
- mftp container ⟶ log file
- mftp‑doctor container ⟶ reads mftp logs & monitors container status
- mftp‑doctor container ⟶ Email/NTFY service for error alerts
5. Architectural Patterns & Principles
- Microservices (two discrete services)
- Twelve‑factor app: config via environment variables, logs to stdout/files, stateless containers
- Pluggable transports: mail.py and ntfy.py implement a strategy pattern for notification channels
- Scheduler abstraction: supports cron or systemd timer
6. Diagramming Guidelines
- Use clear shapes:
• Cylinder/icon for external services (CDC, SMTP, NTFY)
• Container boxes for mftp and mftp‑doctor
• Gear icon for Scheduler (cron/systemd)
• Folder icon for Config volumes
- Color coding:
• Scheduler – green
• Containers – blue
• Config – gray
• External – orange
- Arrows with labels:
• “HTTP GET” from mftp to CDC
• “SMTP/Gmail API/NTFY” from both containers to mail service
• “Reads logs” from doctor to mftp logs
• “Triggers” from Scheduler to containers
- Show two-way arrows where state/config is shared via mounted volumes.
7. Tailoring for This Project
- Emphasize the two separate Docker services and how they share config but have distinct responsibilities.
- Highlight the pluggable notification strategy.
- Include both deployment options: Docker Compose versus systemd/cron, perhaps as two parallel scheduler icons pointing to the same containers.
8. Essential Elements Checklist
[ ] Labeled components (Scheduler, mftp, mftp‑doctor, Config, CDC, Email/NTFY)
[ ] Directional arrows showing data/control flow
[ ] Color or shape differentiation
[ ] Protocols and transport methods on the arrows
[ ] Clear separation of concerns (polling vs. health-check)
By following these guidelines and mapping each code folder or script to a diagram component, you will produce a detailed, accurate system architecture diagram that reflects the real structure and runtime behavior of the MFTP project.
</explanation>