How yesbhautik/public-api-scraper works
Frontend
• Web Interface: templates/index.html
• User Interaction: Input fields for model, endpoint, and search keywords; display of search progress and results.
Backend
• Main Application: main.go
• GitHub Search Module: internal/github/search.go
• API Key Verifier: internal/verifier/api_verifier.go
Real-Time Communication
• WebSocket Server: Facilitates live updates between frontend and backend.
Database (if any)
• Current file tree does not indicate a database; data may be held in-memory or logs.
Docker Services
• Web Service Container: Runs the Go application.
• Docker Compose: Manages multi-container deployment if extended.
CI/CD Pipelines
• GitHub Actions: Defined in .github/workflows/ for testing and Docker image publishing.
b. Determine Relationships and Interactions:
• User ↔ Web Interface: Users interact with the frontend to initiate searches.
• Web Interface ↔ WebSocket Server: Facilitates real-time communication for search progress and results.
• WebSocket Server ↔ Backend Modules: Backend processes handle GitHub searches and API verification, sending updates back to the frontend via WebSockets.
• Backend Modules ↔ GitHub API: Backend interacts with GitHub using personal access tokens to search repositories.
• Docker Services ↔ Host System: Docker containers deploy and run the application in various environments.
c. Highlight Architectural Patterns or Design Principles:
• Layered Architecture: Separation between presentation (frontend), business logic (backend modules), and communication (WebSockets).
• Modular Design: Distinct internal packages (github, verifier) promote maintainability and scalability.
• Containerization: Ensures consistent deployment across different environments.
• Real-Time Data Handling: Utilizes WebSockets for dynamic user experience.
d. Include Relevant Technologies, Frameworks, or Libraries:
• Programming Language: Go
• Web Server: Go’s standard net/http package
• WebSockets: Likely using a Go WebSocket library (e.g., Gorilla WebSocket)
• Docker: For containerization
• GitHub API: For searching repositories and accessing data
• CI/CD Tools: GitHub Actions for automation
Elements
• Clear Labels:
• Data flow (e.g., User Actions → Frontend → Backend → GitHub API).
• Dependencies (e.g., Backend relies on GitHub API).