How sjtu-jcourse/jcourse_go works
Project Type and Purpose
• Project Type: Backend service for a course/community platform.
• Purpose: Provide RESTful APIs for course management, user authentication, course reviews, ratings, teacher information, and asynchronous tasks like statistics and distributed locking.
• It supports database migration and data import capabilities (as seen in the cmd subfolders like migrate and import_from_jwc) and is designed to run as a single backend binary.
File Structure
• Top-Level Directories:
• cmd: Contains multiple executable commands (e.g., code generation, migration, refreshing database, various importers). This indicates that the project uses a modular approach with different entry points for different tasks.
• constant: Houses constant definitions (configured for authentication, email, query handling, etc.), which centralizes configuration strings.
• dal: Database abstraction layer for interfacing with SQL and Redis.
• handler: HTTP request handlers that serve as the API endpoints. They call into business logic without containing the logic themselves.
• middleware: Contains components for HTTP middleware like authentication, CSRF, session management, and statistics collection.
• model: Organized into subdirectories for data transfer objects (dto), persistent objects (po), domain-specific models (if present), and converters. This separation hints an MVC/MV* style architecture for clean separation of concerns.
• repository: Implements the abstracted data access querying logic, isolating business logic from storage details.
• rpc: Provides modules for external communication (e.g., LLM and SMTP service invocation).
• service: Contains the actual business logic implementations that put together multiple domain operations.
• task: Implements asynchronous tasks. There is evidence of multiple submodules such as one using "asynq" for distributed task processing and another for business tasks such as ping and statistics. These tasks operate independently from the regular request handling.
• util: Houses helper and utility functions for various general-purpose tasks (e.g., conversion, environment parsing, error handling, time utilities, and even Selenium integration for data loading).
• Supporting Files:
• Configuration and build files (Makefile, Dockerfile, docker-compose.yml, .env instructions, etc.) indicate deployment and dependency management.
• .github/workflows suggest CI/CD integration for Golang builds and lint checks.
Key Elements
a. Main Components:
• API Layer: Handler components (handler/…) handling HTTP requests.
• Middleware: Components that manage security and session state (middleware/…).
• Business Logic Layer: Service modules (service/…) implementing core functions.
• Data Access Layer: Repository (repository/…), DAL (dal/…), and model definitions (model/).
• Asynchronous Task Processing: Task components (task/asynq, task/base, task/biz) managing background jobs.
• External Integrations: RPC modules (rpc/llm, rpc/smtp) to communicate with external services.
• Command-Line Utilities: Executables in cmd (e.g., migrate, importers) which are used for database initialization, data imports, and code generation.
• Infrastructure: Database systems (Postgres, Vector DB) and Redis as indicated by the configuration in the README.
b. Relationships and Interactions:
• Incoming HTTP requests hit the API layer (handled by the “handler” packages). They pass through a chain of middleware components (authentication, CSRF, session, statistics, etc.) before reaching the business logic.
• The service layer orchestrates business operations, relying on the repository layer to interact with the underlying databases.
• DAL provides abstraction over different types of databases (SQL and Redis) while models (dto, po, converter) define the data structures shared between layers.
• Asynchronous tasks are triggered by events in the service layer or via external scheduling. They run independently to perform background work like statistical computations or distributed locking.
• RPC modules allow communication with systems like language processing services (LLM) or sending emails (SMTP).
• Command-line executables serve as one-off utilities for migrations, imports, and refresh operations.
c. Architectural Patterns and Design Principles:
• Layered Architecture: Clear separation between presentation (handlers), business (services), and data access (dal/repository).
• Separation of Concerns: Task handling and command-line utilities are separated from the main API server logic.
• Modularity: Multiple “cmd” executables indicate a separation of responsibilities.
• Dependency Injection & Abstraction: Repository and service layers abstract underlying DB access, making the system more maintainable and testable.
• Asynchronous Task Processing: Use of “asynq” framework for handling distributed tasks.
d. Relevant Technologies and Frameworks:
• Golang (>=1.21) for its strong concurrency support.
• PostgreSQL (and a distinct VectorDB instance) as the primary data store.
• Redis for caching and possibly for session storage.
• Docker and docker-compose for containerized deployment.
• CI tools defined in the GitHub workflows (linting, Go testing, etc.).
• Asynq for distributed task processing.
• Selenium scripts (in util/selenium-get) for data imports or scraping operations.