How jgmelon22/miniofileconsumer works
Project type and purpose
• This is a standalone .NET “service worker” microservice (System B) in an event‑driven, message‑based architecture.
• Purpose: consume file‑processing requests from a Kafka topic (published by System A), download CSVs from MinIO (S3), validate them, and update their status in a MySQL database.
File structure
• Top‑level solution (.sln) and one project folder: FileUploaderPartB.Worker
• Under Worker:
• Enums, Models – domain types (FileMessage, ContactImport, Status, Gender)
• Infrastructure
– Configurations (AmazonS3Configuration, KafkaOptions)
– Data (DapperDbContext)
– Repositories (FileRepository implements IFileRepository)
– Services (CsvValidatorService, KafkaConsumerService, S3Service)
• Interfaces – service and repository contracts
• Program.cs & Worker.cs – host/entry point
• appsettings.json, launchSettings.json – configuration files
README for additional insights
• Tech stack: .NET 8, Apache Kafka, MinIO (S3), MySQL, Swagger
• Workflow description confirms event flow: Kafka → download (MinIO) → validate → DB update
• Acknowledges GitDiagram for initial structure sketch
Elements
• Clear labels: “Kafka Topic: file‑uploads”, “MinIO Object Store”, “MySQL – FileStatus Table”
• Kafka → KafkaConsumerService
• KafkaConsumerService → S3Service (GetObject)
• S3Service → CsvValidatorService
• CsvValidatorService → FileRepository → MySQL
• Message broker (e.g., yellow for Kafka)
• Storage (blue for MinIO)
• Database (green for MySQL)
• Application components (gray or grouping)