How harmonydata/harmony works
Main Components Identification
The system should be divided into these primary components:
a) Input Processing Layer
• PDF Parser
• Excel Parser
• Text Parser
• File Type Detection
b) Core Processing Layer
• Matching Engine
• Vectorization Service
• Clustering System
• Negation Handler
c) Schema Layer
• Instrument Schemas
• Question Schemas
• Request/Response Models
d) Utility Services
• Model Management
• File Helpers
• Instrument Helpers
Component Relationships
[Input Files (PDF/Excel/Text)] → [Parsing Layer] → [Schema Layer] → [Core Processing Layer] → [Output Results]
Specific Implementation Details
a) Parser Components:
• Tika integration for PDF processing
• Excel parsing capabilities
• Text parsing direct input
b) Matching System:
• Default matcher
• WMD matcher
• Clustering functionality
• Vector generation system
c) External Dependencies:
• Apache Tika Server
• Sentence Transformers
• spaCy Models
• Optional OpenAI integration
Key Architectural Patterns to Highlight
• Modular design with clear separation of concerns
• Plugin architecture for different matchers
• Service-based approach for core functionalities
• Schema-driven data validation
Implementation Details
[File Input] → [Parser Selection] → [Text Extraction] → [Vector Generation] → [Similarity Matching] → [Results]
With parallel flows for:
• Model management
• Schema validation
• Error handling
Technical Stack Indicators
• Python 3.8+
• spaCy
• Sentence Transformers
• Apache Tika
• Docker support