How shivjaiswal-coder/siamese-network-for-voice-pair-matching works
Primary Components
• Data Layer:
• Dataset: Two main folders exist under “dataset” – one for training audio files (train_wav) and one for testing (test_wav). These provide the raw audio signals.
• Preprocessing & Feature Extraction:
• Feature extraction routines extract MFCCs, Mel spectrograms, zero-crossing rate, spectral-roll off, spectral flux, and chroma features from audio signals.
• This module likely takes audio input (wav files) and applies windowing, DFT, log transformation, and a Mel scale conversion (as described in the README).
• Model Building:
• Base CNN model: A standard CNN architecture that acts as one limb of the Siamese network.
• Siamese Network: Two (or more) identical CNNs with shared weights. They take two audio feature inputs and produce embeddings.
• Contrastive loss function: To compare the Euclidean distance between pairs of embeddings, driving similar pairs closer and dissimilar pairs apart.
• Training & Validation:
• Scripts: There is a “Script_training_validation” folder containing a Python script and a Jupyter notebook. These are used to construct the training/validation loop, set up data pairs (positive and negative), and monitor accuracy and loss.
• Model Artifacts:
• The “model” directory holds the saved model (using TensorFlow’s SavedModel format with “saved_model.pb” and associated variable files). This represents the trained Siamese network that can be deployed for inference.
Mapping the Data Flow & Interactions
• Data Ingestion:
• Raw audio files (from train_wav and test_wav directories) are fed into the preprocessing module.
• Feature Extraction:
• Once features such as MFCCs and Mel spectrograms are extracted, they become the inputs to the CNN-based feature embedder.
• Model Training:
• The feature set is input into the base CNN (shared between the two Siamese branches) then merged to compute the similarity distance.
• The Contrastive Loss function computes the error based on a Euclidean distance metric.
• Data flow from the feature extraction module into the Siamese network, and then the loss computation.
• The training script (both .py and .ipynb) orchestrates this pipeline: generating positive/negative training pairs, feeding them into the network, and updating weights.
• Model Evaluation and Persistence:
• Post-training, the final model is saved in the “model” directory.
Architectural Patterns & Design Principles
• Modular Design: Clearly separate data preprocessing, model construction, and training orchestration.
• Loss-Driven Optimization: Emphasize the contrastive loss function layer that drives training.
• End-to-End Pipeline: From raw data ingestion to deployed model artifacts, the flow should be continuous.