How benjimayonez/invertible-scattering-transform works
Main Components
• Core Library (kymatio-main/kymatio): This is the heart of the project where modifications are made. Mark components such as:
– The core modules in “scattering1d”, “scattering2d”, and “scattering3d” (with a specific focus on scattering2d where the invertibleScattering2d function is added).
– Backend modules (e.g., jax_backend, numpy_backend, tensorflow_backend, torch_backend, and torch_skcuda_backend) that offer support for multiple computation frameworks.
– Frontend modules within each scattering dimension (providing simple entry points into the scattering functionality).
• Examples (kymatio-main/examples): This folder demonstrates how the new invertible scattering functionality is used:
– 2D examples (e.g., “plot_scattering_frequencies.py”, “long_mnist_classify_torch.py”, “cifar_torch.py”) to validate the functionality with visualization and classification.
– 1D and 3D examples providing context and additional use cases.
• Texture Reconstruction (texture_reconstruction): A separate module that uses the invertible scattering transform (specifically the 2D function) to perform texture reconstruction. It’s important to show how output from the scattering module flows into this component.
Interactions and Data Flow
• That the input flows into one of the frontend wrappers (e.g., the tensorflow_frontend, torch_frontend, etc.), which then route the data to the appropriate backend (e.g., numpy_backend, torch_backend, etc.). Indicate that the backend and frontend layers work together to compute both the standard and invertible scattering transforms.
– One path leads to the examples demonstrating classification and plotting.
– The other path flows into the texture reconstruction module where the scattering coefficients are used to recreate textures.
• A node for “Tests” which interacts with core libraries—validating backend consistency across frameworks (as seen in tests/general and tests/scattering{1d,2d,3d}) ensuring that selected transformations meet expected norms.
Architectural Patterns and Design Principles
• Modular design pattern: the clear separation between the core scattering implementation (splitting into backend and frontend layers) and the application examples.
• Extension pattern: the project builds upon the Kymatio library by adding additional functionality (invertibility) without overhauling the original API.
• External dependencies such as various deep learning/numerical libraries (PyTorch, TensorFlow, JAX, and NumPy) which are referenced through the backend modules.
• Continuous integration and deployment (CI/CD) aspects via configuration files (see the .github workflows and Jenkinsfile).