How samplics-org/samplics works
Overall Architecture
• The project is a standalone Python library (with tests, documentation, CI/CD workflows, etc.) rather than a full-stack web app.
• The library is organized into multiple sub-packages, each representing a functional component of survey analysis.
Main Components
a. Core Analytical Modules (located under src/samplics):
• Sampling: Contains methods for sample selection and sample size calculation (e.g., SRS, systematic selection, and PPS-based algorithms).
• Weighting: Implements various weight adjustments – handling nonresponse, calibration, and replication-based weight adjustments.
• Estimation: Covers both Taylor linearization and replication-based estimation methods.
• Small Area Estimation (SAE): Provides models for area-level and unit-level estimation when the sample size is small.
b. Supporting Components:
• Datasets: Loads and organizes sample data required for the analytical methods.
• APIs: Exposes higher-level interfaces for users to interact with underlying functionalities (e.g., fit, compare, predict).
• Utilities and Types: Offering shared functions, error handling, and data container definitions used across the package.
• Regression: Implements additional estimation techniques leveraging regression models.
c. Infrastructure and Auxiliary Components:
• Documentation (docs folder): Includes the website that explains usage, tutorials, and examples.
• Tests: A comprehensive test suite in the tests folder, ensuring functionality for each sub-package.
• CI/CD & Configuration Files: A .github/workflows directory and configuration files (pyproject.toml, .gitignore, etc.) indicating automated testing, coverage, and documentation builds.
Demonstrate Relationships and Interactions
• Data Flow:
• The data “entry point” starts with datasets—modules such as load_nhanes2, load_psu_frame, etc., provide standardized input data.
• The Sampling package uses this data to calculate sample sizes or perform selection algorithms and passes results to Weighting or Estimation methods.
• The Weighting module takes design weights from Sampling (or directly from datasets) and performs adjustments for nonresponse or replication.
• The Estimation module then uses outputs from Sampling and Weighting to estimate population parameters using either linearization or replication-based strategies.
• In cases where data for small geographical areas is insufficient, the SAE sub-package’s models (area-level or unit-level) come into play.
• API Layer:
• The APIs subdirectory wraps these core functions so that users can execute high-level calls (e.g., SampleSize.calculate, SampleWeight.adjust, etc.).
• Utilities and Types modules support error handling and specific data structures across these APIs.
• Testing & CI/CD:
• The tests directory mirrors the package structure, ensuring each functional component is verified.
• The GitHub workflows and configuration (like uv.lock) represent the deployment and integration testing environment.
Architectural Patterns and Design Principles
• Modularity: Each sub-package is decoupled by functionality (sampling, weighting, estimation, etc.), facilitating independent development and testing.
• API Abstraction: The “apis” folder provides user-friendly functions that encapsulate the underlying complexity.
• Layered Architecture: From raw data ingestion (Datasets) ➔ analytic processing (Sampling, Weighting, Estimation, SAE) ➔ result prediction (APIs), each layer has a specific role.
• Test-Driven Development: A robust tests directory indicates a commitment to software quality and continuous integration.
Specific Elements
• Relation between “APIs” and these core modules, emphasizing that it encapsulates the analytic functionalities.