How fuscoalejandro04/listas2.0 works
Purpose and Entry Points
AIPDP is a Python product-data processing platform centered on extracting structured product information from Excel workbooks, normalizing it, and assigning taxonomy categories. The interactive entry point is ui/streamlit_app.py; backend processing is organized under backend/pipelines/. Excel ingestion and sheet-structure discovery live in excel/.
Product and Taxonomy Domain
Core business entities are defined in backend/domain/product.py and backend/domain/taxonomy.py. Product records flow through enrichment, validation, and categorization stages. Taxonomy knowledge is externally represented as backend/infrastructure/knowledge/taxonomy.yaml, with product-line reference data in backend/infrastructure/knowledge/lineas_producto.json.
Excel Intake and Structural Detection
The Excel subsystem reads workbooks and identifies usable tabular regions before product interpretation:
• excel/workbook_reader.py loads workbook content.
• excel/worksheet_scanner.py traverses sheets.
• excel/table_region_detector.py identifies probable data-table boundaries.
• excel/header_candidate_generator.py and excel/header_scorer.py infer likely headers.
• excel/style_analyzer.py and excel/type_classifier.py contribute formatting and value-type signals.
This boundary isolates spreadsheet-specific heuristics from the backend product pipeline.
Processing Pipeline
backend/pipelines/processor.py is the likely orchestration boundary for transforming imported source rows into valid product-domain records. Its stages are separated by responsibility:
• backend/pipelines/importers.py: converts source data into pipeline inputs.
• backend/pipelines/normalizers.py: standardizes fields and values.
• backend/pipelines/validators.py: enforces record quality and required structure.
• backend/pipelines/context_detector.py and backend/pipelines/detectors.py: infer product context or attributes from available data.
Categorization and AI Enrichment
Classification uses a layered approach. backend/pipelines/rule_categorizer.py applies deterministic taxonomy rules against the knowledge files. backend/pipelines/ai_enricher.py provides an AI-driven enrichment stage, likely for fields or category signals unresolved by rules. This creates a clear decision boundary between reproducible rule classification and external/model-based inference.
Confidence and Taxonomy Validation
Excel-specific confidence logic is isolated in excel/confidence_calculator.py, supporting selection among candidate headers or detected structures. excel/taxonomy_validator.py checks spreadsheet-derived category data against the taxonomy model. Backend taxonomy behavior is validated at the domain boundary by tests/test_taxonomy.py.
Runtime and Infrastructure
The repository is a Python application with dependencies declared in requirements.txt. Streamlit is the material UI runtime via ui/streamlit_app.py. Local knowledge files under backend/infrastructure/knowledge/ are the primary visible infrastructure dependency; no database, queue, API service, or deployment runtime is represented in the provided tree.
Components 20
Interactive UI
- Streamlit app
ui/streamlit_app.py
Excel Intake
Spreadsheet-specific structure discovery and validation.
- Workbook reader
excel/workbook_reader.py - Worksheet scanner
excel/worksheet_scanner.py - Table region detector
excel/table_region_detector.py - Header inference · Generates and scores candidate headers using style and value signals.
excel/header_candidate_generator.py - Style analyzer
excel/style_analyzer.py - Type classifier
excel/type_classifier.py - Confidence calculator
excel/confidence_calculator.py - Excel taxonomy validator
excel/taxonomy_validator.py
Product Pipeline
Transforms detected source rows into product records.
- Processor
backend/pipelines/processor.py - Importers
backend/pipelines/importers.py - Normalizers
backend/pipelines/normalizers.py - Context detection
backend/pipelines/context_detector.py - Validators
backend/pipelines/validators.py - Rule categorizer
backend/pipelines/rule_categorizer.py - AI enricher · Enriches unresolved fields or category signals.
backend/pipelines/ai_enricher.py
Domain & Knowledge
- Product model
backend/domain/product.py - Taxonomy model
backend/domain/taxonomy.py - Taxonomy knowledge
backend/infrastructure/knowledge/taxonomy.yaml - Product-line reference
backend/infrastructure/knowledge/lineas_producto.json
Connections 20
- Streamlit app → Workbook reader: uploads workbook
- Workbook reader → Worksheet scanner: workbook content
- Worksheet scanner → Table region detector: sheet data
- Table region detector → Header inference: candidate regions
- Style analyzer → Header inference: style signals
- Type classifier → Header inference: type signals
- Header inference → Confidence calculator: header candidates
- Confidence calculator → Processor: selected table rows
- Processor → Importers: imports source rows
- Importers → Normalizers: pipeline inputs
- Normalizers → Context detection: normalized records
- Context detection → Validators: enriched context
- Validators → Rule categorizer: valid records
- Rule categorizer → AI enricher: unresolved signals
- AI enricher → Product model: enriched products
- Rule categorizer → Taxonomy knowledge: applies rules
- Rule categorizer → Product-line reference: uses references
- Taxonomy knowledge → Taxonomy model: defines taxonomy
- Excel taxonomy validator → Taxonomy model: validates against
- Excel taxonomy validator → Taxonomy knowledge: reads taxonomy