How api-rbtest/rbctest works
Project Type and Purpose
• This is a research-focused, command-line tool (an open-source toolkit) for mining and testing RESTful API constraints by combining static (LLM-based) and dynamic (Daikon‐based) approaches. It ingests API specifications and runtime traces, infers constraints, generates tests, and evaluates results.
Top-Level Components
1. RBCTest_dataset/ — self-collected OpenAPI specs and CSV traces for 8 services
2. agora_dataset/ — AGORA’s OpenAPI specs and trace CSVs
3. AGORA-eval/ — dynamic invariant results (Daikon CSVs, Excel sheets)
4. approaches/ —
• ground_truth/ — benchmark “true” constraints (Excel)
• ablation/ — results for ablation studies
• rbctest_agora_data/ & rbctest_our_data/ — detected constraints and evaluation outcomes (Excel, CSV)
B. Core Code Modules (Blue rectangles) under src/
1. Constraint Mining
• request_response_constraint_mining.py
• response_property_constraint_mining.py
• ablation variants: request_response_constraint_mining_ablation.py, response_property_constraint_mining_ablation.py
2. Test Generation
• constraints_test_generation.py
3. Evaluation
• evaluate_constraints_mining.py
• evaluate_test_gen.py
• src/eval/ (eval_invariants.py, request_approach_evaluate.py, response_approach_evaluate.py)
4. Utility Libraries (src/utils/)
• openapi_utils.py (parses OpenAPI specs)
• gptcall.py (wraps OpenAI API calls)
• convert_json_to_excel*.py, extract_request_response.py, dict_utils.py, etc.
5. Response Verification (src/response_body_verification/)
• constraint_inference.py, data_model_building.py, parameter_responsebody_mapping.py
6. Verifier (src/verifier/)
• example_verifier.py, find_example_utils.py
7. Orchestrator / CLI Entry Points
• verifier.py
C. External Services (Orange clouds)
• OpenAI API (for static, LLM-based mining)
• Daikon tool (dynamic invariant generation, CSVs under AGORA-eval/agora/)
1. Input Flow
[RBCTest_dataset/, agora_dataset/] → OpenAPI parser (openapi_utils) → JSON models
[RBCTest_dataset/ CSV traces] → dynamic pipeline (Daikon or overlapping_analysis)
2. Constraint Mining
• Static path: JSON models + gptcall → request/response constraint results → approaches/rbctest_our_data & approaches/rbctest_agora_data
• Dynamic path: Daikon CSVs → AGORA-eval → overlapping_analysis → Excel reports
3. Test Generation
• constraints_test_generation.py reads mined constraints → generates test cases → writes to approaches/rbctest_*_data
4. Evaluation
• evaluate_constraints_mining.py and src/eval/* modules compare mined vs ground truth → outputs CSV/Excel in approaches/ and AGORA-eval
• evaluate_test_gen.py analyzes test outcomes → writes to approaches/rbctest_our_data/test_outcomes_analyze.csv
5. Utility Invocations
• JSON/Excel conversions, request-response extraction flow through utils/* modules
6. Final Outputs
[approaches/, AGORA-eval/] → researcher review
Architectural Patterns and Principles
• Modularity: separate scripts for each major step (mining, generation, evaluation)
• Layered processing: Data ingestion → mining → generation → evaluation → reporting
• Extensibility: new mining strategies or datasets can plug into src/
• Reusability: utility and verification libraries factored out into src/utils/ and src/verifier/