How armaan-dot/nora works
NORA turns spoken robot commands into structured intents, scores candidate actions for usefulness and physical feasibility, plans a skill sequence, and executes it through ROS 2. The principal path is Human → NLU → Intent → Affordance scoring → Orchestrator → primitive skills → MoveIt/controllers → robot or simulator. The repository also contains a replaceable fine-tuning pipeline and simulation integrations. Several ROS nodes and interfaces are documented but unsampled, so those links are shown only where README or source explicitly supports them.
Main entry points: orchestrator_node.py Orchestrator Node · nora_affordance Affordance Service · nlu_node.py NLU Node · local_parser.py Local Parser
Components 23
Language Understanding
- NLU Node
ros2_ws/src/nora_nlu_node/nora_nlu_node/nlu_node.py - Mock Parser
ros2_ws/src/nora_nlu_node/nora_nlu_node/mock_parser.py - Local Parser
ros2_ws/src/nora_nlu_node/nora_nlu_node/local_parser.py - Structured Intent
core/nora_core/intent.py - NLU Data Pipeline
ml/src/nora_nlu/data/synthetic_generator.py - LoRA Model
ml/src/nora_nlu/models/lora_model.py
Planning and Scoring
- Affordance Service
ros2_ws/src/nora_affordance - Usefulness Scorer
ros2_ws/src/nora_affordance/nora_affordance/scorers/llm_usefulness.py - Feasibility Scorers
ros2_ws/src/nora_affordance/nora_affordance/scorers - Score Fusion
core/nora_core/affordance/fusion.py - Skill Planner
core/nora_core/planner.py - Skill Registry
core/nora_core/skills.py
ROS Orchestration
- Orchestrator Node
ros2_ws/src/nora_orchestrator/nora_orchestrator/orchestrator_node.py - Task State Machine
ros2_ws/src/nora_orchestrator/nora_orchestrator/state_machine.py - Object Perception
ros2_ws/src/nora_perception
Robot Execution
- Primitive Skills
ros2_ws/src/nora_skills/nora_skills - Skill Action Base
ros2_ws/src/nora_skills/nora_skills/base_skill.py - MoveIt 2
ros2_ws/src/nora_moveit_config/launch/move_group.launch.py - ROS Controllers
ros2_ws/src/nora_control/launch/controllers.launch.py - Robot Arm
Simulation Targets
- Gazebo Simulation
ros2_ws/src/nora_gazebo/launch/gazebo.launch.py - Isaac Simulation
ros2_ws/src/nora_isaac/scripts/isaac_bridge_stub.py
Other components
- Human Operator
Connections 24
- Human Operator → NLU Node: sends text
- NLU Node → Mock Parser: parses with
- NLU Node → Local Parser: parses with
- Local Parser → LoRA Model: uses model
- Mock Parser → Structured Intent: returns intent
- Local Parser → Structured Intent: returns intent
- NLU Data Pipeline → LoRA Model: trains model
- NLU Node → Orchestrator Node: publishes intent
- Orchestrator Node → Affordance Service: scores skills
- Affordance Service → Usefulness Scorer: evaluates usefulness
- Affordance Service → Feasibility Scorers: evaluates feasibility
- Affordance Service → Score Fusion: fuses scores
- Affordance Service → Orchestrator Node: returns rankings
- Orchestrator Node → Skill Planner: builds plan
- Skill Planner → Skill Registry: reads skills
- Object Perception → Feasibility Scorers: provides state
- Orchestrator Node → Task State Machine: transitions state
- Orchestrator Node → Primitive Skills: dispatches skills
- Primitive Skills → Skill Action Base: inherits server
- Primitive Skills → MoveIt 2: requests motion
- MoveIt 2 → ROS Controllers: executes trajectory
- ROS Controllers → Robot Arm: drives joints
- Gazebo Simulation → ROS Controllers: simulates hardware
- Isaac Simulation → Robot Arm: bridges simulation