wittjan/cognitive_robot_abstract_machine

1. Repository Scope and Entry Points

CRAM is a Python monorepo for cognitive robotics: symbolic task planning, semantic world modeling, perception, motion control, probabilistic reasoning, and event analysis. It is primarily a collection of installable libraries rather than one server application.

Primary integration entry points are package APIs and demos:

• Task-level robot programs: coraplex/src/coraplex/__init__.py, coraplex/demos/coraplex_bullet_world_demo/demo.py

• ROS perception process: robokudo/src/robokudo/scripts/main.py

• Motion-controller ROS launch/config scripts: giskardpy/src/giskardpy/middleware/ros2/giskard.py

• Batch simulation/event replay: segmind/src/segmind/episode_player.py

2. Coraplex: Cognitive Plans and Robot Actions

coraplex is the orchestration layer. It represents plans as graphs of executable nodes, conditions, callbacks, designators, and failure handling; robot-level actions compose motion primitives such as navigation, grasping, placing, and body control.

Key boundaries:

• Plan graph/runtime: coraplex/src/coraplex/plans/plan.py, coraplex/src/coraplex/plans/plan_node.py

• High-level action composition: coraplex/src/coraplex/robot_plans/actions/base.py, coraplex/src/coraplex/robot_plans/actions/composite/transporting.py

• Motion dispatch and ROS-neutral compatibility: coraplex/src/coraplex/motion_executor.py, coraplex/src/coraplex/ros/ros2/action_lib.py, coraplex/src/coraplex/ros/no_ros/action_lib.py

A typical flow is: a plan resolves designators/locations, validates goals, invokes robot actions, and delegates motion execution to Giskard or alternative robot mappings.

3. Semantic Digital Twin: Authoritative World Model

semantic_digital_twin models robots, objects, geometry, joints, state, semantic annotations, and world modifications. It is the central state boundary between planning, perception, collision checking, and simulation.

Core model and behavior:

• World aggregate and entity/state model: semantic_digital_twin/src/semantic_digital_twin/world.py, semantic_digital_twin/src/semantic_digital_twin/world_description/world_entity.py, semantic_digital_twin/src/semantic_digital_twin/world_description/world_state.py

• Kinematics, spatial computations, and semantic queries: semantic_digital_twin/src/semantic_digital_twin/spatial_computations/forward_kinematics.py, semantic_digital_twin/src/semantic_digital_twin/reasoning/world_reasoner.py

• Collision and route-space representations: semantic_digital_twin/src/semantic_digital_twin/collision_checking/collision_manager.py, semantic_digital_twin/src/semantic_digital_twin/world_description/graph_of_convex_sets.py

Adapters import/export URDF, MJCF, meshes, datasets, and ROS world updates.

4. Giskard: Constraint-Based Motion Control

giskardpy turns motion goals and live robot/world state into velocity or trajectory commands. Motion statecharts express goals, monitors, and task constraints; a QP controller solves the active constraint system; behavior-tree branches coordinate the control loop.

• Statechart goals/tasks/monitors: giskardpy/src/giskardpy/motion_statechart/motion_statechart.py, giskardpy/src/giskardpy/motion_statechart/goals/cartesian_goals.py

• Optimization backend and selectable solvers: giskardpy/src/giskardpy/qp/qp_controller.py, giskardpy/src/giskardpy/qp/solvers/qp_solver_gurobi.py

• ROS control-loop execution: giskardpy/src/giskardpy/tree/branches/control_loop.py, giskardpy/src/giskardpy/tree/behaviors/send_trajectory.py

5. RoboKudo: Perception Pipelines

robokudo is an analysis-engine pipeline for camera/point-cloud inputs and semantic scene annotations. It ingests sensor or storage data into a CAS, executes annotators, produces object hypotheses, and can connect results to the semantic twin.

• Pipeline/runtime and CAS: robokudo/src/robokudo/analysis_engine.py, robokudo/src/robokudo/pipeline.py, robokudo/src/robokudo/cas.py

• Perception stages: robokudo/src/robokudo/annotators/pointcloud_cluster_extractor.py, robokudo/src/robokudo/annotators/object_associator.py

• External I/O boundary: robokudo/src/robokudo/io/ros.py, robokudo/src/robokudo/io/semantic_digital_twin.py

6. Knowledge, Queries, and Uncertainty

krrood supplies object-relational mapping, graph-backed Entity Query Language (EQL), rule systems, and query verbalization. It underpins semantic persistence/introspection across packages. probabilistic_model and random_events provide probabilistic circuits, distributions, learning, and event/set algebra.

• EQL expression/query evaluation: krrood/src/krrood/entity_query_language/evaluation.py, krrood/src/krrood/entity_query_language/query/query.py

• ORM/persistence abstraction: krrood/src/krrood/ormatic/ormatic.py, krrood/src/krrood/ormatic/eql_interface.py

• Probability and event foundations: probabilistic_model/src/probabilistic_model/probabilistic_model.py, random_events/src/random_events/sigma_algebra.py

7. Runtime and Infrastructure Boundaries

The primary runtime is Python; ROS 2 is optional for real robots and integration tests. Docker images provide reproducible ROS environments (.github/docker/Dockerfile, coraplex/docker/ros2/Dockerfile). Physics support includes MuJoCo via physics_simulators/src/physics_simulators/mujoco_simulator.py; collision backends include PyBullet and Trimesh. External interfaces include ROS 2 topics/actions/services, sensor hardware, URDF/MJCF assets, SQLAlchemy-oriented ORM generation, and optional QP solvers including Gurobi, PIQP, qpSWIFT, and QPALM.