feat: rai_whoami

docs/create_robots_whoami.md

@@ -4,68 +4,75 @@ RAI Agent needs to understand what kind of robot it is running on.
 This includes its looks, purpose, ethical code, equipment, capabilities and documentation.
 To configure RAI for your robot, provide contents for your robot's so called `whoami` package.
 
-Your robot's `whoami` package serves as a configuration package for the `rai_whoami` node.
-
-> [!TIP]
-> The Human-Machine Interface (HMI), both text and voice versions, relies heavily on the whoami package. It uses the robot's identity, constitution, and documentation to provide context-aware responses and ensure the robot behaves according to its defined characteristics.
-
 ## Configuration example - Franka Emika Panda arm
 
 1. Setup the repository using 1st and 2nd step from [Setup](../README.md#setup)
 
-2. Create a whoami package for Panda
+2. Fill in the `src/examples/panda_whoami/description` folder with data:
 
-   ```shell
-   ./scripts/create_rai_ws.sh --name panda --destination-directory src/examples
-   ```
+   2.1. Save [this image](https://robodk.com/robot/img/Franka-Emika-Panda-robot.png) into `panda/images`
 
-3. Fill in the `src/examples/panda_whoami/description` folder with data:
+   2.2. Save [this document](https://github.com/user-attachments/files/16417196/Franka.Emika.Panda.robot.-.RoboDK.pdf) in `panda/documentation`
 
-   3.1. Save [this image](https://robodk.com/robot/img/Franka-Emika-Panda-robot.png) into `src/examples/panda_whoami/description/images`
+   2.3. Save [this urdf](https://github.com/frankaemika/franka_ros/blob/develop/franka_description/robots/panda/panda.urdf.xacro) in `panda/urdf`
 
-   3.2. Save [this document](https://github.com/user-attachments/files/16417196/Franka.Emika.Panda.robot.-.RoboDK.pdf) in `src/examples/panda_whoami/description/documentation`
+3. Build the embodiment info using `build_whoami.py`:
 
-   3.3. Save [this urdf](https://github.com/frankaemika/franka_ros/blob/develop/franka_description/robots/panda/panda.urdf.xacro) in `src/examples/panda_whoami/description/urdf`
-
-4. Run the `parse_whoami_package`. This will process the documentation, building it into a vector database, which is used by RAI agent to reason about its identity.
+```shell
+python src/rai_whoami/rai_whoami/build_whoami.py panda/ --build-vector-db
+```
 
 > [!IMPORTANT]
-> Parsing bigger documents with Cloud vendors might lead to costs. Consider using the
-> local `ollama` provider for this task. Embedding model can be configured in
+> Building the vector database with cloud vendors might lead to costs. Consider using the
+> local `ollama` provider for this task. The embedding model can be configured in
 > [config.toml](../config.toml) (`ollama` works locally, see [docs/vendors.md](./vendors.md#ollama)).
 
-```shell
-./scripts/parse_whoami_package.sh src/examples/panda_whoami
-```
+4. Examine the generated files
 
-5. Optional: Examine the generated files
+After running the build command, inspect the generated files in the `panda/generated` directory. The folder should contain a info.json file containing:
 
-After running the `parse_whoami_package` command, you can inspect the generated files in the `src/examples/panda_whoami/description/generated` directory. These files contain important information about your robot:
+- `rules`: List of rules
+- `capabilities`: List of capabilities
+- `behaviors`: List of behaviors
+- `description`: Description of the robot
+- `images`: Base64 encoded images
 
-- `robot_identity.txt`: Contains a detailed description of the robot's identity, capabilities, and characteristics.
-- `robot_description.urdf.txt`: Provides a summary of the robot's URDF (Unified Robot Description Format), describing its physical structure.
-- `robot_constitution.txt`: Outlines the ethical guidelines and operational rules for the robot.
-- `faiss_index`: A directory containing the vector store of the robot's documentation, used for efficient information retrieval.
+## Testing (ROS 2)
 
-## Testing
+You can test the generated package by using the RAI Whoami services:
 
-You can test your new `panda_whoami` package by calling `rai_whoami` services:
+1. Using the RAI Whoami services:
 
-1. Building the `rai_whoami` package and running the `rai_whoami_node` for your `Panda` robot:
+Run the RAI Whoami services:
 
-```shell
-colcon build --symlink-install
-ros2 run rai_whoami rai_whoami_node --ros-args -p robot_description_package:="panda_whoami"
+```bash
+python src/rai_whoami/rai_whoami/scripts/ros2_embodiment_service.py panda/ &
+python src/rai_whoami/rai_whoami/scripts/ros2_vector_store_retrieval_service.py panda/
 ```
 
-2. Calling the rai_whoami services
-
 ```shell
-ros2 service call /rai_whoami_identity_service std_srvs/srv/Trigger # ask for identity
-ros2 service call /rai_whoami_selfimages_service std_srvs/srv/Trigger # ask for images folder
-ros2 service call /rai_whoami_constitution_service std_srvs/srv/Trigger # ask for robot constitution
-ros2 service call /rai_whoami_urdf_service std_srvs/srv/Trigger # ask for urdf description
-ros2 service call /rai_whoami_documentation_service rai_interfaces/srv/VectorStoreRetrieval "query:  'maximum load'" # ask for Panda's maximum load
+# Get robot's identity
+ros2 service call /rai_whoami_embodiment_info_service rai_interfaces/srv/EmbodimentInfo
+
+# Query the vector database
+ros2 service call /rai_whoami_documentation_service rai_interfaces/srv/VectorStoreRetrieval "query: 'maximum load'"
 ```
 
-If your service calls succeed, your `panda_whoami` package has been properly initialized.
+If your service calls succeed and you can access the embodiment info and vector database, your robot's whoami package has been properly initialized.
+
+2. Alternatively, you can use the RAI Whoami tools directly in your Python code:
+
+```python
+from rai_whoami import EmbodimentInfo
+from rai_whoami.tools import QueryDatabaseTool
+
+# Load embodiment info
+info = EmbodimentInfo.from_directory("panda/generated")
+
+# Create a system prompt for your LLM
+system_prompt = info.to_langchain()
+
+# Use the vector database tool
+query_tool = QueryDatabaseTool(root_dir="panda/generated")
+query_tool._run(query="maximum load")
+```

pyproject.toml

@@ -15,6 +15,7 @@ package-mode = false
 python = "^3.10, <3.13"
 
 rai = {path = "src/rai_core", develop = true}
+rai_whoami = {path = "src/rai_whoami", develop = true}
 pre-commit = "^3.7.0"
 tabulate = "^0.9.0"
 pytest = "^8.2.0"

src/rai_core/rai/agents/langchain/react_agent.py

@@ -20,6 +20,7 @@
 from rai.agents.langchain.agent import LangChainAgent
 from rai.agents.langchain.runnables import ReActAgentState, create_react_runnable
 from rai.communication.hri_connector import HRIConnector, HRIMessage
+from rai.messages.multimodal import SystemMultimodalMessage
 
 
 class ReActAgent(LangChainAgent):
@@ -29,7 +30,7 @@ def __init__(
         llm: Optional[BaseChatModel] = None,
         tools: Optional[List[BaseTool]] = None,
         state: Optional[ReActAgentState] = None,
-        system_prompt: Optional[str] = None,
+        system_prompt: Optional[str | SystemMultimodalMessage] = None,
     ):
         runnable = create_react_runnable(
             llm=llm, tools=tools, system_prompt=system_prompt

src/rai_core/rai/agents/langchain/runnables.py

@@ -33,7 +33,7 @@
 
 from rai.agents.tool_runner import ToolRunner
 from rai.initialization import get_llm_model
-from rai.messages import HumanMultimodalMessage
+from rai.messages import HumanMultimodalMessage, SystemMultimodalMessage
 
 
 class ReActAgentState(TypedDict):
@@ -48,7 +48,11 @@ class ReActAgentState(TypedDict):
     messages: List[BaseMessage]
 
 
-def llm_node(llm: BaseChatModel, system_prompt: Optional[str], state: ReActAgentState):
+def llm_node(
+    llm: BaseChatModel,
+    system_prompt: Optional[str | SystemMultimodalMessage],
+    state: ReActAgentState,
+):
     """Process messages using the LLM.
 
     Parameters
@@ -68,7 +72,10 @@ def llm_node(llm: BaseChatModel, system_prompt: Optional[str], state: ReActAgent
     ValueError
         If state is invalid or LLM processing fails
     """
-    if system_prompt:
+    if isinstance(system_prompt, SystemMultimodalMessage):
+        if not isinstance(state["messages"][0], SystemMessage):
+            state["messages"].insert(0, system_prompt)
+    elif system_prompt:
         # at this point, state['messages'] length should at least be 1
         if not isinstance(state["messages"][0], SystemMessage):
             state["messages"].insert(0, SystemMessage(content=system_prompt))
@@ -79,7 +86,7 @@ def llm_node(llm: BaseChatModel, system_prompt: Optional[str], state: ReActAgent
 def create_react_runnable(
     llm: Optional[BaseChatModel] = None,
     tools: Optional[List[BaseTool]] = None,
-    system_prompt: Optional[str] = None,
+    system_prompt: Optional[str | SystemMultimodalMessage] = None,
 ) -> Runnable[ReActAgentState, ReActAgentState]:
     """Create a react agent that can process messages and optionally use tools.
 

src/rai_core/rai/communication/ros2/connectors/base.py

@@ -252,8 +252,8 @@ def get_transform(
     def create_service(
         self,
         service_name: str,
-        on_request: Callable,
-        on_done: Optional[Callable] = None,
+        on_request: Callable[[Any, Any], Any],
+        on_done: Optional[Callable[[Any, Any], Any]] = None,
         *,
         service_type: str,
         **kwargs: Any,