AIMA开放平台文档中心AimDK A3-ultra

7.3 Microphone Management

7.3 Microphone Management

7.3.1 Overview

Currently, the robot only supports using an external microphone (the Shike lavalier microphone) as the interaction input source; the built-in microphone is not yet configured or enabled. Additionally, the system supports setting Silent Mode via AimMaster and provides RPC interfaces for programmatic control of this mode. We also provide access to raw microphone audio output (including audio data processed with noise reduction, echo cancellation, and VAD). After disabling the A2 interaction pipeline, users can utilize this interface to capture microphone audio and integrate it with custom interaction systems. Please note that Silent Mode is an operational state within Normal Mode, allowing for flexible switching without the need to restart the agent.

7.3.2 Set Silent Mode RPC Interface

Interface Namepb:/aimdk.protocol.AgentControlService/SetVoiceEnable
Function OverviewSet Silent Mode
Interface Type HTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.AgentControlService/SetVoiceEnable
Input Parameters
text
{
  "enable_voice": false
}
  • enable_voice: Set to false to enable Silent Mode, set to true for Normal Mode
Output Parameters
text
{
  "header": {
    "code": "0",
    "msg": "SetVoiceEnable successfully",
    "trace_id": "",
    "domin": ""
  },
  "state": "CommonState_UNKNOWN"
}
  • state: No need to pay attention to this field
Example Scriptexamples/agent/SetVoiceEnable.sh
Remarks

7.3.3 Query Silent Mode RPC Interface

Interface Namepb:/aimdk.protocol.AgentControlService/GetVoiceEnable
Function OverviewQuery Silent Mode Status
Interface Type HTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.AgentControlService/GetVoiceEnable
Input Parameters
text
{}
Output Parameters
text
{
  "header": {
    "code": "0",
    "msg": "GetVoiceEnable successfully",
    "trace_id": "",
    "domin": ""
  },
  "enable_voice": true
}
  • enable_voice: Set to false to enable Silent Mode, set to true for Normal Mode
Example Scriptexamples/agent/GetVoiceEnable.sh
Remarks

7.3.4 Set Interaction Operation Mode RPC Interface

Interface Namepb:/aimdk.protocol.AgentControlService/SetAgentPropertiesRequest
Function OverviewSet Interaction Operation Mode
Interface Type HTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.AgentControlService/SetAgentPropertiesRequest
Input Parameters
text
{
  "contents": {
    "properties": {
      "2": "only_voice"
    }
  }
}
Modes:
  • only_voice: Only output noise-reduced microphone audio /agent/process_audio_output, subsequent pipeline is completely disconnected
  • normal: Standard operation mode, interaction runs normally
Output Parameters
text
{
  "state": "CommonState_UNKNOWN"
}
Example Scriptexamples/agent/SetAgentPropertiesRequest.sh
Remarks
  • Requires a robot reboot to take effect after calling
  • It is normal for the return value to be CommonState_UNKNOWN after calling. You can call the GetAgentPropertiesRequest interface to check if the interaction mode switch was successful

7.3.5 Get Interaction Operation Mode RPC Interface

Interface Namepb:/aimdk.protocol.AgentControlService/GetAgentPropertiesRequest
Function OverviewQuery Interaction Operation Mode
Interface Type HTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.AgentControlService/GetAgentPropertiesRequest
Input Parameters
text
{}
Output Parameters
text
{
  "contents": {
    "properties": {
      "2": "only_voice"
    }
  }
}
Modes:
  • only_voice: Only output noise-reduced microphone audio /agent/process_audio_output, subsequent pipeline is completely disconnected
  • normal: Standard operation mode, interaction runs normally
Example Scriptexamples/agent/GetAgentPropertiesRequest.sh
Remarks

7.3.6 Noise-Reduced Microphone Audio Topic Interface

Interface Name/agent/process_audio_output
Function OverviewNoise-Reduced Microphone Audio Interface
Interface Type ROS2 Topic
Output Parameters
text
{
  "stream_id": 2,
  "vad_state": "AUDIO_VAD_STATE_PROCESSING",
  "audio_data": "..."
}
  • stream_id: Microphone identifier, 1 for built-in mic, 2 for external mic
  • vad_state: Voice activity detection status
    • AUDIO_VAD_STATE_NONE = 0
    • AUDIO_VAD_STATE_BEGIN = 1
    • AUDIO_VAD_STATE_PROCESSING = 2
    • AUDIO_VAD_STATE_END = 3
  • audio_data: Audio byte stream data
Example Scriptexamples/agent/get_voice.py
Remarks
  • The ROS2 message type for this message is ros2_plugin_proto/msg/RosMsgWrapper, requires sourcing prebuilt/ros2_plugin_proto_aarch64/share/ros2_plugin_proto/local_setup.bash before use.
  • Note: To obtain the following audio, the robot needs to be connected to the internet for at least 2 minutes after startup to complete audio-related authentication operations, otherwise there will be no raw audio output. For offline use, please first ensure that this interface has audio output before disconnecting from the network.
  • Note: In the current version, the vad_state output of the external microphone interface has issues. The expected state for one voice input is 122222222223, but the actual output state is 0111111111112. This issue only appears in external microphone scenarios (built-in microphone works normally) and is planned to be fixed in subsequent versions. In the current version, it is recommended to manually perform +1 compensation on the state.

7.3.7 Wake-Up Result Reporting

Interface Name /agent/wakeup
Function OverviewWake-Up Result Reporting
Interface Type ROS2 Topic
Output Parameters
text
{
    "language":"zh",
    "keyword":"Expedition Expedition",
    "timestamp":"1768812507657",
    "confidence":1.0,
    "wakeup_id":"event_wMXK9pT05JMXeRAOSc706",
    "is_success":true,
    "wakeup_type":"WAKEUP_NORMAL"
}
  • confidence: Speech matching confidence, numerical value between 0-1
  • Keyword: Wake-up word
  • language: Currently only supports Chinese
  • wakeup_type:
    • WAKEUP_UNKNOWN : Wake-up with unknown source or status (placeholder/exception scenario)
    • WAKEUP_NORMAL : Normal voice wake-up, for example user speaks default wake-up word.
    • WAKEUP_CUSTOM_TRIGGERED : Wake-up triggered by custom method, such as custom wake-up word or external event.
Example Scriptexamples/agent/get_wakeup_result.py
Remarks
  • The ROS2 message type for this message is ros2_plugin_proto/msg/RosMsgWrapper, requires sourcing prebuilt/ros2_plugin_proto_aarch64/share/ros2_plugin_proto/local_setup.bash before use.

7.3.8 Built-in and External Microphone Switching RPC Interface

Interface Namepb:/aimdk.protocol.AgentControlService/SetMicSourceRequest
OverviewSwitch the microphone source between the built-in microphone and the external microphone
Interface Type HTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.AgentControlService/SetMicSourceRequest
Input
text
{
  "mic_source": 1
}
  • mic_source: 0 indicates the built-in microphone, 1 indicates the external microphone, and all other values are invalid.
Output
text
{
  "header": {
    "code": "0",
    "msg": "SetVoiceEnable successfully",
    "trace_id": "",
    "domin": ""
  },
  "state": "CommonState_UNKNOWN"
}
  • state: This field can be ignored.
Example Scriptexamples/agent/SetMicSource.sh
Remarks

7.3.9 Built-in and External Microphone Status Query RPC Interface

Interface Namepb:/aimdk.protocol.AgentControlService/GetMicSourceRequest
OverviewGet the microphone source currently in use
Interface Type HTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.AgentControlService/GetMicSourceRequest
Input
text
{}
Output
text
{
  "header": {
    "code": "0",
    "msg": "Get mic source successfully",
    "trace_id": "",
    "domin": ""
  },
  "mic_source": 0
}
  • mic_source: 0 indicates the built-in microphone, 1 indicates the external microphone, and all other values are invalid.
Example Scriptexamples/agent/GetMicSource.sh
Remarks