AIMA开放平台文档中心AimDK A3-ultra

7.2 TTS/Audio Playback Section

7.2 TTS/Audio Playback Section

7.2.1 Overview

A3 robot is equipped with a speaker. With network access, it can perform TTS playback. Without network, it can play basic audio files and adjust volume. It also supports playback status query and interruption control logic.

7.2.2 TTS Playback RPC Interface

Interface Namepb:/aimdk.protocol.TTSService/PlayTTS
Function OverviewTTS playback interface, requires network connection
Interface TypeHTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.TTSService/PlayTTS
Input Parameters
text
{
  "text": "Hello",
  "priority_level": "INTERACTION_L6",
  "domain": "example",
  "trace_id": "hafhjkqwjwefk",
  "is_interrupted": true
}
  • text: Text content to be played
  • priority_level: Priority, keep as INTERACTION_L6
  • domain: Source identifier, can pass a custom client string for troubleshooting
  • trace_id: Playback id, required if you want to query playback status; pass this value as the parameter for status query
Output Parameters
text
{
  "text": "Hello",
  "priority_level": "INTERACTION_L6",
  "priority_weight": 0,
  "domain": "example",
  "trace_id": "hafhjkqwjwefk_18bZZLTk5VfJGSy8Cylsu4",
  "is_sucess": true,
  "error_message": "",
  "estimated_duration": 0
}
  • text: Text content to be played
  • priority_level: Priority level, can be ignored
  • priority_weight: Priority weight, can be ignored
  • domain: Source identifier, returns the custom string passed in the request for troubleshooting
  • trace_id: Playback id, returns the custom string passed in the request plus a random string, used for playback status query
  • is_success: Whether priority check is successful, usually true. Does not guarantee playback; for example, an incorrect or non-existent file name will still return true. Only when there is higher priority content being played will it return true, e.g., during a fault alarm broadcast
  • error_message: Error message
  • estimated_duration: Invalid field, can be ignored, playback duration cannot be estimated
Example Scriptexamples/agent/tts_broadcast.sh
Remarks
  • Only supports short text requests, up to 1024 bytes, about 200 Chinese/English characters

7.2.3 Audio File Playback RPC Interface

Interface Namepb:/aimdk.protocol.TTSService/PlayMediaFile
Function OverviewAudio file playback
Interface TypeHTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.TTSService/PlayMediaFile
Input Parameters
text
{
  "file_name": "wake.pcm",
  "priority_level": "INTERACTION_L6",
  "domain": "example",
  "trace_id": "hafhjkqwjwefk",
  "is_interrupted": true
}
  • file_name: File name (supports relative or absolute path. If using relative path, defaults to reading from HDU /agibot/data/var/interaction/audio/, e.g., wake.pcm; if using absolute path, e.g., /agibot/data/home/agi/Desktop/wake.pcm, reads from the specified path. File must be 24kHz 16-bit mono PCM)
  • priority_level: Priority, keep as INTERACTION_L6
  • domain: Source identifier, can pass a custom client string for troubleshooting
  • trace_id: Playback id, required if you want to query playback status; pass this value as the parameter for status query
Output Parameters
text
{
  "text": "wake.pcm",
  "priority_level": "INTERACTION_L6",
  "priority_weight": 0,
  "domain": "example",
  "trace_id": "hafhjkqwjwefk",
  "is_sucess": true,
  "error_message": "",
  "estimated_duration": 0
}
  • text: File name
  • priority_level: Priority level, can be ignored
  • priority_weight: Priority weight, can be ignored
  • domain: Source identifier, returns the custom string passed in the request for troubleshooting
  • trace_id: Playback id, returns the custom string passed in the request, used for playback status or interruption
  • is_success: Whether priority check is successful, usually true. Does not guarantee playback; for example, an incorrect or non-existent file name will still return true. Only when there is higher priority content being played will it return true, e.g., during a fault alarm broadcast
  • error_message: Error message
  • estimated_duration: Invalid field, can be ignored, playback duration cannot be estimated
Example Scriptexamples/agent/play_media_file.sh
Remarks
  • Also supports standard 44-byte header Linear PCM wav files. Other header formats and compressed formats are not supported. 24kHz 16-bit mono PCM files are recommended.
  • If an incorrect or non-existent file name is provided, playback will fail silently and is_success will still return true.

7.2.4 TTS/Audio File Playback Status Query RPC Interface

Interface Namepb:/aimdk.protocol.TTSService/GetAudioStatus
Function OverviewTTS/Audio file playback status query
Interface TypeHTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.TTSService/GetAudioStatus
Input Parameters
text
{
  "trace_id": "hafhjkqwjwefk"
}
  • trace_id: Playback id, fill in the id used when calling PlayTTS or PlayMediaFile interface
Output Parameters
text
{
  "tts_status": {
    "text": "",
    "priority": 0,
    "trace_id": "",
    "tts_status": "TTSStatusType_Playing",
    "domain": "",
    "error_message": ""
  }
}
  • tts_status: Playback status, enum values
    • TTSStatusType_Unknown: Unknown status
    • TTSStatusType_Begin: Start playback, very brief, usually not seen in query
    • TTSStatusType_Playing: Playing
    • TTSStatusType_End: Playback ended, very brief, usually not seen in query
    • TTSStatusType_Stop: Paused/Cancelled/Interrupted
    • TTSStatusType_Error: Playback failed
    • TTSStatusType_InQue: In playback queue, not started yet
    • TTSStatusType_NOTInQue: Not in playback queue, not playing, will enter this state after playback ends
  • Other fields are invalid and can be ignored
Example Scriptexamples/agent/tts_status_rpc.sh
Remarks
  • Successful playback queried through this RPC interface will generally return InQue, Playing, and NOTInQue states

7.2.5 TTS/Audio File Playback Status Topic Interface

Interface Name/interaction/tts_status
Function OverviewTTS/Audio file playback status
Interface TypeROS2 Topic
Output Parameters
text
{
  "text": "Hello",
  "priority": 600,
  "trace_id": "hafhjkqwjwefk_5WC7uy69bqCuO101aalJ1T",
  "tts_status": "TTSStatusType_End",
  "domain": "example",
  "header": {
    "seq": 110
  }
}
  • text: File name
  • trace_id: Playback id
  • domain: Source
  • tts_status: Playback status, enum values
    • TTSStatusType_Unknown: Unknown status
    • TTSStatusType_Begin: Start playback, very brief, usually not seen in query
    • TTSStatusType_Playing: Playing
    • TTSStatusType_End: Playback ended, very brief, usually not seen in query
    • TTSStatusType_Stop: Paused/Cancelled/Interrupted
    • TTSStatusType_Error: Playback failed
    • TTSStatusType_InQue: In playback queue, not started yet
    • TTSStatusType_NOTInQue: Not in playback queue, not playing, will enter this state after playback ends
  • Other fields are invalid and can be ignored
Example Scriptexamples/agent/tts_status_topic.py
Remarks
  • The ROS2 type of this message is ros2_plugin_proto/msg/RosMsgWrapper. You need to source prebuilt/ros2_plugin_proto_aarch64/share/ros2_plugin_proto/local_setup.bash before using it.
  • This interface requires the interaction mode to be set to normal to publish. No messages will be published in only_voice mode.

7.2.6 TTS/Audio File Playback Interruption RPC Interface

Interface Namepb:/aimdk.protocol.TTSService/StopTTSTraceId
Function OverviewInterrupt single file/TTS playback
Interface TypeHTTP JSON RPC
URLhttp://10.42.10.10:59301/rpc/aimdk.protocol.TTSService/StopTTSTraceId
Input Parameters
text
{
  "trace_id": "hafhjkqwjwefk"
}
  • trace_id: Playback id, fill in the id used when calling PlayTTS or PlayMediaFile interface
Output Parameters
text
{
  "state":"CommonState_UNKNOWN"
}
  • state: Request status, specific value can be ignored. HTTP 200 means success.
Example Scriptexamples/agent/stop_tts.sh
Notes

7.2.7 Audio Volume Get/Set RPC Interface

Interface Namepb:/aimdk.protocol.HalAudioService/GetAudioVolume
Function OverviewGet current volume
Interface TypeHTTP JSON RPC
URLhttp://10.42.10.10:56666/rpc/aimdk.protocol.HalAudioService/GetAudioVolume
Input Parameters
text
{}
Output Parameters
text
{
  "header": {
    "code": "0",
    "msg": "GetAudioVolume successfully",
    "trace_id": "",
    "domin": ""
  },
  "audio_volume": 30,
  "is_mute": false,
  "type": "SPEAKRE_BUILT_IN"
}
  • audio_volume: Volume, value from 0-100
  • is_mute: Mute status
  • type: Speaker type
    • SPEAKRE_BUILT_IN: Built-in speaker
    • SPEAKER_BLUETOOTH: Bluetooth speaker
Example Scriptexamples/hal_audio/GetAudioVolume.sh
Remarks
Interface Namepb:/aimdk.protocol.HalAudioService/SetAudioVolume
Function OverviewSet volume
Interface TypeHTTP JSON RPC
URLhttp://10.42.10.10:56666/rpc/aimdk.protocol.HalAudioService/SetAudioVolume
Input Parameters
text
{
  "audio_volume": 70,
  "is_mute": false,
  "type": "SPEAKER_BUILT_IN"
}
  • audio_volume: Volume, value from 0-100. Do not set volume above 70. Exceeding this range may damage the speaker due to amplifier overdrive.
  • is_mute: Mute status
  • type: Speaker type
Output Parameters
text
{
  "header": {
    "code": "0",
    "msg": "SetAudioVolume successfully",
    "trace_id": "",
    "domin": ""
  },
  "pkg_name": "",
  "is_success": false
}
  • pkg_name: Invalid field, can be ignored
  • is_success: Invalid field, can be ignored
Example Scriptexamples/hal_audio/SetAudioVolume.sh
Remarks
  • To mute, set audio_volume to 0 and is_mute to true.

7.2.8 Request Audio Focus

Interface Name/audio_5Fmsgs/srv/RequestAudioFocus
Function OverviewRequest audio focus
Interface Typeservices
Input Parameters
text
{
    focus_requester: {
        pkg_name: audio_examples_sender, 
        priority: 6, 
        priority_weight: 1}
 }
  • pkg_name: ROS package name
  • priority: Priority (audio playback only uses 6)
  • priority_weight: Audio weight
Example Scriptexamples/agent/RequestAudioFocus.sh
RemarksBefore use, source the ROS environment: source prebuilt/audio_msgs_proto_aarch64/share/audio_msgs/local_setup.bash

7.2.9 Release Audio Focus

Interface Name/audio_5Fmsgs/srv/AbandonAudioFocus
Function OverviewRelease audio focus
Interface Typeservices
Input Parameters
text
{
    focus_requester: {
        pkg_name: audio_examples_sender, 
        priority: 6, 
        priority_weight: 1}
 }
  • pkg_name: ROS package name
  • priority: Priority (audio playback only uses 6)
  • priority_weight: Audio weight
Example Scriptexamples/agent/AbandonAudioFocus.sh
RemarksBefore use, source the ROS environment: source prebuilt/audio_msgs_proto_aarch64/share/audio_msgs/local_setup.bash

7.2.10 Play Streaming Audio

Interface Name/audiohal/audio/playback
Function OverviewPlay streaming audio
Interface Typetopic
Input Parameters
text
{
  stamps: <current time>   

focus_requester:
pkg_name: audio_examples_sender
priority: 6
priority_weight: 1
info:
channels: 1
sample_rate: 16000
sample_format: S16LE
coding_format: pcm
data:
audio_bytes: \x00\x01\x02\x03\x04\x05\x06\x07   # uint8 array (PCM data)
pkg_name: audio_examples_sender
token_id: token-0001
}
  • stamp: Timestamp
  • pkg_name: ROS package name
  • priority: Priority (audio playback only uses 6)
  • priority_weight: Audio weight
  • channels: Number of channels
  • sample_rate: Sample rate
  • sample_format: Sample format
  • coding_format: Data format
  • audio_bytes: Data stream
  • token_id: Optional, can be left as default
Example Scriptexamples/agent/audio_examples_sender
RemarksBefore use, source the ROS environment: source prebuilt/audio_msgs_proto_aarch64/share/audio_msgs/local_setup.bash