#
Large Model Game Interaction Access (iOS)
In games that support AI interaction, users can interact with large model AI players through voice or text. The premise for interaction is that there must be at least one real person in the game, and then the large model AI player can be added through the corresponding interface. Subsequent interactions can be conducted through the ISudAiAgent-related interfaces.
ISudAiAgent Interface
Interface Definition:
@protocol ISudAiAgent;
/// AI Agent
@protocol ISudAiAgent <NSObject>
/// Set room message listener
/// @param roomMsgListener roomMsgListener description
- (void)setOnRoomChatMessageListener:(void(^)(NSString *json))roomMsgListener;
/// The audio slice passed in is PCM data obtained from RTC
/// PCM data format must be: sample rate: 16000, bit depth: 16, number of channels: MONO
/// PCM data length can be adjusted according to the effect; longer length: better accuracy but longer delay; shorter length: shorter delay but sacrifices accuracy
/// @param data pcm data
- (void)pushAudio:(NSData *_Nonnull)data;
/// Pause pushing voice data, will return the previously pushed voice data after sentence completion
- (void)pauseAudio;
/// Call when no longer pushing voice data, used to release the current voice processing context
- (void)stopAudio;
/// Send text content
- (void)sendText:(NSString *_Nonnull)text;
@end
I. Obtaining ISudAiAgent
After loading the game, obtain ISudAiAgent through the ISudFSTAPP interface method getAiAgent. Once the ISudAiAgent interface is obtained, you can interact with the large model AI player via voice and text and set listeners to receive replies from the large model AI player.
Obtain the ISudAiAgent instance interface through the ISudFSTAPP game instance:
@protocol ISudFSTAPP <NSObject>
/// Create an AI agent operation instance
- (id<ISudAiAgent>)getAiAgent;
@end
Reference Example:
// AI agent
@property(nonatomic, strong) id<ISudAiAgent> aiAgent;
/// Game start
- (void)onGameStarted {
DDLogDebug(@"onGameStarted");
[self.vc handleGameStarted];
/// Create AI large model interaction
[self createAiAgent:self.sudFSTAPPDecorator.iSudFSTAPP];
}
- (void)createAiAgent:(id<ISudFSTAPP>)iSudFSTAPP {
// Zhenhun Stone and Flying Chess
if (self.loadConfigModel.gameId == 1890346721291059202L || self.loadConfigModel.gameId == 1468180338417074177L) {
self.aiAgent = [iSudFSTAPP getAiAgent];
WeakSelf
/// Listen for AI large model interaction information
[self.aiAgent setOnRoomChatMessageListener:^(NSString * _Nonnull json) {
[weakSelf handleAiRoomChatMsg:json];
}];
}
}
II. Adding Large Model AI Players
Interface definition document: General Status notifyStateChange
52. Set large model AI players in the game
app_common_game_add_big_scale_model_ai_players
When there is at least one real person in the game, the client can send a message to the game in the protocol format defined by this interface to add large model AI players. Correspondingly, you can use the interaction interface provided by the SudGIPWrapper module to add the AI large model robot to the game, as follows:
/// Set large model AI players in the game APP_COMMON_GAME_ADD_BIG_SCALE_MODEL_AI_PLAYERS
/// @param appCommonGameAddAiPlayersModel configuration information
- (void)notifyAppCommonGameAddBigScaleModelAPlayers:(AppCommonGameAddBigScaleModelAiPlayersModel *)appCommonGameAddAiPlayersModel;
Reference Example:
// Add large model AI
NSMutableArray *aiPlayers = [[NSMutableArray alloc] init];
BigScaleModelAiPlayerInfoModel *aiPlayerInfoModel = [BigScaleModelAiPlayerInfoModel alloc];
aiPlayerInfoModel.userId = [NSString stringWithFormat:@"%@", @(robotInfoModel.userId)];
aiPlayerInfoModel.name = robotInfoModel.name;
aiPlayerInfoModel.avatar = robotInfoModel.avatar;
aiPlayerInfoModel.gender = robotInfoModel.gender;
aiPlayerInfoModel.aiIdStr = @"1";
[aiPlayers addObject:aiPlayerInfoModel];
AppCommonGameAddBigScaleModelAiPlayersModel *appCommonGameAddAiPlayersModel = [[AppCommonGameAddBigScaleModelAiPlayersModel alloc] init];
appCommonGameAddAiPlayersModel.aiPlayers = aiPlayers;
appCommonGameAddAiPlayersModel.isReady = YES;
[weakSelf.gameEventHandler.sudFSTAPPDecorator notifyAppCommonGameAddBigScaleModelAPlayers:appCommonGameAddAiPlayersModel];
III. Text Dialogue
The app can choose to send text messages to the game to interact with the large model AI player. Use the ISudAiAgent interface method sendText to send text content to the game.
Interface:
/// Send text content
- (void)sendText:(NSString *_Nonnull)text;
Reference Example:
- (void)sendTextToAiAgent:(NSString *)text {
if (self.aiAgent) {
[self.aiAgent sendText:text];
}
}
IV. Voice Dialogue
The app can choose to send voice messages to the game to interact with the large model AI player. Use the ISudAiAgent interface method pushAudio to send voice messages to the game. pauseAudio is used to pause voice dialogue, and stopAudio is used to stop voice dialogue.
Interface:
/// The audio slice passed in is PCM data obtained from RTC
/// PCM data format must be: sample rate: 16000, bit depth: 16, number of channels: MONO
/// PCM data length can be adjusted according to the effect; longer length: better accuracy but longer delay; shorter length: shorter delay but sacrifices accuracy
/// @param data pcm data
- (void)pushAudio:(NSData *_Nonnull)data;
/// Pause pushing voice data, will return the previously pushed voice data after sentence completion
- (void)pauseAudio;
/// Call when no longer pushing voice data, used to release the current voice processing context
- (void)stopAudio;
Reference Example:
/// Push voice data
- (void)pushAudioToAiAgent:(NSData *)pcmData {
if (self.aiAgent) {
[self.aiAgent pushAudio:pcmData];
}
}
/// Pause pushing voice data
- (void)pauseAudioToAiAgent {
if (!self.aiAgent) {
return;
}
[self.aiAgent pauseAudio];
}
V. Receiving Messages from Large Model AI Players
When the app sends messages to the large model AI player through voice or text, the large model AI player will reply with one or more messages to the players in the room. Use the ISudAiAgent interface method setOnRoomChatMessageListener to set a listener for the replies from the large model AI player.
Callback Interface:
/// Set room message listener
/// @param roomMsgListener roomMsgListener description
- (void)setOnRoomChatMessageListener:(void(^)(NSString *json))roomMsgListener;
Sample Callback Message Format:
{
"uid": "123456",
"audioData": "UklGRhQAAABXQVZFZm10IBAAAAA",
"content": "Hello"
}
Reference Example:
/// Large model interaction callback JSON data structure
@interface AiRoomChatMsgModel : NSObject
/// Sender ID
@property(nonatomic, strong)NSString *uid;
/// Base64 voice data (mp3)
@property(nonatomic, strong)NSString *audioData;
/// Text content
@property(nonatomic, strong)NSString *content;
@end
/// Handle large model callback information
/// - Parameter json: json string data
- (void)handleAiRoomChatMsg:(NSString *)json {
WeakSelf
AiRoomChatMsgModel *aiRoomChatMsgModel = [AiRoomChatMsgModel mj_objectWithKeyValues:json];
NSString *audioDataBase64 = aiRoomChatMsgModel.audioData; // infoDic[@"audioData"];
NSString *playerId = aiRoomChatMsgModel.uid; // infoDic[@"userId"];
if (audioDataBase64) {
NSData *audioData = [[NSData alloc] initWithBase64EncodedString:audioDataBase64 options:0];
BOOL isPlayByRtc = YES;
if (isPlayByRtc) {
id audioEngine = AudioEngineFactory.shared.audioEngine;
// If RTC supports local playback, use it
if ([audioEngine respondsToSelector:@selector(playLocalAudio:)]) {
SudRtcAudioItem *audioItem = [[SudRtcAudioItem alloc] init];
audioItem.audioData = audioData;
audioItem.extra = playerId;
audioItem.playStateChangedBlock = ^(SudRtcAudioItem *item, SudRtcAudioItemPlayerState playerState) {
[weakSelf handleUserPlayerAudioState:item.extra state:playerState];
};
[audioEngine playLocalAudio:audioItem];
return;
}
}
// Create player to play
SudAudioItem *audioItem = [[SudAudioItem alloc] init];
audioItem.audioData = audioData;
audioItem.extra = playerId;
audioItem.playStateChangedBlock = ^(SudAudioItem *item, SudAudioItemPlayerState playerState) {
[weakSelf handleUserPlayerAudioState:item.extra state:playerState];
};
[SudAudioPlayer.shared playAudioMulti:audioItem];
}
}
When the onRoomChatMessage callback is received, the large model AI player's speaking content can be displayed as text messages or played as voice messages according to the app's product needs.
VI. Voice Wave Control
When the app receives the following game message, it indicates that the app can choose to control the display of the corresponding player's voice wave effect in the game.
Interface definition document: General Status - Game
69. Notify the app that it can start pushing the microphone speaking status
mg_common_game_player_mic_state
Corresponding SudGIPWrapper callback interface:
/// Notify the APP that the player's microphone state is ready OK MG_COMMON_GAME_PLAYER_MIC_STATE
- (void)onGameMgCommonGamePlayerMicState:(nonnull id<ISudFSMStateHandle>)handle model:(MgCommonGamePlayerMicState *)model;
After receiving the instruction, you can control it through the following interface.
Interface definition document: General Status notifyStateChange
51. App sends player mic state to the game
app_common_game_player_mic_state
Corresponding SudGIPWrapper call interface:
/// App notifies the game to update microphone state APP_COMMON_GAME_PLAYER_MIC_STATE
- (void)notifyAppCommonGamePlayerMicState:(AppCommonGamePlayerMicState *)model;
This is used to control whether to display the specified player's voice wave in the game, and it is a toggle-type instruction; it only needs to be sent once when enabling or disabling.
Reference Example: ```objc
- (void)onGameMgCommonGamePlayerMicState:(id
)handle model:(MgCommonGamePlayerMicState )model { self.isGamePlayerMicStateOk = YES; for (NSString key in self.userAudioPlayStateMap.allKeys) {
} } /// Send play state to the gameNSInteger state = [self.userAudioPlayStateMap[key] integerValue]; [self sendGamePlayerAudioState:key state:state]; - (void)sendGamePlayerAudioState:(NSString *)userId state:(NSInteger)state {
// If AI is not enabled or the game has not notified that the player's microphone is ready, do not send
if (!self.isOpenAiAgent || !self.isGamePlayerMicStateOk) {
} AppCommonGamePlayerMicState *stateModel = AppCommonGamePlayerMicState.new; stateModel.state = SudAudioItemPlayerStatePlaying == state ? 1 : 0; stateModel.uid = userId; [self.sudFSTAPPDecorator notifyAppCommonGamePlayerMicState:stateModel]; }return;