Merge pull request 'feature/agent' (#1) from feature/agent into master

Reviewed-on: http://git.lzwcai.com:3000/tanjianbin/lzwcai-terminal-temi/pulls/1

README.md

@@ -1,114 +1,25 @@
 # Temi 终端控制应用
 
-本项目是基于 Temi SDK 的 Android 应用，提供 MQTT 指令控制、导航/巡逻/接待流程，以及设置页管理网络 IP、当前位置与特殊任务模式。
+基于 Temi SDK 的 Android 应用，通过 MQTT 指令控制机器人导航、接待、巡逻与语音播报。
 
-## 1. 运行环境
+## 功能
+- MQTT 指令控制导航与播报
+- 接待与巡逻模式
+- 设置页：网络 IP、当前位置、特殊任务模式
 
-### 推荐工具：Android Studio
+## 运行
-应用依赖 Gradle 与 Android 运行环境，无法通过浏览器直接预览。请使用 Android Studio 导入项目并运行到 Temi 机器人或模拟器。
+- 推荐使用 Android Studio 连接 Temi 设备运行
+- 模拟器仅用于 UI 预览
 
-### 运行方式
+## MQTT
-- 方式 A：连接 Temi 机器人（推荐）
+- Broker：`tcp://<IP>:1883`
-  - 确保 Temi 已开启开发者模式
+- 主题：`robot/cmd`
-  - USB 连接或 ADB over Wi-Fi
+- 示例：
-  - Android Studio 选择设备后 Run
-- 方式 B：Android 模拟器（仅 UI 预览）
-  - Temi 系统服务缺失，Robot SDK 可能报错或不可用
-
----
-
-## 2. 功能概览
-
-- 主界面表情与状态反馈（AnimatedEmojiView）
-- MQTT 控制机器人行为与 TTS 播报
-- 接待模式：到达指定地点后检测到人出现确认按钮
-- 巡逻模式：按指定或随机地点巡逻
-- 特殊任务模式开关
-- 设置页：网络 IP、当前位置、重启应用
-
----
-
-## 3. 页面说明
-
-### 主界面（MainActivity）
-- 顶部设置按钮进入设置页
-- 表情组件根据 TTS 和任务状态切换表情
-- 接待模式触发时显示 “是的” 按钮，点击后前往目标地点
-
-### 设置页（SettingsActivity）
-- 网络 IP 配置：保存后作为 MQTT Broker 地址
-- 当前位置下拉选择：来源于 Temi 已保存地点
-- 特殊任务模式开关与指示灯
-- 长按 3 秒重启应用
-- 版本号显示（当前为硬编码字符串）
-
----
-
-## 4. MQTT 指令协议
-
-应用连接 `tcp://<IP>:1883`，订阅主题 `robot/cmd`，仅处理 JSON payload。
-
-### 基础指令
-- `recharge`：回充电桩
-- `goto`：前往地点（字段 `location` 或 `target`）
-- `repose`：重新定位
-- `stop`：停止移动并暂停 TTS（保留队列与 stream buffer）
-- `continue`：继续播报（优先重播被中断语句）
-- `terminate`：停止移动并清空所有 TTS 队列与 buffer
-
-### 播报指令
-- `speak`：立即播报（字段 `text` 或 `speech`，可选 `lang`）
-- `stream`：流式播报（字段 `text` 或 `content`，可选 `lang`）
-  - 按 `。！？!?\n` 分句进入队列
-
-### 任务指令
-- `patrol`：巡逻
-  - `flag=true` 随机抽取 3~6 个地点（排除 home base）
-  - `flag=false` + `locations` 指定地点列表
-- `reception`：接待
-  - `location` 接待地点（默认 “前台”）
-  - `text` 询问语（默认 “你是我要接待的贵宾吗？”）
-  - `destination` 目的地（默认 “会议室”）
-
-### 示例
 ```json
 {"action":"goto","location":"前台"}
 ```
 
-```json
+## 构建
-{"action":"speak","text":"欢迎光临","lang":"zh"}
-```
-
-```json
-{"action":"patrol","flag":false,"locations":["前台","会议室","大厅"]}
-```
-
----
-
-## 5. 特殊任务模式
-
-设置页开关 `special_task_mode` 控制以下行为：
-- 跳过 Home Base 的开门/关门语音逻辑
-- 跳过检测到人时的问候语
-- 机器人到达地点时，不播报“已到达”提示（仅在当前无任务时生效）
-
----
-
-## 6. 项目结构
-
-- `MainActivity.kt`：主逻辑与事件监听
-- `SettingsActivity.kt`：设置页逻辑
-- `MqttManager.kt`：MQTT 连接与指令解析
-- `NavController.kt`：导航与巡逻封装
-- `AnimatedEmojiView.kt`：表情动画绘制
-- `LogManager.kt`：Logcat 监听分发
-- `HttpManager.kt`：HTTP Workflow 请求封装（目前未在流程中启用）
-- `activity_main.xml` / `activity_settings.xml`：主界面与设置页布局
-
----
-
-## 7. 本地构建与安装
-
 ```bash
 .\gradlew.bat :app:installDebug
 ```

app/build.gradle.kts

@@ -44,9 +44,10 @@ dependencies {
     implementation(libs.androidx.appcompat)
     implementation(libs.material)
     implementation(libs.androidx.emoji2.views)
+    implementation(libs.livekit.android)
     implementation("org.eclipse.paho:org.eclipse.paho.android.service:1.1.1")
     implementation("org.eclipse.paho:org.eclipse.paho.client.mqttv3:1.2.5")
     testImplementation(libs.junit)
     androidTestImplementation(libs.androidx.junit)
     androidTestImplementation(libs.androidx.espresso.core)
-}
+}

app/src/main/AndroidManifest.xml

@@ -5,6 +5,9 @@
     <uses-permission android:name="android.permission.INTERNET" />
     <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
     <uses-permission android:name="android.permission.WAKE_LOCK" />
+    <uses-permission android:name="android.permission.RECORD_AUDIO" />
+    <uses-permission android:name="android.permission.CAMERA" />
+    <uses-feature android:name="android.hardware.camera" android:required="false" />
 
     <application
         android:allowBackup="true"

app/src/main/java/com/example/lzwcai_terminal_temi/LiveKitManager.kt

@@ -0,0 +1,84 @@
+package com.example.lzwcai_terminal_temi
+
+import android.content.Context
+import android.util.Log
+import io.livekit.android.ConnectOptions
+import io.livekit.android.LiveKit
+import io.livekit.android.events.RoomEvent
+import io.livekit.android.events.collect
+import io.livekit.android.room.Room
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.SupervisorJob
+import kotlinx.coroutines.cancel
+import kotlinx.coroutines.launch
+
+enum class LiveKitStatus {
+    Connected,
+    Disconnected,
+    Failed
+}
+
+class LiveKitManager(appContext: Context, private val statusListener: (LiveKitStatus) -> Unit) {
+
+    private val context = appContext.applicationContext
+    private val scope = CoroutineScope(Dispatchers.Main + SupervisorJob())
+    private var room: Room? = null
+    private var eventsJob: Job? = null
+
+    fun connect(url: String, token: String, enableMic: Boolean, enableCamera: Boolean) {
+        val finalUrl = url.trim()
+        val finalToken = token.trim()
+        if (finalUrl.isEmpty() || finalToken.isEmpty()) {
+            Log.w("LiveKitManager", "LiveKit connect skipped: empty url or token.")
+            return
+        }
+        scope.launch {
+            val currentRoom = room ?: LiveKit.create(context).also { room = it }
+            eventsJob?.cancel()
+            eventsJob = launch {
+                currentRoom.events.collect { event ->
+                    when (event) {
+                        is RoomEvent.Connected -> {
+                            Log.i("LiveKitManager", "LiveKit connected.")
+                            statusListener(LiveKitStatus.Connected)
+                        }
+                        is RoomEvent.Disconnected -> {
+                            Log.i("LiveKitManager", "LiveKit disconnected.")
+                            statusListener(LiveKitStatus.Disconnected)
+                        }
+                        else -> {}
+                    }
+                }
+            }
+            runCatching {
+                val options = ConnectOptions(
+                    autoSubscribe = false,
+                    audio = enableMic,
+                    video = enableCamera
+                )
+                currentRoom.connect(finalUrl, finalToken, options)
+            }.onFailure { e ->
+                Log.e("LiveKitManager", "LiveKit connect error: ${e.message}", e)
+                statusListener(LiveKitStatus.Failed)
+            }
+        }
+    }
+
+    fun disconnect() {
+        scope.launch {
+            eventsJob?.cancel()
+            runCatching { room?.disconnect() }
+            statusListener(LiveKitStatus.Disconnected)
+        }
+    }
+
+    fun release() {
+        eventsJob?.cancel()
+        runCatching { room?.disconnect() }
+        room = null
+        scope.cancel()
+        statusListener(LiveKitStatus.Disconnected)
+    }
+}

app/src/main/java/com/example/lzwcai_terminal_temi/MainActivity.kt

@@ -1,13 +1,20 @@
 package com.example.lzwcai_terminal_temi
 
+import android.Manifest
 import android.annotation.SuppressLint
 import android.content.Context
 import android.content.Intent
 import android.content.SharedPreferences
+import android.content.pm.PackageManager
 import android.os.Bundle
+import android.os.Build
 import android.util.Log
 import android.view.WindowManager
+import android.util.Base64
+import android.graphics.drawable.GradientDrawable
 import androidx.appcompat.app.AppCompatActivity
+import androidx.core.app.ActivityCompat
+import androidx.core.content.ContextCompat
 import com.example.lzwcai_terminal_temi.databinding.ActivityMainBinding
 import com.robotemi.sdk.Robot
 import com.robotemi.sdk.TtsRequest
@@ -29,6 +36,10 @@ import kotlinx.coroutines.Job
 import kotlinx.coroutines.delay
 import kotlinx.coroutines.isActive
 import kotlin.random.Random
+import org.json.JSONObject
+import java.nio.charset.StandardCharsets
+import javax.crypto.Mac
+import javax.crypto.spec.SecretKeySpec
 
 class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnGoToLocationStatusChangedListener,
     OnDetectionStateChangedListener, OnReposeStatusChangedListener, SharedPreferences.OnSharedPreferenceChangeListener,
@@ -41,6 +52,18 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
     private lateinit var prefs: SharedPreferences
     private lateinit var navCon: NavController
     private lateinit var permissionManager: PermissionManager
+    private var liveKitManager: LiveKitManager? = null
+    private val liveKitUrlKey = "livekit_url"
+    private val liveKitRoomKey = "livekit_room"
+    private val liveKitTokenKey = "livekit_token"
+    private val liveKitEnabledKey = "livekit_enabled"
+    private val liveKitPermissionRequestCode = 2001
+    private val liveKitUrlDefault = "ws://192.168.2.236:7880"
+    private val liveKitApiKeyDefault = "devkey"
+    private val liveKitApiSecretDefault = "secret"
+    private val liveKitRoomDefault = "temi-room"
+    private var isLiveKitConnected = false
+    private var isMqttConnected = false
 
     private var lastArrivalLocation: String? = null
     private var lastArrivalAt: Long = 0L
@@ -87,6 +110,13 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
 
         prefs = getSharedPreferences("app_prefs", Context.MODE_PRIVATE)
         prefs.registerOnSharedPreferenceChangeListener(this)
+        liveKitManager = LiveKitManager(applicationContext) { status ->
+            when (status) {
+                LiveKitStatus.Connected -> setLiveKitStatus(true)
+                LiveKitStatus.Disconnected -> setLiveKitStatus(false)
+                LiveKitStatus.Failed -> setLiveKitStatus(false)
+            }
+        }
         if (lastArrivalLocation == null) {
             lastArrivalLocation = prefs.getString("current_location", null)
         }
@@ -109,6 +139,7 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
         }
         
         updateMqttConnection()
+        updateLiveKitStatusSnapshot()
     }
 
     override fun onStart() {
@@ -125,6 +156,7 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
         } else {
             mqttManager?.connect()
         }
+        updateLiveKitConnection()
         startBlinking()
     }
 
@@ -137,6 +169,7 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
         robot.removeOnReposeStatusChangedListener(this)
         robot.removeOnRequestPermissionResultListener(this)
         // mqttManager?.disconnect()  // Keep MQTT alive in background/settings
+        liveKitManager?.disconnect()
         stopBlinking()
     }
 
@@ -144,6 +177,7 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
         super.onDestroy()
         prefs.unregisterOnSharedPreferenceChangeListener(this)
         mqttManager?.disconnect()
+        liveKitManager?.release()
         LogManager.stopLogcatListener()
         mainScope.cancel()
         Log.i("MainActivity", "All resources released on destroy.")
@@ -455,6 +489,7 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
         if (key == "network_ip") {
             Log.i("MainActivity", "IP address changed, re-initializing MQTT connection.")
             updateMqttConnection()
+            updateLiveKitConnection()
         }
         if (key == "current_location") {
             lastArrivalLocation = sharedPreferences?.getString("current_location", null)
@@ -464,6 +499,9 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
             val isSpecial = sharedPreferences?.getBoolean("special_task_mode", false) == true
             Log.i("MainActivity", "Special mode pref changed: $isSpecial, currentTask: $currentTask")
         }
+        if (key == liveKitUrlKey || key == liveKitRoomKey || key == liveKitTokenKey || key == liveKitEnabledKey) {
+            updateLiveKitConnection()
+        }
     }
 
     private fun isSpecialModeEnabled(): Boolean {
@@ -474,15 +512,183 @@ class MainActivity : AppCompatActivity(), OnRobotReadyListener, TtsListener, OnG
         mqttManager?.shutdown()
         val ip = prefs.getString("network_ip", null)
         if (!ip.isNullOrEmpty()) {
-            mqttManager = MqttManager(this, ip, robot, navCon)
+            mqttManager = MqttManager(this, ip, robot, navCon) { connected ->
+                setMqttConnectionStatus(connected)
+            }
             mqttManager?.connect()
             Log.i("MainActivity", "MQTT Manager updated with new IP: $ip")
         } else {
             mqttManager = null
+            setMqttConnectionStatus(false)
             Log.w("MainActivity", "MQTT Manager disabled: IP address is not set.")
         }
     }
 
+    private fun updateLiveKitConnection() {
+        val enabled = prefs.getBoolean(liveKitEnabledKey, true)
+        val url = resolveLiveKitUrl()
+        val room = prefs.getString(liveKitRoomKey, liveKitRoomDefault).orEmpty()
+        val savedToken = prefs.getString(liveKitTokenKey, "").orEmpty()
+        val token = if (savedToken.isBlank()) {
+            buildLiveKitToken(
+                apiKey = liveKitApiKeyDefault,
+                apiSecret = liveKitApiSecretDefault,
+                room = room,
+                identity = buildLiveKitIdentity()
+            )
+        } else {
+            savedToken
+        }
+        if (!enabled) {
+            liveKitManager?.disconnect()
+            setLiveKitStatus(false)
+            return
+        }
+        if (url.isBlank() || room.isBlank()) {
+            liveKitManager?.disconnect()
+            setLiveKitStatus(false)
+            return
+        }
+        if (!hasAudioPermission() || !hasCameraPermission()) {
+            setLiveKitStatus(false)
+            requestMediaPermissions()
+            return
+        }
+        setLiveKitStatus(false)
+        liveKitManager?.connect(url, token, true, true)
+    }
+
+    private fun hasAudioPermission(): Boolean {
+        return ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED
+    }
+
+    private fun hasCameraPermission(): Boolean {
+        return ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
+    }
+
+    private fun requestMediaPermissions() {
+        ActivityCompat.requestPermissions(
+            this,
+            arrayOf(Manifest.permission.RECORD_AUDIO, Manifest.permission.CAMERA),
+            liveKitPermissionRequestCode
+        )
+    }
+
+    override fun onRequestPermissionsResult(
+        requestCode: Int,
+        permissions: Array<out String>,
+        grantResults: IntArray
+    ) {
+        super.onRequestPermissionsResult(requestCode, permissions, grantResults)
+        if (requestCode == liveKitPermissionRequestCode) {
+            val granted = grantResults.isNotEmpty() && grantResults.all { it == PackageManager.PERMISSION_GRANTED }
+            if (granted) {
+                updateLiveKitConnection()
+            } else {
+                Log.w("MainActivity", "LiveKit media permission denied.")
+            }
+        }
+    }
+
+    private fun buildLiveKitIdentity(): String {
+        val model = Build.MODEL?.trim().orEmpty()
+        val normalized = model.replace("\\s+".toRegex(), "-").lowercase()
+        val suffix = if (normalized.isNotEmpty()) normalized else "temi"
+        return "temi-$suffix"
+    }
+
+    private fun buildLiveKitToken(
+        apiKey: String,
+        apiSecret: String,
+        room: String,
+        identity: String
+    ): String {
+        val nowSeconds = System.currentTimeMillis() / 1000
+        val header = JSONObject()
+            .put("alg", "HS256")
+            .put("typ", "JWT")
+        val grants = JSONObject()
+            .put("roomJoin", true)
+            .put("room", room)
+            .put("canPublish", true)
+            .put("canSubscribe", true)
+            .put("canPublishData", true)
+        val claims = JSONObject()
+            .put("iss", apiKey)
+            .put("sub", identity)
+            .put("nbf", nowSeconds)
+            .put("exp", nowSeconds + 3600)
+            .put("video", grants)
+        val headerB64 = base64Url(header.toString().toByteArray(StandardCharsets.UTF_8))
+        val claimsB64 = base64Url(claims.toString().toByteArray(StandardCharsets.UTF_8))
+        val signingInput = "$headerB64.$claimsB64"
+        val signature = hmacSha256(signingInput, apiSecret)
+        return "$signingInput.${base64Url(signature)}"
+    }
+
+    private fun hmacSha256(data: String, secret: String): ByteArray {
+        val mac = Mac.getInstance("HmacSHA256")
+        val keySpec = SecretKeySpec(secret.toByteArray(StandardCharsets.UTF_8), "HmacSHA256")
+        mac.init(keySpec)
+        return mac.doFinal(data.toByteArray(StandardCharsets.UTF_8))
+    }
+
+    private fun base64Url(input: ByteArray): String {
+        return Base64.encodeToString(input, Base64.URL_SAFE or Base64.NO_PADDING or Base64.NO_WRAP)
+    }
+
+    private fun updateLiveKitStatusSnapshot() {
+        val enabled = prefs.getBoolean(liveKitEnabledKey, true)
+        val url = resolveLiveKitUrl()
+        val room = prefs.getString(liveKitRoomKey, liveKitRoomDefault).orEmpty()
+        if (!enabled) {
+            setLiveKitStatus(false)
+            return
+        }
+        if (url.isBlank() || room.isBlank()) {
+            setLiveKitStatus(false)
+            return
+        }
+        if (!hasAudioPermission() || !hasCameraPermission()) {
+            setLiveKitStatus(false)
+            return
+        }
+        setLiveKitStatus(false)
+    }
+
+    private fun setLiveKitStatus(connected: Boolean) {
+        isLiveKitConnected = connected
+        updateConnectionIndicator()
+    }
+
+    private fun setMqttConnectionStatus(connected: Boolean) {
+        isMqttConnected = connected
+        updateConnectionIndicator()
+    }
+
+    private fun updateConnectionIndicator() {
+        val colorRes = when {
+            !isLiveKitConnected && !isMqttConnected -> android.R.color.holo_red_dark
+            !isLiveKitConnected && isMqttConnected -> android.R.color.holo_blue_light
+            isLiveKitConnected && !isMqttConnected -> android.R.color.holo_orange_light
+            else -> android.R.color.holo_green_light
+        }
+        val indicatorDrawable = binding.statusIndicator.background as GradientDrawable
+        indicatorDrawable.setColor(ContextCompat.getColor(this, colorRes))
+    }
+
+    private fun resolveLiveKitUrl(): String {
+        val savedUrl = prefs.getString(liveKitUrlKey, "").orEmpty().trim()
+        if (savedUrl.isNotEmpty()) {
+            return savedUrl
+        }
+        val ip = prefs.getString("network_ip", "").orEmpty().trim()
+        if (ip.isNotEmpty()) {
+            return "ws://$ip:7880"
+        }
+        return liveKitUrlDefault
+    }
+
     private fun startBlinking() {
         stopBlinking()
         blinkJob = mainScope.launch {

app/src/main/java/com/example/lzwcai_terminal_temi/MqttManager.kt

@@ -2,6 +2,8 @@ package com.example.lzwcai_terminal_temi
 
 import android.content.Context
 import android.util.Log
+import android.os.Handler
+import android.os.Looper
 import com.robotemi.sdk.Robot
 import com.robotemi.sdk.TtsRequest
 import kotlinx.coroutines.*
@@ -15,7 +17,8 @@ class MqttManager(
     private val context: Context,
     private val serverIp: String,
     private val robot: Robot,
-    private val navController: NavController
+    private val navController: NavController,
+    private val statusListener: (Boolean) -> Unit
 ) {
 
     private var mqttClient: MqttClient? = null
@@ -25,6 +28,8 @@ class MqttManager(
     private val job = SupervisorJob()
     private val scope = CoroutineScope(Dispatchers.IO + job)
     private var reconnectJob: Job? = null
+    private val prefs = context.getSharedPreferences("app_prefs", Context.MODE_PRIVATE)
+    private val agentDempIdKey = "agent_demp_id"
 
     // Streaming text buffer
     private val speechBuffer = StringBuilder()
@@ -39,6 +44,8 @@ class MqttManager(
     private var interruptedLanguage: TtsRequest.Language? = null
     private var lastStreamLangCode: String? = null
     private val ttsLanguageMap = mutableMapOf<TtsRequest, TtsRequest.Language>()
+    private var currentStreamSessionId: String? = null
+    private var currentStreamMessageId: String? = null
 
     init {
         try {
@@ -47,10 +54,13 @@ class MqttManager(
                 override fun connectComplete(reconnect: Boolean, serverURI: String?) {
                     Log.i(TAG, "MQTT connection complete. Reconnect: $reconnect")
                     subscribeTopic("robot/cmd")
+                    subscribeTopic("soul2user")
+                    updateConnectionStatus(true)
                 }
 
                 override fun connectionLost(cause: Throwable?) {
                     Log.e(TAG, "Connection lost: ${cause?.message}")
+                    updateConnectionStatus(false)
                     scheduleReconnect()
                 }
 
@@ -74,6 +84,7 @@ class MqttManager(
         scope.launch {
             if (mqttClient?.isConnected == true) {
                 Log.d(TAG, "MQTT client is already connected.")
+                updateConnectionStatus(true)
                 return@launch
             }
             Log.i(TAG, "Attempting to connect to MQTT broker at $brokerUri")
@@ -89,6 +100,7 @@ class MqttManager(
                 mqttClient?.connect(options)
             } catch (e: MqttException) {
                 Log.e(TAG, "Initial connection failed: ${e.message}")
+                updateConnectionStatus(false)
                 scheduleReconnect()
             }
         }
@@ -113,8 +125,10 @@ class MqttManager(
                     mqttClient?.disconnect()
                     Log.i(TAG, "Disconnected from MQTT broker.")
                 }
+                updateConnectionStatus(false)
             } catch (e: MqttException) {
                 Log.e(TAG, "Error disconnecting from MQTT broker: ${e.message}")
+                updateConnectionStatus(false)
             }
         }
     }
@@ -131,6 +145,13 @@ class MqttManager(
             Log.e(TAG, "Error shutting down MQTT client: ${e.message}")
         } finally {
             mqttClient = null
+            updateConnectionStatus(false)
+        }
+    }
+
+    private fun updateConnectionStatus(connected: Boolean) {
+        Handler(Looper.getMainLooper()).post {
+            statusListener(connected)
         }
     }
 
@@ -176,7 +197,11 @@ class MqttManager(
         }
         try {
             val obj = JSONObject(trimmed)
-            handleJsonCommand(obj)
+            if (topic == "soul2user") {
+                handleSoulStream(obj)
+            } else {
+                handleJsonCommand(obj)
+            }
         } catch (e: Exception) {
             Log.e(TAG, "Invalid JSON payload: $payload", e)
             val ttsRequest = TtsRequest.create("指令格式错误，请检查指令格式", false, language = TtsRequest.Language.ZH_CN)
@@ -184,6 +209,63 @@ class MqttManager(
         }
     }
 
+    private fun handleSoulStream(obj: JSONObject) {
+        val configuredDempId = prefs.getString(agentDempIdKey, "").orEmpty().trim()
+        val dempId = obj.optString("demp_id", "").trim()
+        if (configuredDempId.isNotEmpty() && configuredDempId != dempId) {
+            Log.i(TAG, "Stream ignored: demp_id mismatch ($dempId)")
+            return
+        }
+        val sessionId = obj.optString("session_id", "").trim()
+        val messageId = obj.optString("message_id", "").trim()
+        ensureStreamContext(sessionId, messageId)
+        val text = obj.optString("text", "")
+        val isFinal = obj.optBoolean("is_final", false)
+        val lang = obj.optString("lang", "").trim()
+        scope.launch(Dispatchers.Main) {
+            (context as? MainActivity)?.markSpeechTaskActive()
+        }
+        processStreamText(text, lang)
+        if (isFinal) {
+            flushStreamRemainder(lang)
+            currentStreamSessionId = null
+            currentStreamMessageId = null
+        }
+    }
+
+    private fun ensureStreamContext(sessionId: String, messageId: String) {
+        if (sessionId.isEmpty() && messageId.isEmpty()) {
+            return
+        }
+        val sessionChanged = currentStreamSessionId != null && sessionId.isNotEmpty() && sessionId != currentStreamSessionId
+        val messageChanged = currentStreamMessageId != null && messageId.isNotEmpty() && messageId != currentStreamMessageId
+        if (currentStreamSessionId == null && currentStreamMessageId == null) {
+            currentStreamSessionId = sessionId.ifEmpty { null }
+            currentStreamMessageId = messageId.ifEmpty { null }
+            return
+        }
+        if (sessionChanged || messageChanged) {
+            stopTts()
+            currentStreamSessionId = sessionId.ifEmpty { null }
+            currentStreamMessageId = messageId.ifEmpty { null }
+            return
+        }
+        if (currentStreamSessionId == null && sessionId.isNotEmpty()) {
+            currentStreamSessionId = sessionId
+        }
+        if (currentStreamMessageId == null && messageId.isNotEmpty()) {
+            currentStreamMessageId = messageId
+        }
+    }
+
+    private fun flushStreamRemainder(langCode: String?) {
+        val remaining = speechBuffer.toString().trim()
+        if (remaining.isNotEmpty()) {
+            speechBuffer.setLength(0)
+            speak(remaining, langCode)
+        }
+    }
+
     private fun handleJsonCommand(obj: JSONObject) {
         val action = obj.optString("action", obj.optString("cmd", obj.optString("type", ""))).lowercase()
         when (action) {

app/src/main/java/com/example/lzwcai_terminal_temi/SettingsActivity.kt

@@ -29,6 +29,13 @@ class SettingsActivity : AppCompatActivity() {
     private lateinit var locationAdapter: ArrayAdapter<String>
     private val currentLocationKey = "current_location"
     private lateinit var prefs: SharedPreferences
+    private val liveKitUrlKey = "livekit_url"
+    private val liveKitRoomKey = "livekit_room"
+    private val liveKitTokenKey = "livekit_token"
+    private val liveKitEnabledKey = "livekit_enabled"
+    private val agentDempIdKey = "agent_demp_id"
+    private val liveKitUrlDefault = "ws://192.168.2.236:7880"
+    private val liveKitRoomDefault = "temi-room"
 
     override fun onCreate(savedInstanceState: Bundle?) {
         super.onCreate(savedInstanceState)
@@ -41,9 +48,20 @@ class SettingsActivity : AppCompatActivity() {
         prefs = getSharedPreferences("app_prefs", Context.MODE_PRIVATE)
         val savedIp = prefs.getString("network_ip", "")
         binding.etIpAddress.setText(savedIp)
+        val savedDempId = prefs.getString(agentDempIdKey, "")
+        binding.etAgentDempId.setText(savedDempId)
+        val savedLiveKitUrl = prefs.getString(liveKitUrlKey, resolveLiveKitUrl())
+        val savedLiveKitRoom = prefs.getString(liveKitRoomKey, liveKitRoomDefault)
+        val savedLiveKitToken = prefs.getString(liveKitTokenKey, "")
+        val isLiveKitEnabled = prefs.getBoolean(liveKitEnabledKey, true)
+        binding.etLiveKitUrl.setText(savedLiveKitUrl)
+        binding.etLiveKitRoom.setText(savedLiveKitRoom)
+        binding.etLiveKitToken.setText(savedLiveKitToken)
+        binding.switchLiveKitAuto.setOnCheckedChangeListener(null)
+        binding.switchLiveKitAuto.isChecked = isLiveKitEnabled
 
         // Set Version Name
-        val versionName = "2603121722"
+        val versionName = "2603131822"
         binding.tvVersion.text = getString(R.string.version_prefix, versionName)
 
         binding.root.setOnClickListener { hideKeyboard() }
@@ -51,8 +69,12 @@ class SettingsActivity : AppCompatActivity() {
         binding.btnSave.setOnClickListener {
             hideKeyboard()
             val ip = binding.etIpAddress.text.toString().trim()
+            val dempId = binding.etAgentDempId.text?.toString()?.trim().orEmpty()
             if (ip.isNotEmpty()) {
-                prefs.edit().putString("network_ip", ip).apply()
+                prefs.edit()
+                    .putString("network_ip", ip)
+                    .putString(agentDempIdKey, dempId)
+                    .apply()
                 Toast.makeText(this, getString(R.string.msg_ip_saved, ip), Toast.LENGTH_SHORT).show()
                 finish()
             } else {
@@ -61,6 +83,30 @@ class SettingsActivity : AppCompatActivity() {
             }
         }
 
+        binding.switchLiveKitAuto.setOnCheckedChangeListener { _, isChecked ->
+            prefs.edit().putBoolean(liveKitEnabledKey, isChecked).apply()
+        }
+
+        binding.btnLiveKitSave.setOnClickListener {
+            hideKeyboard()
+            val url = binding.etLiveKitUrl.text?.toString()?.trim().orEmpty()
+            val room = binding.etLiveKitRoom.text?.toString()?.trim().orEmpty()
+            val token = binding.etLiveKitToken.text?.toString()?.trim().orEmpty()
+            val enabled = binding.switchLiveKitAuto.isChecked
+            prefs.edit()
+                .putString(liveKitUrlKey, url)
+                .putString(liveKitRoomKey, room)
+                .putString(liveKitTokenKey, token)
+                .putBoolean(liveKitEnabledKey, enabled)
+                .apply()
+            if (url.isBlank() || room.isBlank()) {
+                Toast.makeText(this, getString(R.string.msg_livekit_cleared), Toast.LENGTH_SHORT).show()
+            } else {
+                Toast.makeText(this, getString(R.string.msg_livekit_saved), Toast.LENGTH_SHORT).show()
+            }
+            finish()
+        }
+
         binding.btnBack.setOnClickListener {
             hideKeyboard()
             finish()
@@ -195,4 +241,16 @@ class SettingsActivity : AppCompatActivity() {
             imm.hideSoftInputFromWindow(it.windowToken, 0)
         }
     }
+
+    private fun resolveLiveKitUrl(): String {
+        val savedUrl = prefs.getString(liveKitUrlKey, "").orEmpty().trim()
+        if (savedUrl.isNotEmpty()) {
+            return savedUrl
+        }
+        val ip = prefs.getString("network_ip", "").orEmpty().trim()
+        if (ip.isNotEmpty()) {
+            return "ws://$ip:7880"
+        }
+        return liveKitUrlDefault
+    }
 }

app/src/main/res/layout-land/activity_main.xml

@@ -20,6 +20,16 @@
         app:layout_constraintEnd_toEndOf="parent"
         app:layout_constraintTop_toTopOf="parent" />
 
+    <View
+        android:id="@+id/statusIndicator"
+        android:layout_width="16dp"
+        android:layout_height="16dp"
+        android:layout_marginStart="24dp"
+        android:layout_marginTop="24dp"
+        android:background="@drawable/status_indicator"
+        app:layout_constraintStart_toStartOf="parent"
+        app:layout_constraintTop_toTopOf="parent" />
+
     <com.example.lzwcai_terminal_temi.AnimatedEmojiView
         android:id="@+id/animatedEmojiView"
         android:layout_width="0dp"
@@ -49,4 +59,4 @@
         app:layout_constraintEnd_toEndOf="parent"
         app:layout_constraintStart_toStartOf="parent" />
 
-</androidx.constraintlayout.widget.ConstraintLayout>
+</androidx.constraintlayout.widget.ConstraintLayout>

app/src/main/res/layout-land/activity_settings.xml

@@ -102,6 +102,22 @@
                                 android:textColor="@color/text_primary" />
                         </com.google.android.material.textfield.TextInputLayout>
 
+                        <com.google.android.material.textfield.TextInputLayout
+                            style="@style/Widget.App.TextInputLayout"
+                            android:layout_width="match_parent"
+                            android:layout_height="wrap_content"
+                            android:layout_marginTop="12dp"
+                            android:hint="@string/hint_agent_id">
+
+                            <com.google.android.material.textfield.TextInputEditText
+                                android:id="@+id/etAgentDempId"
+                                android:layout_width="match_parent"
+                                android:layout_height="72dp"
+                                android:inputType="text"
+                                android:textSize="18sp"
+                                android:textColor="@color/text_primary" />
+                        </com.google.android.material.textfield.TextInputLayout>
+
                         <Button
                             android:id="@+id/btnSave"
                             style="@style/Widget.App.Button"
@@ -114,6 +130,109 @@
                     </LinearLayout>
                 </androidx.cardview.widget.CardView>
 
+                <androidx.cardview.widget.CardView
+                    style="@style/CardView.App"
+                    android:layout_width="match_parent"
+                    android:layout_height="wrap_content"
+                    android:layout_marginBottom="24dp">
+
+                    <LinearLayout
+                        android:layout_width="match_parent"
+                        android:layout_height="wrap_content"
+                        android:orientation="vertical"
+                        android:padding="24dp">
+
+                        <TextView
+                            android:layout_width="wrap_content"
+                            android:layout_height="wrap_content"
+                            android:layout_marginBottom="16dp"
+                            android:text="@string/label_livekit_config"
+                            android:textColor="@color/text_primary"
+                            android:textSize="22sp"
+                            android:textStyle="bold" />
+
+                        <com.google.android.material.textfield.TextInputLayout
+                            style="@style/Widget.App.TextInputLayout"
+                            android:layout_width="match_parent"
+                            android:layout_height="wrap_content"
+                            android:hint="@string/hint_livekit_url">
+
+                            <com.google.android.material.textfield.TextInputEditText
+                                android:id="@+id/etLiveKitUrl"
+                                android:layout_width="match_parent"
+                                android:layout_height="72dp"
+                                android:inputType="textUri"
+                                android:textSize="18sp"
+                                android:textColor="@color/text_primary" />
+                        </com.google.android.material.textfield.TextInputLayout>
+
+                        <com.google.android.material.textfield.TextInputLayout
+                            style="@style/Widget.App.TextInputLayout"
+                            android:layout_width="match_parent"
+                            android:layout_height="wrap_content"
+                            android:layout_marginTop="12dp"
+                            android:hint="@string/hint_livekit_room">
+
+                            <com.google.android.material.textfield.TextInputEditText
+                                android:id="@+id/etLiveKitRoom"
+                                android:layout_width="match_parent"
+                                android:layout_height="72dp"
+                                android:inputType="text"
+                                android:textSize="18sp"
+                                android:textColor="@color/text_primary" />
+                        </com.google.android.material.textfield.TextInputLayout>
+
+                        <com.google.android.material.textfield.TextInputLayout
+                            style="@style/Widget.App.TextInputLayout"
+                            android:layout_width="match_parent"
+                            android:layout_height="wrap_content"
+                            android:layout_marginTop="12dp"
+                            android:hint="@string/hint_livekit_token">
+
+                            <com.google.android.material.textfield.TextInputEditText
+                                android:id="@+id/etLiveKitToken"
+                                android:layout_width="match_parent"
+                                android:layout_height="72dp"
+                                android:inputType="text"
+                                android:textSize="18sp"
+                                android:textColor="@color/text_primary" />
+                        </com.google.android.material.textfield.TextInputLayout>
+
+                        <LinearLayout
+                            android:layout_width="match_parent"
+                            android:layout_height="wrap_content"
+                            android:layout_marginTop="16dp"
+                            android:gravity="center_vertical"
+                            android:orientation="horizontal">
+
+                            <TextView
+                                android:layout_width="0dp"
+                                android:layout_height="wrap_content"
+                                android:layout_weight="1"
+                                android:text="@string/label_livekit_auto"
+                                android:textColor="@color/text_primary"
+                                android:textSize="18sp" />
+
+                            <com.google.android.material.switchmaterial.SwitchMaterial
+                                android:id="@+id/switchLiveKitAuto"
+                                android:layout_width="wrap_content"
+                                android:layout_height="wrap_content"
+                                android:scaleX="1.2"
+                                android:scaleY="1.2" />
+                        </LinearLayout>
+
+                        <Button
+                            android:id="@+id/btnLiveKitSave"
+                            style="@style/Widget.App.Button"
+                            android:layout_width="match_parent"
+                            android:layout_height="72dp"
+                            android:layout_marginTop="16dp"
+                            android:textSize="20sp"
+                            android:text="@string/btn_save_livekit" />
+
+                    </LinearLayout>
+                </androidx.cardview.widget.CardView>
+
                 <androidx.cardview.widget.CardView
                     style="@style/CardView.App"
                     android:layout_width="match_parent"

app/src/main/res/layout/activity_main.xml

@@ -20,6 +20,16 @@
         app:layout_constraintEnd_toEndOf="parent"
         app:layout_constraintTop_toTopOf="parent" />
 
+    <View
+        android:id="@+id/statusIndicator"
+        android:layout_width="16dp"
+        android:layout_height="16dp"
+        android:layout_marginStart="24dp"
+        android:layout_marginTop="24dp"
+        android:background="@drawable/status_indicator"
+        app:layout_constraintStart_toStartOf="parent"
+        app:layout_constraintTop_toTopOf="parent" />
+
     <com.example.lzwcai_terminal_temi.AnimatedEmojiView
         android:id="@+id/animatedEmojiView"
         android:layout_width="0dp"
@@ -49,4 +59,4 @@
         app:layout_constraintEnd_toEndOf="parent"
         app:layout_constraintStart_toStartOf="parent" />
 
-</androidx.constraintlayout.widget.ConstraintLayout>
+</androidx.constraintlayout.widget.ConstraintLayout>

app/src/main/res/layout/activity_settings.xml

@@ -93,6 +93,22 @@
                             android:textColor="@color/text_primary" />
                     </com.google.android.material.textfield.TextInputLayout>
 
+                    <com.google.android.material.textfield.TextInputLayout
+                        style="@style/Widget.App.TextInputLayout"
+                        android:layout_width="match_parent"
+                        android:layout_height="wrap_content"
+                        android:layout_marginTop="12dp"
+                        android:hint="@string/hint_agent_id">
+
+                        <com.google.android.material.textfield.TextInputEditText
+                            android:id="@+id/etAgentDempId"
+                            android:layout_width="match_parent"
+                            android:layout_height="72dp"
+                            android:inputType="text"
+                            android:textSize="18sp"
+                            android:textColor="@color/text_primary" />
+                    </com.google.android.material.textfield.TextInputLayout>
+
                     <Button
                         android:id="@+id/btnSave"
                         style="@style/Widget.App.Button"
@@ -105,6 +121,109 @@
                 </LinearLayout>
             </androidx.cardview.widget.CardView>
 
+            <androidx.cardview.widget.CardView
+                style="@style/CardView.App"
+                android:layout_width="match_parent"
+                android:layout_height="wrap_content"
+                android:layout_marginBottom="24dp">
+
+                <LinearLayout
+                    android:layout_width="match_parent"
+                    android:layout_height="wrap_content"
+                    android:orientation="vertical"
+                    android:padding="16dp">
+
+                    <TextView
+                        android:layout_width="wrap_content"
+                        android:layout_height="wrap_content"
+                        android:layout_marginBottom="16dp"
+                        android:text="@string/label_livekit_config"
+                        android:textColor="@color/text_primary"
+                        android:textSize="22sp"
+                        android:textStyle="bold" />
+
+                    <com.google.android.material.textfield.TextInputLayout
+                        style="@style/Widget.App.TextInputLayout"
+                        android:layout_width="match_parent"
+                        android:layout_height="wrap_content"
+                        android:hint="@string/hint_livekit_url">
+
+                        <com.google.android.material.textfield.TextInputEditText
+                            android:id="@+id/etLiveKitUrl"
+                            android:layout_width="match_parent"
+                            android:layout_height="72dp"
+                            android:inputType="textUri"
+                            android:textSize="18sp"
+                            android:textColor="@color/text_primary" />
+                    </com.google.android.material.textfield.TextInputLayout>
+
+                    <com.google.android.material.textfield.TextInputLayout
+                        style="@style/Widget.App.TextInputLayout"
+                        android:layout_width="match_parent"
+                        android:layout_height="wrap_content"
+                        android:layout_marginTop="12dp"
+                        android:hint="@string/hint_livekit_room">
+
+                        <com.google.android.material.textfield.TextInputEditText
+                            android:id="@+id/etLiveKitRoom"
+                            android:layout_width="match_parent"
+                            android:layout_height="72dp"
+                            android:inputType="text"
+                            android:textSize="18sp"
+                            android:textColor="@color/text_primary" />
+                    </com.google.android.material.textfield.TextInputLayout>
+
+                    <com.google.android.material.textfield.TextInputLayout
+                        style="@style/Widget.App.TextInputLayout"
+                        android:layout_width="match_parent"
+                        android:layout_height="wrap_content"
+                        android:layout_marginTop="12dp"
+                        android:hint="@string/hint_livekit_token">
+
+                        <com.google.android.material.textfield.TextInputEditText
+                            android:id="@+id/etLiveKitToken"
+                            android:layout_width="match_parent"
+                            android:layout_height="72dp"
+                            android:inputType="text"
+                            android:textSize="18sp"
+                            android:textColor="@color/text_primary" />
+                    </com.google.android.material.textfield.TextInputLayout>
+
+                    <LinearLayout
+                        android:layout_width="match_parent"
+                        android:layout_height="wrap_content"
+                        android:layout_marginTop="16dp"
+                        android:gravity="center_vertical"
+                        android:orientation="horizontal">
+
+                        <TextView
+                            android:layout_width="0dp"
+                            android:layout_height="wrap_content"
+                            android:layout_weight="1"
+                            android:text="@string/label_livekit_auto"
+                            android:textColor="@color/text_primary"
+                            android:textSize="18sp" />
+
+                        <com.google.android.material.switchmaterial.SwitchMaterial
+                            android:id="@+id/switchLiveKitAuto"
+                            android:layout_width="wrap_content"
+                            android:layout_height="wrap_content"
+                            android:scaleX="1.2"
+                            android:scaleY="1.2" />
+                    </LinearLayout>
+
+                    <Button
+                        android:id="@+id/btnLiveKitSave"
+                        style="@style/Widget.App.Button"
+                        android:layout_width="match_parent"
+                        android:layout_height="72dp"
+                        android:layout_marginTop="16dp"
+                        android:textSize="20sp"
+                        android:text="@string/btn_save_livekit" />
+
+                </LinearLayout>
+            </androidx.cardview.widget.CardView>
+
             <androidx.cardview.widget.CardView
                 style="@style/CardView.App"
                 android:layout_width="match_parent"

app/src/main/res/values/strings.xml

@@ -16,5 +16,23 @@
     <string name="btn_restart_app">长按重启应用</string>
     <string name="label_location_config">当前位置</string>
     <string name="hint_current_location">请选择当前地点</string>
+    <string name="label_agent_filter">Agent 过滤</string>
+    <string name="hint_agent_id">请输入 demp_id（留空接收全部）</string>
     <string name="location_unknown">未知</string>
+    <string name="label_livekit_config">LiveKit 配置</string>
+    <string name="hint_livekit_url">请输入 LiveKit 地址 (wss://your-host)</string>
+    <string name="hint_livekit_token">请输入 LiveKit Token</string>
+    <string name="label_livekit_auto">LiveKit 自动连接</string>
+    <string name="btn_save_livekit">保存 LiveKit 配置</string>
+    <string name="msg_livekit_saved">LiveKit 配置已保存</string>
+    <string name="msg_livekit_cleared">LiveKit 配置已清除</string>
+    <string name="hint_livekit_room">请输入房间名 (例如 temi-room)</string>
+    <string name="label_livekit_status">LiveKit 状态</string>
+    <string name="livekit_status_disabled">LiveKit 未启用</string>
+    <string name="livekit_status_missing">LiveKit 待配置</string>
+    <string name="livekit_status_connecting">LiveKit 连接中</string>
+    <string name="livekit_status_connected">LiveKit 已连接</string>
+    <string name="livekit_status_disconnected">LiveKit 已断开</string>
+    <string name="livekit_status_failed">LiveKit 连接失败</string>
+    <string name="livekit_status_permission">需要麦克风/摄像头权限</string>
 </resources>

gradle/libs.versions.toml

@@ -8,6 +8,7 @@ espressoCore = "3.7.0"
 appcompat = "1.7.1"
 material = "1.13.0"
 emoji2-views = "1.5.0"
+livekit = "2.23.5"
 
 [libraries]
 androidx-core-ktx = { group = "androidx.core", name = "core-ktx", version.ref = "coreKtx" }
@@ -17,9 +18,9 @@ androidx-espresso-core = { group = "androidx.test.espresso", name = "espresso-co
 androidx-appcompat = { group = "androidx.appcompat", name = "appcompat", version.ref = "appcompat" }
 material = { group = "com.google.android.material", name = "material", version.ref = "material" }
 androidx-emoji2-views = { group = "androidx.emoji2", name = "emoji2-views", version.ref = "emoji2-views" }
+livekit-android = { group = "io.livekit", name = "livekit-android", version.ref = "livekit" }
 
 
 [plugins]
 android-application = { id = "com.android.application", version.ref = "agp" }
 kotlin-android = { id = "org.jetbrains.kotlin.android", version.ref = "kotlin" }
-

monitor.py

@@ -0,0 +1,106 @@
+import logging
+import asyncio
+import cv2
+import numpy as np
+from time import perf_counter
+from livekit import rtc, api
+
+# 直接写死你的 API_KEY 和 API_SECRET
+API_KEY = "devkey"
+API_SECRET = "secret"
+URL = "ws://192.168.2.236:7880"  # 直接使用你的 LIVEKIT_URL
+IDENTITY = "win-client"
+ROOM_NAME = "temi-room"
+
+# 生成 token
+def generate_token():
+    token = api.AccessToken(API_KEY, API_SECRET) \
+        .with_identity(IDENTITY) \
+        .with_name("Monitor Client1") \
+        .with_grants(api.VideoGrants(
+            room_join=True,
+            room=ROOM_NAME,
+        )).to_jwt()
+    return token
+
+async def main():
+    logging.basicConfig(level=logging.INFO)
+    logger = logging.getLogger(__name__)
+    room = rtc.Room()
+
+    try:
+        # 生成连接房间的 token
+        token = generate_token()
+
+        # 连接到 LiveKit 房间
+        await room.connect(URL, token)
+        logger.info("Connected to room: %s", room.name)
+
+        # 监听参与者连接事件
+        @room.on("participant_connected")
+        def on_participant_connected(participant: rtc.RemoteParticipant):
+            logger.info("Participant connected: %s %s", participant.sid, participant.identity)
+
+        async def receive_frames(stream: rtc.VideoStream, track_sid: str):
+            window_name = f"LiveKit Video Stream ({track_sid})"
+            cv2.namedWindow(window_name, cv2.WINDOW_AUTOSIZE)
+            last_time = perf_counter()
+            frame_count = 0
+            fps = 0.0
+            
+            try:
+                async for event in stream:
+                    frame = event.frame
+                    rgb_frame = frame.convert(rtc.VideoBufferType.RGB24)
+                    arr = np.frombuffer(rgb_frame.data, dtype=np.uint8)
+                    try:
+                        img = arr.reshape((rgb_frame.height, rgb_frame.width, 3))
+                        img_bgr = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)
+                        frame_count += 1
+                        now = perf_counter()
+                        elapsed = now - last_time
+                        if elapsed >= 1.0:
+                            fps = frame_count / elapsed
+                            frame_count = 0
+                            last_time = now
+                        cv2.putText(
+                            img_bgr,
+                            f"FPS: {fps:.1f}",
+                            (10, 30),
+                            cv2.FONT_HERSHEY_SIMPLEX,
+                            1.0,
+                            (0, 255, 0),
+                            2,
+                            cv2.LINE_AA,
+                        )
+                        cv2.imshow(window_name, img_bgr)
+                        if cv2.waitKey(1) & 0xFF == ord('q'):
+                            break
+                    except Exception as e:
+                        logger.error("Error processing frame: %s", e)
+            finally:
+                cv2.destroyWindow(window_name)
+
+        # 监听 track 订阅事件
+        @room.on("track_subscribed")
+        def on_track_subscribed(track: rtc.Track, publication: rtc.RemoteTrackPublication, participant: rtc.RemoteParticipant):
+            logger.info("Track subscribed: %s", publication.sid)
+            if track.kind == rtc.TrackKind.KIND_VIDEO:
+                video_stream = rtc.VideoStream(track)
+                asyncio.ensure_future(receive_frames(video_stream, publication.sid))
+
+        # 保持连接并处理事件
+        await asyncio.Event().wait()
+
+    except Exception as e:
+        logger.error("Failed to connect or handle events: %s", e)
+
+    finally:
+        # 确保断开连接
+        await room.disconnect()
+
+if __name__ == "__main__":
+    try:
+        asyncio.run(main())
+    except KeyboardInterrupt:
+        print("连接已被用户中断")

settings.gradle.kts

@@ -16,9 +16,10 @@ dependencyResolutionManagement {
     repositories {
         google()
         mavenCentral()
+        maven { url = uri("https://jitpack.io") }
     }
 }
 
 rootProject.name = "lzwcai-terminal-temi"
 include(":app")
- 
+ 

technique.md

@@ -0,0 +1,160 @@
+# Temi 终端控制应用 — 技术文档
+
+## 1. 项目概览
+本项目为基于 Temi SDK 的 Android 应用，核心能力是通过 MQTT 指令驱动机器人导航、接待/巡逻与语音播报，并配合表情状态展示与 LiveKit 音视频连接能力。
+
+主要入口与核心逻辑集中在：
+- MainActivity（主界面与机器人事件监听）
+- MqttManager（MQTT 连接与指令解析）
+- NavController（导航控制封装）
+
+## 2. 系统架构
+
+### 2.1 分层结构
+- **UI 层**
+  - MainActivity：主界面、机器人事件监听、任务状态流转
+  - SettingsActivity：配置页面（IP、当前位置、特殊任务模式、LiveKit 参数）
+  - AnimatedEmojiView：表情动画组件
+- **业务层**
+  - MqttManager：MQTT 连接与指令协议处理、TTS 队列与流式播报
+  - NavController：导航/巡逻/重定位封装
+- **连接能力**
+  - LiveKitManager：LiveKit 房间连接与状态管理
+
+### 2.2 核心模块职责
+- **MainActivity**
+  - 监听 TTS、导航状态、人检测事件
+  - 驱动表情变化与任务状态
+  - 管理 MQTT / LiveKit 连接生命周期
+- **MqttManager**
+  - MQTT 连接、订阅主题
+  - JSON 指令解析与动作下发
+  - 流式播报与 TTS 队列管理
+- **NavController**
+  - 封装 goTo、patrol、repose、recharge 等指令
+
+## 3. 运行与构建
+
+### 3.1 环境要求
+- Android Studio
+- Gradle
+- Temi SDK 依赖
+- 真实 Temi 设备或 Android 模拟器（模拟器仅可预览 UI）
+
+### 3.2 构建与安装
+```bash
+.\gradlew.bat :app:installDebug
+```
+
+## 4. MQTT 通信协议
+
+### 4.1 连接信息
+- Broker：`tcp://<IP>:1883`
+- 订阅主题：
+  - `robot/cmd`：指令通道
+  - `soul2user`：流式播报通道
+
+### 4.2 基础指令
+| action | 说明 |
+|---|---|
+| recharge | 前往充电桩 |
+| goto | 前往地点（location / target） |
+| repose | 重新定位 |
+| stop | 停止移动并暂停播报 |
+| continue | 继续播报 |
+| terminate | 停止移动并清空 TTS |
+
+### 4.3 播报指令
+| action | 说明 |
+|---|---|
+| speak | 立即播报（text / speech，可选 lang） |
+| stream | 流式播报（text / content，可选 lang） |
+
+### 4.4 任务指令
+| action | 说明 |
+|---|---|
+| patrol | 巡逻：flag=true 随机地点，flag=false 使用 locations |
+| reception | 接待：location / text / destination |
+
+### 4.5 示例
+```json
+{"action":"goto","location":"前台"}
+```
+
+```json
+{"action":"speak","text":"欢迎光临","lang":"zh"}
+```
+
+```json
+{"action":"patrol","flag":false,"locations":["前台","会议室","大厅"]}
+```
+
+## 5. 任务与导航逻辑
+
+### 5.1 接待模式
+- 接收到 reception 指令后进入接待任务
+- 到达接待点后检测到人显示确认按钮并播报提示
+- 点击确认后前往目的地并结束接待模式
+
+### 5.2 巡逻模式
+- 随机巡逻：从已有地点中随机抽取 3~6 个（排除 home base）
+- 指定巡逻：按 locations 列表执行
+- 完成全部地点后自动结束巡逻任务
+
+### 5.3 到达逻辑
+- 到达地点后更新 current_location
+- 若为巡逻任务则推进巡逻索引
+- 特殊任务模式下可跳过“已到达”播报
+
+## 6. 表情与语音联动
+
+- **TTS STARTED**：表情变为 TALKING
+- **TTS COMPLETED/CANCELED**：
+  - 巡逻中显示 ANGRY
+  - 非巡逻显示 SMILE
+- **TTS ERROR**：表情显示 SAD
+- 闲置时自动眨眼动画
+
+## 7. 配置项与持久化
+
+### 7.1 SharedPreferences 关键字段
+- `network_ip`：MQTT Broker 地址
+- `current_location`：当前位置
+- `special_task_mode`：特殊任务模式开关
+- `livekit_url / livekit_room / livekit_token / livekit_enabled`
+- `agent_demp_id`：stream 匹配字段
+
+### 7.2 设置页功能
+- MQTT IP 设置
+- LiveKit 连接参数设置
+- 特殊任务模式开关
+- 当前位置选择与保存
+- 长按 3 秒重启应用
+
+## 8. 特殊任务模式
+
+开启后会：
+- 跳过 Home Base 的开门/关门语音逻辑
+- 跳过检测到人时的问候语
+- 到达地点时不播报“已到达”（无任务状态下）
+
+## 9. LiveKit 连接
+
+- 默认 URL 会根据 IP 自动拼接 `ws://<IP>:7880`
+- 若未配置 token，则在本地使用默认 key/secret 生成临时 token
+- 连接状态与 MQTT 状态共同驱动主界面状态指示灯
+
+## 10. 权限要求
+- Temi 权限：MAP / SEQUENCE / FACE_RECOGNITION / SETTINGS
+- Android 权限：RECORD_AUDIO / CAMERA（LiveKit）
+
+## 11. 依赖说明
+- Temi SDK：`com.robotemi:sdk:1.137.1`
+- MQTT：`org.eclipse.paho:org.eclipse.paho.android.service:1.1.1`
+- LiveKit：`io.livekit:android`
+- AndroidX / Material 等基础依赖
+
+## 12. 安全注意事项
+- MQTT 用户名/密码在代码内配置
+- LiveKit 默认 key/secret 也在代码内生成 token
+- 建议正式环境将敏感信息迁移至安全配置源