Skip to content

Port native device control tools to Flutter agent #30

Description

@SebastianBoehler

Context

The previous Android app already had native device/app-control tools in the Java layer. The Flutter migration copied those classes into the Android runner and currently exposes a subset through the LiteRT local tool protocol.

Relevant existing code:

  • flutter_app/android/app/src/main/java/com/example/studyOS/offline/Tools.java
  • flutter_app/android/app/src/main/kotlin/com/studyos/studyos_agent/AndroidLiteRtToolExecutor.kt
  • flutter_app/android/app/src/main/kotlin/com/studyos/studyos_agent/MainActivity.kt
  • flutter_app/lib/src/studyos_tool_catalog.dart
  • flutter_app/lib/src/studyos_tool_executor.dart

Current state

Android already has implementations for:

  • device status (getDeviceStatus)
  • flashlight control (toggleFlashlight)
  • opening installed apps (openApp)
  • YouTube search (searchYoutube)
  • volume and settings-panel helpers for Wi-Fi/Bluetooth/GPS/mobile data
  • phone/calendar/share helpers that are not all exposed through Flutter tools

The Flutter Android bridge currently advertises device capabilities and exposes GET_STATUS, LIGHT_CONTROL, OPEN_APP, and SEARCH_YOUTUBE for LiteRT local model use. The normal Dart tool catalog does not yet expose a polished cross-platform/native-device tool surface.

Proposed scope

Add a clear Flutter-native device control tool layer that routes to Android implementations where supported and returns explicit unsupported errors elsewhere.

Suggested first tools:

  • get_device_status
  • set_flashlight
  • open_installed_app
  • search_youtube
  • optionally open_system_setting for Wi-Fi/Bluetooth/location panels instead of pretending direct toggles are reliable

Acceptance criteria

  • Flutter tool catalog includes the supported device control tools with JSON schemas and user-safe descriptions.
  • Dart executor routes device-control tool calls through NativeBridge instead of duplicating Android logic in Dart.
  • Android MethodChannel exposes stable methods for the selected tools.
  • Unsupported platforms return explicit, user-facing errors instead of mock success.
  • Permission-sensitive actions check capabilities before execution and report missing permissions clearly.
  • The assistant prompt/tool descriptions do not claim Android can directly toggle settings that modern Android blocks.

Platform/security notes

  • Android can open settings panels for Wi-Fi/Bluetooth/location, but direct toggles are restricted on modern Android versions.
  • Opening arbitrary apps is Android-only and should stay behind a user-visible tool action.
  • iOS arbitrary app control is generally not feasible; only specific App Intents/URL schemes should be considered.

Test expectations

  • Add focused Dart tests for tool schema/executor routing where practical.
  • Add Android-side smoke/manual QA notes for flashlight, app launch, status read, and YouTube search on a real device or emulator.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions