A real-time object detection iOS app built using Swift, Core ML, and Create ML. It uses a custom-trained YOLOv2 model to recognize all 26 American Sign Language (ASL) alphabet letters directly from the device camera — fully on-device, no internet required.
- Real-time camera input via
AVFoundation - Custom-trained YOLOv2 CoreML model (
asl.mlpackage) — trained in Create ML - 26 ASL alphabet classes with bounding-box detection
- On-device inference via Core ML + Vision (no server, no network calls)
- SwiftUI interface with UIKit camera integration (MVVM architecture)
- Light and dark mode support
| Launch View (Light) | Launch View (Dark) |
|---|---|
 |
 |
| App Icon (Light) | App Icon (Dark) |
|---|---|
 |
 |
A live demo video is included at screenshots/Demo.MP4.
| Layer | Details |
|---|---|
| ML Model | YOLOv2 object detection, trained in Create ML, exported as .mlpackage |
| Inference | Core ML + Vision framework — runs entirely on-device |
| Camera | AVFoundation (AVCaptureSession) |
| Language | Swift 5 |
| UI | SwiftUI (views) + UIKit (UIViewControllerRepresentable camera controller) |
| Architecture | MVVM |
Requirements
- macOS 13.0+
- Xcode 14.0+
- iOS 15.0+ physical device (camera required — simulator has no camera)
- Apple Developer account (free tier is fine for device testing)
Steps
- Clone the repo and open
ObjectDetection.xcodeprojin Xcode. - Select your device in the scheme selector.
- Set a bundle ID and signing team under Signing & Capabilities.
- Build and run (
Cmd+R). Grant camera permission when prompted.
The model file
ObjectDetection/Resources/MLModels/asl.mlpackageis already bundled — no extra download needed.
- Labeled training images with bounding boxes for each of the 26 ASL letters.
- Trained using Create ML's Object Detection template (YOLOv2 backbone).
- Exported the trained model as
.mlpackageand bundled it directly in the Xcode project. - At runtime,
Visionwraps the CoreML model for camera-frame inference.