Skip to content

sim/camera: add AVFoundation backend and multi-cam support#18745

Merged
xiaoxiang781216 merged 5 commits intoapache:masterfrom
PeterBee97:avfoundation
May 8, 2026
Merged

sim/camera: add AVFoundation backend and multi-cam support#18745
xiaoxiang781216 merged 5 commits intoapache:masterfrom
PeterBee97:avfoundation

Conversation

@PeterBee97
Copy link
Copy Markdown
Contributor

@PeterBee97 PeterBee97 commented Apr 15, 2026

Summary

Add AVFoundation backend and multi-cam support, enabling macOS users as well as Linux users to use multiple cameras on Simulator.

Impact

  • Impact on users: Backward compatible
  • Affected platforms:
    • SIM on macOS
    • SIM on Linux
  • Scope:
    • Enhances the SIM camera framework for multi-instance support
    • Adds a new macOS AVFoundation camera backend
    • Fixes mkallsyms host-tool behavior for Objective-C symbols on macOS
  • Non-affected platforms:
    • No impact on other boards or non-SIM targets
    • Existing Linux SIM camera behavior remains supported

Testing

Tested on:

  • Host OS: macOS Tahoe on Apple Silicon, Ubuntu x86_64
  • Target configurations: sim:nxcamera
image

@github-actions github-actions Bot added Area: Build system Arch: simulator Issues related to the SIMulator Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. Board: simulator labels Apr 15, 2026
anchao
anchao previously approved these changes Apr 15, 2026
View reviewed changes
acassis
acassis requested changes Apr 15, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@acassis acassis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PeterBee97 WOW! Amazing! Please update the nxcamera Documentation/ to include this new parameter: https://nuttx.apache.org/docs/latest/applications/system/nxcamera/index.html

@acassis
Copy link
Copy Markdown
Contributor

acassis commented Apr 15, 2026

@PeterBee97 please include this picture in the Documentation page, it is funny :-)

simbit18
simbit18 previously approved these changes Apr 15, 2026
View reviewed changes
@PeterBee97 PeterBee97 dismissed stale reviews from simbit18 and anchao via 1131750 April 16, 2026 04:21
@PeterBee97
Copy link
Copy Markdown
Contributor Author

PeterBee97 commented Apr 16, 2026

@PeterBee97 WOW! Amazing! Please update the nxcamera Documentation/ to include this new parameter: https://nuttx.apache.org/docs/latest/applications/system/nxcamera/index.html

As u wish :)

@PeterBee97 PeterBee97 requested a review from acassis April 16, 2026 06:26
simbit18
simbit18 previously approved these changes Apr 16, 2026
View reviewed changes
xiaoxiang781216
xiaoxiang781216 reviewed Apr 16, 2026
View reviewed changes
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
Comment thread arch/sim/src/sim/sim_camera.c Outdated
acassis
acassis previously approved these changes Apr 16, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@acassis acassis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PeterBee97 Amazing! Thank you very much!

@PeterBee97 PeterBee97 dismissed stale reviews from acassis and simbit18 via c2ea6da April 17, 2026 09:16
@simbit18
Copy link
Copy Markdown
Contributor

simbit18 commented Apr 17, 2026

Hi @PeterBee97 please fix

====================================================================================
Configuration/Tool: sim/nxcamera
2026-04-17 09:51:22
------------------------------------------------------------------------------------
  Cleaning...
  Configuring...
  Building NuttX...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  485k  100  485k    0     0  1893k      0 --:--:-- --:--:-- --:--:-- 1893k
sim/posix/sim_host_v4l2.c: In function 'host_video_get_device_path_by_index':
Error: sim/posix/sim_host_v4l2.c:147:15: error: implicit declaration of function 'strlcpy'; did you mean 'strncpy'? [-Werror=implicit-function-declaration]
  147 |           if (strlcpy(devpath, path, devpathlen) >= devpathlen)
      |               ^~~~~~~
      |               strncpy
cc1: all warnings being treated as errors
make[1]: *** [Makefile:408: sim_host_v4l2.o] Error 1
make[1]: Target 'nuttx' not remade because of errors.
make: *** [tools/Unix.mk:568: nuttx] Error 2
make: Target 'all' not remade because of errors.
/github/workspace/sources/nuttx/tools/testbuild.sh: line 386: /github/workspace/sources/nuttx/../nuttx/nuttx.manifest: No such file or directory
  [1/1] Normalize sim/nxcamera
Removing libs/libbuiltin/bin/
Removing libs/libbuiltin/kbin/
Removing libs/libc/kbin/
Removing libs/libnx/bin/
Removing libs/libnx/kbin/
Removing mm/kbin/
/github/workspace/sources/nuttx /github/workspace/sources/nuttx
HEAD detached at pull/18745/merge
nothing to commit, working tree clean
/github/workspace/sources/nuttx
/github/workspace/sources/apps /github/workspace/sources/nuttx
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
/github/workspace/sources/nuttx
Build Attempt 1 of 1
===================================================================================

xiaoxiang781216
xiaoxiang781216 reviewed Apr 19, 2026
View reviewed changes
Comment thread tools/mkallsyms.sh
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
@PeterBee97 PeterBee97 force-pushed the avfoundation branch 2 times, most recently from 6b24b6b to 6ae23c9 Compare April 20, 2026 02:50
@PeterBee97 PeterBee97 force-pushed the avfoundation branch 4 times, most recently from 28dda56 to 13efe13 Compare April 29, 2026 09:29
jerpelea
jerpelea previously approved these changes Apr 29, 2026
View reviewed changes
acassis
acassis previously approved these changes Apr 29, 2026
View reviewed changes
xiaoxiang781216
xiaoxiang781216 requested changes Apr 29, 2026
View reviewed changes
Comment thread drivers/video/v4l2_cap.c Outdated
Comment thread drivers/video/v4l2_cap.c Outdated
Comment thread drivers/video/v4l2_cap.c Outdated
@xiaoxiang781216
Copy link
Copy Markdown
Contributor

xiaoxiang781216 commented Apr 29, 2026

@acassis @jerpelea this patch change the common code, before I approve the change, please don't merge the pr.

@PeterBee97 PeterBee97 dismissed stale reviews from acassis and jerpelea via 408638b April 30, 2026 03:46
@PeterBee97 PeterBee97 force-pushed the avfoundation branch 2 times, most recently from 408638b to 60bdeaf Compare April 30, 2026 03:52
xiaoxiang781216
xiaoxiang781216 reviewed Apr 30, 2026
View reviewed changes
Comment thread arch/sim/src/sim/sim_camera.c Outdated
Comment thread arch/sim/src/sim/sim_camera.c Outdated
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated
xiaoxiang781216
xiaoxiang781216 previously approved these changes May 8, 2026
View reviewed changes
Peter Bee added 4 commits May 8, 2026 14:10
sim: support camera framework multi-instance and dynamic mounting
b177473 
Allow the SIM camera/V4L2 capture framework to manage multiple
imgdata instances with dynamic mounting. This avoids cross-talk
between camera streams when multiple devices are used.

Signed-off-by: Peter Bee <bijunda@bytedance.com>
sim: adapt macos avfoundation backend for camera indexing and capture
ec86f24 
Adapt the macOS AVFoundation backend for multi-device discovery,
camera index mapping and capture startup compatibility. This enables
reliable use of multiple cameras through the SIM camera framework.

Signed-off-by: Peter Bee <bijunda@bytedance.com>
sim: fix mkallsyms for Objective-C symbols on macOS
65d5821 
/Library/Developer/CommandLineTools/usr/bin/c++filt: error: unknown argument '-['

Signed-off-by: Peter Bee <bijunda@bytedance.com>
Documentation: add nxcamera documentation
f519f30 
Document the nxcamera command usage, including device selection,
resolution, and pixel format options. Also mention recent support
for multi-instance camera mounting and the macOS AVFoundation
backend on the SIM platform.

Signed-off-by: Peter Bee <bijunda@bytedance.com>
@PeterBee97
Copy link
Copy Markdown
Contributor Author

PeterBee97 commented May 8, 2026

Looks like CI breaks on unrelated paths. The previous build succeeded on all except Linux sim-02, which was fixed by the latest change.

@PeterBee97 PeterBee97 requested review from jerpelea and simbit18 May 8, 2026 08:48
@xiaoxiang781216 xiaoxiang781216 merged commit 99d639b into apache:master May 8, 2026
36 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jerpelea jerpelea jerpelea approved these changes

@xiaoxiang781216 xiaoxiang781216 xiaoxiang781216 approved these changes

@anchao anchao anchao left review comments

@yamt yamt Awaiting requested review from yamt yamt is a code owner

@masayuki2009 masayuki2009 Awaiting requested review from masayuki2009 masayuki2009 is a code owner

@pkarashchenko pkarashchenko Awaiting requested review from pkarashchenko pkarashchenko is a code owner

@Donny9 Donny9 Awaiting requested review from Donny9 Donny9 is a code owner

@GUIDINGLI GUIDINGLI Awaiting requested review from GUIDINGLI

@acassis acassis Awaiting requested review from acassis

@simbit18 simbit18 Awaiting requested review from simbit18

Assignees

No one assigned

Labels

Arch: simulator Issues related to the SIMulator Area: Build system Board: simulator Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@PeterBee97 @acassis @simbit18 @xiaoxiang781216 @anchao @jerpelea