Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assets/core/VideoService.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/light/effects/AmbilightEffect.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/light/effects/AmbilightEffect.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 34 additions & 0 deletions docs/moonmodules/core/services.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,24 @@

[Tests](../../reference/tests/unit-tests.md#audioservice)

<a id="video"></a>

### Video

<img src="../../assets/core/VideoService.png" width="300" alt="Video service card">

A user-added Service: the video source a screen-follow effect reads. The counterpart of [Audio](#audio) for a picture, decoded once per tick however many effects want it. `source` decides which controls show. The pattern, the file format and the tone curve: ⌄ details.

- `source`: `test pattern` needs no hardware, `file` reads a PPM off the filesystem.
- `patternSpeed`: (test pattern) sweep rate of the white block, in pixels per second. 0 parks it.
- `file`: (file) path to a binary PPM (P6, maxval 255), uploaded through the File Manager.
- `reload`: (file) re-read the file in place, without rebuilding the pipeline.
- status: the live frame's dimensions (`64x36`), or the reason there is no frame.

Detail: [technical](moxygen/VideoService.md)

[Tests](../../reference/tests/unit-tests.md#videoservice)

<a id="osc"></a>

### OSC
Expand Down Expand Up @@ -249,6 +267,22 @@

Prior art: the WLED-MM audio-reactive usermod by **Frank ([@softhack007](https://github.com/softhack007))**, the most-used open-source audio-reactive LED implementation, whose adaptive noise-gate concept the analysis here descends from (analyzed with his permission); and **[@troyhacks](https://github.com/troyhacks/WLED)**, who reworked that DSP onto Espressif's [esp-dsp](https://github.com/espressif/esp-dsp) FFT, the same choice this service makes. The line-in path exists because **wladi ([myhome-control](https://shop.myhome-control.de))** supplied the hardware and pinout for the [MHC-WLED ESP32-P4 shield](../../reference/hardware/mhc-wled-esp32-p4-shield.md): its onboard PCM1808 I2S ADC is what `mclkPin` is for.

## Video, details

**What reads it.** The service publishes one decoded RGB frame per tick through a static accessor, and any number of effects read that same frame: it is decoded once here however many want it. A consumer holds the frame for one tick and never across frames, and before any frame exists the pointer is non-null but its pixels are, which every reader has to tolerate. That contract is what lets a source be added and removed under a running pipeline.

**The test pattern is a diagnostic, not decoration.** Four colored border bands (red top, green right, blue bottom, yellow left) and a white block sweeping along the top edge. On a border-mounted strip that makes orientation self-evident: a strip mapped the wrong way round shows as the wrong physical edge lighting, rather than a subtly wrong picture. The sweeping block shows liveness and which way "forward" runs. Parking it with `patternSpeed` 0 turns the pattern into a still reference, so one light can be compared against a known color without the block passing through its zone mid-read.

**Why PPM for the file source.** There is no software JPEG decoder in this codebase, and adding one so the desktop build could open a `.jpg` would buy a dependency for a development convenience. PPM is a raw RGB dump behind a three-line ASCII header, so it needs no decoder and is one command from any source material:

Check warning on line 277 in docs/moonmodules/core/services.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/core/services.md","range":{"start":{"line":277,"column":66},"end":{"line":277,"column":70}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}
```sh

Check warning on line 278 in docs/moonmodules/core/services.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/core/services.md","range":{"start":{"line":278,"column":319},"end":{"line":278,"column":328}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}

Check warning on line 278 in docs/moonmodules/core/services.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/core/services.md","range":{"start":{"line":278,"column":141},"end":{"line":278,"column":144}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}
ffmpeg -i clip.mp4 -frames:v 1 -vf scale=64:36 -pix_fmt rgb24 frame.ppm
```

Check warning on line 281 in docs/moonmodules/core/services.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/core/services.md","range":{"start":{"line":281,"column":223},"end":{"line":281,"column":231}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}

Check warning on line 281 in docs/moonmodules/core/services.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/core/services.md","range":{"start":{"line":281,"column":1},"end":{"line":281,"column":4}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}
The header is parsed strictly, because the file is whatever a user uploaded: comments and whitespace runs are skipped per Netpbm, a 16-bit maxval is refused rather than misread as 8-bit, and exactly one whitespace byte is consumed before the pixels. Accepting a non-whitespace separator there would eat a pixel and shift every channel one place, which tints the whole image instead of failing.

**Every frame carries its tone curve.** A published frame names the curve from its own encoding to linear light, and a consumer reads pixels through it. SDR is sRGB, a curve like any other, so the frame always carries one rather than leaving the reader to assume. This matters because averaging is what a screen-follow effect does, and the mean of encoded bytes is not the mean of the light: half black and half white averages to 128 as bytes where the light is 188. The table is 12-bit rather than 8, since linear has no headroom at the dark end, which is what encodings exist for.

Check warning on line 285 in docs/moonmodules/core/services.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/core/services.md","range":{"start":{"line":285,"column":1},"end":{"line":285,"column":6}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}
## OSC, details
**Feedback: the device answers.** With `feedback` on, a control that changes anywhere (the web UI, a preset recall, an audio-reactive effect) is mirrored back to the surface, which is what keeps a client honest and what moves a motorized fader. `addressing` picks where it goes: `unicast` to each of `hosts`, addresses or names, or to whoever last wrote when `hosts` is empty; `multicast` to `group`, which every listening device with the same `group` joins; the default, `239.255.77.78`, sits beside discovery's own group, so multicast works with nothing to fill in, and a second rig on the same network picks another. `feedbackPort` is where that client LISTENS, which is not the port we listen on (Open Stage Control calls its own `osc-port`).

Expand Down
34 changes: 34 additions & 0 deletions docs/moonmodules/light/effects.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,28 @@

## MoonLight effects

<a id="ambilight"></a>

### Ambilight 📺

<img src="../../assets/light/effects/AmbilightEffect.gif" width="300" alt="Ambilight effect preview">

Paints the layer with the live frame from the [Video](../core/services.md#video) service, so lights around a display glow the color of the picture nearest them. Each light shows the **mean** of the rectangle mapping to it. Sampling, black bars and the layout's part: ⌄ details.

- `brightness`: scales the sampled color, dimming the video not the output.
- `saturation`: how far each channel is pushed from its zone's luma.
- `smoothing`: how much of the gap to a new color closes per frame.
- `snapAbove`: a channel moving further than this jumps instead of easing.
- `edgeDepth`: how far into the picture the outermost lights look.
- `detectBlackBars`: map the lights across the picture, not the frame.
- `barLevel`: how dark a pixel counts as bar (0 to 64).

Origin: MoonLight original, on the Ambilight / Hyperion behavior

Detail: [technical](moxygen/AmbilightEffect.md)

[Tests](../../reference/tests/unit-tests.md#ambilighteffect)

<a id="colortrails"></a>

### ColorTrails 💫🖌️💨🌫️ · 3D
Expand Down Expand Up @@ -1367,3 +1389,15 @@
Detail: [technical](moxygen/SineEffect.md)

[Tests](../../reference/tests/unit-tests.md#sineeffect)

## Ambilight, details

**Two spaces, and the layout decides the shape.** The SOURCE is the video frame, counted in pixels; the DESTINATION is the layer's logical box, counted in light positions. The source is far the bigger, a 640x480 picture onto a strip of 60 positions, so each position owns a whole rectangle of pixels and shows their average. The effect asks the mapping only one question, whether a position lights anything, and skips the averaging where the answer is no. On a border layout that is most of the box: the interior of a backlight strip's bounding box maps to no LED, so the perimeter shows the frame's border for free. On a full grid the same effect is a video wall. A [MoonLive layout](layouts.md#moonlive) running the library's `backlight.mll` is how a strip gets placed on a wall's edge pixels.

Check warning on line 1395 in docs/moonmodules/light/effects.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/light/effects.md","range":{"start":{"line":1395,"column":457},"end":{"line":1395,"column":459}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}

**Averaging happens in linear light.** Each zone's pixels are summed through the frame's tone curve, averaged, then encoded once per light. Averaging the raw bytes instead averages a quantity that is not proportional to light: a zone half black and half white comes out at 128 where the light is 188, which reads as a dim mush rather than a bright scene. The accumulators are 32-bit because a 640x480 frame onto 32x18 positions is about 520 pixels each, and 520 full-scale 12-bit samples overflow 16 bits many times over.

Check warning on line 1397 in docs/moonmodules/light/effects.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/light/effects.md","range":{"start":{"line":1397,"column":141},"end":{"line":1397,"column":150}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}

**`edgeDepth` widens the outermost zones.** A border light's own share is one row or column of the frame, a sliver at the very edge, which picks up the letterbox and the overscan junk before it picks up the picture. `edgeDepth` lets the outermost positions reach a percentage of the way in instead, without changing how many of them there are. It costs work in proportion: on a layout where every light is an edge light, 15% sends the pixel count up about sixfold.

Check warning on line 1399 in docs/moonmodules/light/effects.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.Weasel] 'very' adds emphasis, not information. Raw Output: {"message":"[MoonLight.Weasel] 'very' adds emphasis, not information.","location":{"path":"docs/moonmodules/light/effects.md","range":{"start":{"line":1399,"column":123},"end":{"line":1399,"column":127}}},"severity":"INFO","code":{"value":"MoonLight.Weasel"}}

Check warning on line 1399 in docs/moonmodules/light/effects.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma. Raw Output: {"message":"[MoonLight.SentenceLength] Sentence runs past 30 words; it is probably two thoughts joined by a comma.","location":{"path":"docs/moonmodules/light/effects.md","range":{"start":{"line":1399,"column":45},"end":{"line":1399,"column":46}}},"severity":"INFO","code":{"value":"MoonLight.SentenceLength"}}

**Black bars move the zones, which `edgeDepth` cannot.** A letterboxed film puts bars exactly where the top and bottom lights look, so those lights go dark while the picture is bright. `edgeDepth` only widens a zone from its edge, so the bar stays inside it. `detectBlackBars` finds the bar and maps the lights across the picture within it. A reading is adopted only once several frames agree, because bars come and go at scene changes and a mapping that follows every dark frame twitches worse than one that ignores them. Reaching the scan ceiling reports no bar at all: darkness that deep is a dark scene rather than a letterbox.

**No source paints black.** With nothing publishing, the effect fills its layer with black rather than returning early, or the previous effect's picture would stay frozen on the strip. A merely dropped frame never reaches that path, since the service keeps its buffer and only a removed service drops it.
5 changes: 5 additions & 0 deletions moondeck/docs/screenshot_modules.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@
("RainbowEffect", "Layer", {}, True),
("NoiseEffect", "Layer", {}, True),
# The generative-fields showcases: each is Dim::D3, so the preview shows a volume.
# Paints the live video frame, so with no source it photographs black. The capture points it
# at the test pattern, whose four coloured bands and sweeping block are what the effect is for.

Check failure on line 110 in moondeck/docs/screenshot_modules.py

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [MoonLight.Spelling] American spelling: use 'colored'. Raw Output: {"message":"[MoonLight.Spelling] American spelling: use 'colored'.","location":{"path":"moondeck/docs/screenshot_modules.py","range":{"start":{"line":110,"column":39},"end":{"line":110,"column":47}}},"severity":"ERROR","code":{"value":"MoonLight.Spelling"}}
("AmbilightEffect", "Layer", {}, True),
("AuroraEffect", "Layer", {}, True),
("ColorTrailsEffect", "Layer", {}, True),
("TrailsEffect", "Layer", {}, True),
Expand Down Expand Up @@ -178,6 +181,7 @@
# (not in state); capture these against a board when needed.
"ImprovProvisioningModule",
"AudioService",
"VideoService",
"AnalogService",
"I2cBusModule",
"InfraredService",
Expand Down Expand Up @@ -209,6 +213,7 @@
# the desktop tree — so they're captured against an ESP32, where these entries route the
# shot to the right nav root.
"AudioService": "Services",
"VideoService": "Services",
"AnalogService": "Services",
"InfraredService": "Services",
"ButtonService": "Services",
Expand Down
Loading
Loading