Repository navigation
Turn Synesthesia's audio orb into a charge cloud - #230
Merged
Merged
Conversation
Skins built on pixi-kit had no way to reach the renderer, so they could not step a simulation in render textures. queueGpuPass() lets a skin queue work while it builds a frame; the surface runs the queued passes on its own renderer right before drawing that frame. A failing pass is reported and does not stop the rest.
Five small dark orbs hang in a GPU-simulated fluid of particles. Each orb listens to its own slice of the spectrum and is a light: a hit fires a one-frame flash with light shafts that falls off in about 0.1 s, then a glow that fades over about a second. The particles are lit by the orbs (pink to white near a flaring orb, deep indigo elsewhere) and moved by charge: drawn to orbs of the opposite sign, driven off by their own, spun round each orb's axis, with each particle tuned to one spectrum band. The particles also flock through a coarse 32^3 velocity/density grid that every particle splats into each frame (alignment toward the local mean velocity, cohesion/separation around a target density), so the cloud folds into smoky sheets instead of scattering. The select screen's foreground orb and the gameplay idle monitor now use it. The CPU particle shell, its fibres, the smaller background orb and the black moon go, along with the helpers only they used.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
be-music-player-demo | 0a756c1 | Commit Preview URL Branch Preview URL |
Oct 11 2026, 05:23 PM |
Exports Benchmark
Summary
Top Regressions
Top ImprovementsNo improvement over threshold. Excluded (sub-timer-resolution)Per-call latency at or below 0.001ms on at least one side — the reported time is measurement noise, not the case's real cost, so no percent change is shown.
|
…exture Loud passages washed the cloud out: the orb lights followed raw band energy, so a loud mix kept every orb lit and flashing on every beat, and the light piled up on the particles without limit. - Each orb now reacts to how far its band stands above its own running average (about 2.5 s), so a sustained loud passage settles back down and only real jumps light it up. - After a flash an orb waits 0.18 s before it can flash again. - The light a particle receives saturates softly instead of adding up. The particles are now drawn as the same crisp square dots as the skin's other point clouds (0.6-2 px by weight and depth) instead of 1 px strokes, shimmering with the highs, with their opacity scaled by the cloud's on-screen size so the select screen and the monitor match. The colours are unchanged.
The gameplay chrome cleared the monitor cloud's clock at the top of every frame, so each update saw no elapsed time: the simulation never stepped and the orb lights never decayed, leaving the monitor frozen and lit instead of reacting like the select screen's cloud. The clock now carries on across frames; the cloud already clamps each step, so frames spent hidden behind a BGA just resume.
…fine grain The charge cloud is drawn as dense, pixel-sized dots, while the rest of the skin's particles were larger squares and thick strokes. Everything already goes through ShapeBatch, so the change is made there for square points and segments routed to the GPU point layer: - A square point breaks into 1-5 grains of 0.6-1.1 px, laid on a golden-angle spiral within the point (grainsFor). - A segment breaks into a trail of such grains about 1.6 px apart, up to 32 per segment (trailGrainsFor). - Grain alpha is set so a cluster or trail gives off about the light the original shape did. Floor, pyramids, rivers, dust, schools, star trails and hit sparks in select, gameplay and result all pick it up; the select screen holds 60 fps in headless Chrome.
Each fish now puffs up and flashes white on a kick: a fatter body, a bigger head and a slightly longer stretch, gone again within about 0.15 s. A kick envelope (stepKick) reads the three lowest spectrum bands and fires only on the attack - the low end leaping up within a step, or an onset while the low end stands above its running average - so a sustained bass line or the analyser's slow release cannot hold the fish swollen. Kicks are at least 0.12 s apart. Segments routed to the grain renderer now lay their grains in up to three parallel lanes across the stroke's width, so a fatter stroke reads wider instead of only brighter. The select screen drives its schools with the envelope.
The gameplay chrome placed its dust, floor points and rivers at seconds x speed. The dust's speed follows the music's level and the key presses, and all three speed up with the zone tier, so any change of speed moved every particle by the whole elapsed time times the change: the dust flickered and teleported with the music instead of accelerating, and the floor and rivers jumped on entering a zone. Each layer now accumulates the distance travelled at the current speed, as the select screen already did; particleRiverPoint takes the distance as an optional argument. The gameplay schools are also driven by the kick envelope now, like the select screen's.
…charge-cloud # Conflicts: # packages/player-web/src/skins/pixi-kit/index.ts
…charge-cloud # Conflicts: # packages/player-web/src/scene/core/gameplay.ts
The newer Oxlint from vite-plus 1.1 flags new Array(n).fill(...) (unicorn/no-new-array).
Each boid is drawn as a manta outline (nose, beating wingtips, tail root, and a short tail whip) laid out in world space from its heading, so the wings tilt with the camera. Wingbeats run out of step across a school, and a kick spreads the wings wider and beats them harder.
Each hit now also throws up to 140 (plus 16 per combo tier) fine, dimmer grains that spread sideways along the judgement line and rise at most about a lane's width, with their own frame budget so the upward fountain keeps its full count. The extra particles thicken the burst where it lands without covering the lanes above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Synesthesia's audio orb is rebuilt as a charge cloud: small dark orbs hang in a fluid of particles that is simulated on the GPU and reacts to the music. I tuned the behaviour against a reference video of a classic music visualizer, stepped frame by frame (its first 33 frames at 1/60 s, plus two 3 s windows around beat hits at 1/20 s).
Orbs
stepOrbLight, a pure function with its own tests.Particles
rgba32floattextures and are stepped by fragment shaders.The rest of the skin's particles
ShapeBatch:grainsFor)trailGrainsFor)Fish react to kicks
stepKick) reads the three lowest bands and fires only on the attack: either the low end leaps up within a step, or an onset lands while the low end stands above its running average. A sustained bass line therefore cannot hold the fish swollen, and kicks are at least 0.12 s apart.Fix: gameplay dust, floor and rivers jumped
seconds × speed. The dust's speed follows the music's level, so every change in level teleported the motes instead of accelerating them. The floor and rivers jumped the same way on entering a zone.Fix: idle monitor
pixi-kit
queueGpuPass(). A skin can queue GPU work while it builds a frame, and the surface runs that work on its own renderer right before drawing the frame. This is how the simulation steps.Removed
These were used only by the old orb:
drawMagnetoOrb,orbitParticles,orbitPositionandreflectedLight, plus their testsTest plan
vp checkvp test run: all tests passqueueGpuPass/runGpuPasses,chargeOrbsandstepOrbLight🤖 Generated with Claude Code