Implicit state handling for p5.strands addons

[davepagurek]

Increasing access

I've been developing on the side an SDF renderer addon for p5. It currently looks something like this, as an API:

let mySDF

function setup() {
  createCanvas(windowWidth, windowHeight, WEBGL)

  mySDF = buildSDF(function() {
    sdfScene.begin()
    let sdf = distanceFunction(sdfScene)

    sdf.smoothUnion(30)
    sdf.push()
    sdf.translate(100 + 50 * sin(millis() * 0.001), 0, 0)
    sdf.sphere(40)
    sdf.pop()
    sdf.sphere(60)

    // TODO deal with materials etc here somehow
    sdfScene.dist = sdf.get()
    sdfScene.end()
  })
}

function draw() {
  background(0)
  orbitControl()
  noStroke()
  lights()
  fill('red')
  specularMaterial(200)
  shininess(300)
  mySDF.draw(200)

  push()
  fill('green')
  const angle = millis() * 0.001
  const r = 150
  translate(r*cos(angle), 0, r*sin(angle))
  box(50)
  pop()
}

However, if statements and for loops do not work with it. e.g. this will not work:

for (let row = 0; row < 4; row++) {
  for (let col = 0; col < cols; col++) {
    sdf.push()
    sdf.fill(68, 68, 74)
    sdf.translate(kX0 + col * pitchX, kY, kZ0 + row * pitchZ)
    sdf.roundBox(keyW, keyH, keyD, keyR)
    sdf.pop()
  }
}

However, this would:

let minDist = 10000
for (let row = 0; row < 4; row++) {
  for (let col = 0; col < cols; col++) {
    minDist = min(minDist, boxSDF(...))
  }
}

To a lesser extent, my p5.env library also needs this: the current color is also hidden state: https://github.com/davepagurek/p5.env/blob/241ab1d5d13bc44ba8dda4563e159c841c824f50/p5.env.js#L400 But other aspects of this library do work with if/else etc because you can use if/else for methods that input to the ones linked above that use hidden state.

The reason is that strands needs to convert control flow into its internal graph representation of your program, and to do that, it needs to know what gets modified in each part of the control flow. In the minDist example, the state updates are explicit so strands knows about it but can handle it.

However, the implicit internal state updates in my initial library design more closely mirror how p5 APIs work. If we can find a way to let addon authors make strands APIs that look more like regular p5, then it can help make strands feel more familiar and more p5-y/magical.

While the main point of strands is not to be "magical", and is about creating teachable moments with enough scaffolding to keep people unblocked, it is also a way that we can let more shader functionality be exposed in a similar mental model to the rest of p5 to make artists more productive. This would be most useful for that latter use case.

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

Feature enhancement details

The main approach I can think of is to let addons write a function that does explicitly return state, i.e it takes in the current state and returns a new one. In TypeScript terms:

type StateFunction<State> = (currentState: State, ...otherArgs: any[]) => State

...and then we mark the function in some way that would tell the transpiler to turn this, where these different state-updating functions look like they are reading some implicit internal state and return nothing:

for (let row = 0; row < 4; row++) {
  for (let col = 0; col < cols; col++) {
    push()
    fill(68, 68, 74)
    translate(kX0 + col * pitchX, kY, kZ0 + row * pitchZ)
    roundBox(keyW, keyH, keyD, keyR)
    pop()
  }
}

...into this, where it's all explicit:

let state = initialState()
for (let row = 0; row < 4; row++) {
  for (let col = 0; col < cols; col++) {
    state = push(state)
    state = fill(state, 68, 68, 74)
    state = translate(state, kX0 + col * pitchX, kY, kZ0 + row * pitchZ)
    state = roundBox(state, keyW, keyH, keyD, keyR)
    state = pop(state)
  }
}

Now this could look something like this, where a plugin registers a bunch of methods that operate on some state, with initialState being something special that we call at the start of a hook whenever we see one of these tagged methods used. API could look like this:

function myAddon(p5, fn) {
  p5.registerStrandsState(
    // State initializer
    function initialState() {
      return [{ minDist: 1000, fill: [255, 255, 255] }]
    },
    // Other methods mutating the state
    {
      push: (state) =>  [...state, {...state.at(-1)}],
      pop: (state) => state.slice(0, -1),
      fill: (state, r, g, b) => [...state, {...state.at(-1), fill: p5.strandsNode([r, g, b])}],
      // etc
    },
    // Other methods that just read the state but then do something else
    {
      getDistance: (state) => state.at(-1).minDist
    }
  )
}

...however I imagine we maybe want push/pop to globally work across state that different addons may add, so we maybe would want that to be generalized. The plugin API could be something like this:

function myAddon(p5, fn) {
  p5.registerStrandsState(
    // State initializer
    function initialState() {
      return { fill: [255, 255, 255], minDist: 1000 } 
    },
    'sdf', // unique key for this plugin's state
    // Other methods using the state. no push/pop this time
    {
      fill: (state, r, g, b) => [...state, {...state.at(-1), fill: p5.strandsNode([r, g, b])}],
      // etc
    },
    // Other methods that just read the state but then do something else
    {
      getDistance: (state) => state.minDist
    }
  )
}

...and then we auto-namespace addon state in the transpiled output:

let state = {}
state.sdf = __p5.strandsStateInitializers.sdf()
for (let row = 0; row < 4; row++) {
  for (let col = 0; col < cols; col++) {
    state = push(state) // Push and pop are still shared
    state.sdf = fill(state.sdf, 68, 68, 74) // This is scoped though
    state.sdf = translate(state.sdf, kX0 + col * pitchX, kY, kZ0 + row * pitchZ)
    state.sdf = roundBox(state.sdf, keyW, keyH, keyD, keyR)
    state = pop(state)
  }
}

Registering methods would both add them to p5.prototype but also add some extra properties to those functions, like how in #8817 we're starting to add .argTypes as an optional property of a strands function to help with FES and to allow arrays to be used in function signatures without them all being converted to strands vectors. We could call it something like .stateMetadata and we can check in the transpiler if this exists, and update it if so.

[bhabishnu]

In the public SDF source, pop() restores settings but keeps the accumulated shape result.

For shared push/pop, should addons mark which fields to restore, or provide save/restore functions? For example, restore fill while keeping the accumulated minDist.

[bhabishnu]

Also, would registerStrandsState support two independent envLight(...) helpers in the same hook?

[davepagurek]

Good point @bhabishnu, I suppose the analogy in that case would be like the canvas in regular p5: push/pop doesn't affect the canvas. One idea could be to force state to always have two parts, an ephemeral part that gets saved/restored, and a permanent/cumulative part? Maybe like type StateFunction<State extends { scoped: any; global: any }> = (currentState: State, ...otherArgs: any[]) => State so it always returns an object with those two parts, and you could modify either?

Also, would registerStrandsState support two independent envLight(...) helpers in the same hook?

With this API, the answer would be no, similar to how regular p5 doesn't have a way to distinguish two separate transformations being active at once. If we wanted to support it I suppose we could make a way to use it in "instance mode", similar to the initial SDF example and current p5.sdf code where you have to explicitly initialize it and then call the methods on an object? Do you have thoughts on how that might fit in?