Image FX#

onejs-unity/fx makes a texture something you can hold, chain operations on, and hand to an element. Generate a shape, load a photo and grade it, blur a mask, composite two layers: all from JavaScript, all on the GPU.

Related: Shader Effects (per-pixel effects into an element background), GPU Compute (raw compute shaders)

Quick Start#

import { image, useTexture } from "onejs-unity/fx"
import { View } from "onejs-react"

function Badge() {
    const tex = useTexture(() =>
        image.sdf(200, 200, "hexagon", { r: 0.36, rounded: 0.04 })
            .outline(6, [1, 0.42, 0.06, 1], "luminance"),
        [])

    return <View style={{ width: 200, height: 200, backgroundImage: tex }} />
}

No art, no shader file, no C#. That is a hexagon with an orange ring, drawn on the GPU.

The one idea#

Nothing runs until the chain is rendered. A chain is a description, not a sequence of dispatches, and that single decision is what makes it fast:

• A run of per-pixel operations becomes one draw, not one per operation.
• The whole chain crosses to C# once, as a single numeric buffer.
• Intermediates come from a pool, so a chain costs about one allocation rather than one per step.

Every operation returns a new image and none of them writes into its input, so a chain can be branched safely:

const base = image.load("photo").grayscale()
const warm = base.ramp(warmStops)   // these two do not
const cool = base.ramp(coolStops)   // interfere with each other

Sources#

A chain starts with one of these.

image.noise(512, 512, { scale: 6, octaves: 4, seed: 7 })
image.gradient(512, 512, [
    { color: [0.1, 0.2, 0.9, 1], at: 0 },
    { color: [1, 0.4, 0.1, 1], at: 1 },
], 45)
image.sdf(256, 256, "star", { r: 0.38, points: 6, rotation: 15 })

sdf takes the same 42 shapes and the same options as fx.sdf in TextureFX, including rounded and onion.

Operations#

Any operation that takes an operand accepts a number, a [r, g, b, a] array, or another image:

img.multiply(0.5)                   // every channel
img.multiply([1, 0.8, 0.6, 1])      // per channel
img.multiply(otherImage)            // per pixel

ramp colours by luminance, which is what turns a greyscale field into an image:

image.noise(512, 512, { scale: 5 }).ramp([
    { color: "#080d33", at: 0 },
    { color: "#e6591a", at: 0.55 },
    { color: "#fff2bf", at: 1 },
])

A colour is a hex string (#rgb, #rrggbb, #rrggbbaa) or an [r, g, b, a] tuple in 0..1, read as sRGB the way CSS reads it: #808080 is the grey that swatch shows. Gradients and ramps interpolate between stops in sRGB, so a white to black gradient is an even ramp, and a mask made from one has perceptual values (its midpoint is about 0.21, not 0.5). A stop can carry its opacity separately as alpha, so { color: "#260000", alpha: 0, at: 0 } reads as what it is. A list of bare colours spreads them evenly, so ramp(["#000", "#f00", "#fff"]) is the three-stop version above. gradient takes stops the same way, and both accept a readonly list, so a constant declared as const passes straight in.

threshold(low, high) keeps what lies between the two and stretches it to 0..1, which is how a noise field becomes flames or a blurred shape becomes a mask. It is levels without the gamma, under the name people reach for.

blend covers all 27 Photoshop modes: normal dissolve darken multiply colorBurn linearBurn darkerColor lighten screen colorDodge linearDodge lighterColor overlay hardLight softLight vividLight linearLight pinLight hardMix difference exclusion subtract divide hue saturation color luminosity.

useTexture#

useTexture(build, deps) builds a chain, renders it, and returns the texture. It rebuilds when deps change and releases the previous chain first, which is the part you would have to remember by hand:

function Swatch({ hue }: { hue: number }) {
    const tex = useTexture(() =>
        image.gradient(128, 128, stops, 90).hueShift(hue),
        [hue])

    return <View style={{ width: 128, height: 128, backgroundImage: tex }} />
}

It releases everything the callback caused to be created, including an operand built inside it:

const tex = useTexture(() => {
    const mask = image.sdf(256, 256, "star", { r: 0.4 })  // released too
    return image.noise(256, 256).blend(mask, "multiply")
}, [])

An image created outside the callback stays yours to dispose. The rule is that the hook owns what it caused to exist, not what it merely used.

Do not build a chain in the render body. image.noise(...) inside a component body runs on every render and allocates a target each time. Either hook it, or build it at module level where it happens once.

canvas#

Every source takes a size, and a program builds all of them at one size. canvas(size) decides it once:

const canvas = fx.canvas(512)
const mask = canvas.sdf("egg", { r: 0.2 }).blur(60)
const field = canvas.noise({ type: "turbulence", scroll: [0, -0.2] })

It is the same image factory with the size baked in, and useAnimatedTexture takes one in place of a width and height.

useImage and useAnimatedTexture#

useImage(build, deps) is useTexture for a chain you want to keep using as an operand: a mask, an envelope, a lookup. It builds synchronously, so the result is always an Image, never null, and it is released when deps change or the component unmounts.

useAnimatedTexture(w, h, build, deps) rebuilds a chain every frame into one stable texture, so the element is assigned once and nothing re-renders per frame. build receives seconds on the frame loop's clock, and it is always the function from your latest render, so it can read state directly. A noise built inside it can pan itself with scroll, in noise space per second:

const mask = useImage(() => canvas.sdf("unevenCapsule", { rBottom: 0.2, rTop: 0.01 }).blur(60), [])

const fire = useAnimatedTexture(canvas, () =>
    canvas.noise({ type: "turbulence", scale: [0.4, 0.3], scroll: [0, -0.2] })
        .multiply(mask)
        .threshold(0.1, 0.35)
        .ramp(["#26000000", "#b30f008c", "#ff4705eb", "#ff9e14", "#ffed9e"]),
    [mask])

<View style={{ width: 512, height: 512, backgroundImage: fire }} />

Outside an animated build, scroll is a still at time zero. Every type is fBm, octaves layers each lacunarity times finer and gain times quieter; the types differ in the base and in how the octaves combine. "value" and "simplex" sum signed octaves and read as cloud. "turbulence" sums the absolute value of each simplex octave, so the zero crossings stack into the veins and licks of fire, smoke and marble. "ridged" makes that crease bright and squares it, for mountain ridges, lightning and cracks. None of them needs tuning numbers.

Performance#

Operations fuse. This is one draw, not four:

img.multiply(0.15).add(0.5).pow(2.2).clamp(0, 1)

Fusion stops at three things, none of which you have to think about: a full window (16 operations), a second texture operand, and any spatial or filter operation. Those take passes of their own, because they read the image differently: a spatial operation moves the coordinate before sampling, and a filter reads a whole neighbourhood.

Blur has no radius limit. A wide blur costs passes, not quality: past what one pass can sample, the pass repeats rather than spreading its taps.

Platform support#

This runs everywhere OneJS does, WebGL included, because it draws with fragment shaders rather than compute shaders. WebGL2 has no compute shaders at all, so a compute-based pipeline could not run in a browser.

Gotchas#

load reads from Resources. The path is relative to a Resources folder and carries no extension, so "art/portrait", not "Assets/Art/portrait.png". Paths that work with <Image src> do not work here.

Values are not clamped. Targets are floating point, so multiply(2) on white really does give you 2.0, which is what makes the maths composable. Call saturate() before handing a result somewhere that expects 0..1.

outline needs to know where the shape is. A loaded sprite keeps its shape in alpha; sdf and noise put theirs in rgb and leave alpha at 1. The default reads alpha, so pass "luminance" for the generated sources or you get an empty ring.

image.load("sprite").outline(4, black)                  // alpha
image.sdf(256, 256, "heart").outline(4, red, "luminance") // rgb

See Also#

• Shader Effects for effects that animate into an element's background
• Vector Drawing for paths and strokes
• GPU Compute for data-parallel work on buffers