Image FX#

onejs-unity/fx makes a texture something you can hold, chain operations on, and hand to an element. Generate a shape, load a photo and grade it, blur a mask, composite two layers: all from JavaScript, all on the GPU.

Related: Shader Effects (per-pixel effects into an element background), GPU Compute (raw compute shaders)

Quick Start#

import { image, useTexture } from "onejs-unity/fx"
import { View } from "onejs-react"

function Badge() {
    const tex = useTexture(() =>
        image.sdf(200, 200, "hexagon", { r: 0.36, rounded: 0.04 })
            .outline(6, [1, 0.42, 0.06, 1], "luminance"),
        [])

    return <View style={{ width: 200, height: 200, backgroundImage: tex }} />
}

No art, no shader file, no C#. That is a hexagon with an orange ring, drawn on the GPU.

The one idea#

Nothing runs until the chain is rendered. A chain is a description, not a sequence of dispatches, and that single decision is what makes it fast:

A run of per-pixel operations becomes one draw, not one per operation.
The whole chain crosses to C# once, as a single numeric buffer.
Intermediates come from a pool, so a chain costs about one allocation rather than one per step.

Every operation returns a new image and none of them writes into its input, so a chain can be branched safely:

const base = image.load("photo").grayscale()
const warm = base.ramp(warmStops)   // these two do not
const cool = base.ramp(coolStops)   // interfere with each other

Sources#

A chain starts with one of these.

Source	What it gives you
`image.load(path)`	A texture from `Resources`
`image.color(w, h, rgba)`	A flat colour
`image.blank(w, h)`	Transparent
`image.noise(w, h, opts)`	Scrolling fBm value noise, greyscale
`image.gradient(w, h, stops, angle)`	A linear gradient, up to 8 stops
`image.sdf(w, h, kind, opts)`	Any of 42 signed distance shapes, as a mask

image.noise(512, 512, { scale: 6, octaves: 4, seed: 7 })
image.gradient(512, 512, [
    { color: [0.1, 0.2, 0.9, 1], at: 0 },
    { color: [1, 0.4, 0.1, 1], at: 1 },
], 45)
image.sdf(256, 256, "star", { r: 0.38, points: 6, rotation: 15 })

sdf takes the same 42 shapes and the same options as fx.sdf in TextureFX, including rounded and onion.

Operations#

Group	Methods
Maths	`add` `subtract` `multiply` `divide` `pow` `sqrt` `clamp` `frac` `min` `max` `oneMinus` `remap` `saturate` `abs` `exp` `log` `modulo` `negate` `posterize` `reciprocal` `lerp` `smoothstep` `inverseLerp`
Colour	`grayscale` `brightness` `contrast` `saturation` `hueShift` `levels` `swizzle` `ramp`
Composite	`blend(operand, mode, opacity)`
Spatial	`transform` `tile` `flip` `crop`
Filters	`blur` `sharpen` `edge` `dilate` `erode` `outline`

Any operation that takes an operand accepts a number, a [r, g, b, a] array, or another image:

img.multiply(0.5)                   // every channel
img.multiply([1, 0.8, 0.6, 1])      // per channel
img.multiply(otherImage)            // per pixel

ramp colours by luminance, which is what turns a greyscale field into an image:

image.noise(512, 512, { scale: 5 }).ramp([
    { color: [0.03, 0.05, 0.2, 1], at: 0 },
    { color: [0.9, 0.35, 0.1, 1], at: 0.55 },
    { color: [1, 0.95, 0.75, 1], at: 1 },
])

blend covers all 27 Photoshop modes: normal dissolve darken multiply colorBurn linearBurn darkerColor lighten screen colorDodge linearDodge lighterColor overlay hardLight softLight vividLight linearLight pinLight hardMix difference exclusion subtract divide hue saturation color luminosity.

useTexture#

useTexture(build, deps) builds a chain, renders it, and returns the texture. It rebuilds when deps change and releases the previous chain first, which is the part you would have to remember by hand:

function Swatch({ hue }: { hue: number }) {
    const tex = useTexture(() =>
        image.gradient(128, 128, stops, 90).hueShift(hue),
        [hue])

    return <View style={{ width: 128, height: 128, backgroundImage: tex }} />
}

It releases everything the callback caused to be created, including an operand built inside it:

const tex = useTexture(() => {
    const mask = image.sdf(256, 256, "star", { r: 0.4 })  // released too
    return image.noise(256, 256).blend(mask, "multiply")
}, [])

An image created outside the callback stays yours to dispose. The rule is that the hook owns what it caused to exist, not what it merely used.

Do not build a chain in the render body. image.noise(...) inside a component body runs on every render and allocates a target each time. Either useTexture it, or build it at module level where it happens once.

Performance#

Operations fuse. This is one draw, not four:

img.multiply(0.15).add(0.5).pow(2.2).clamp(0, 1)

Fusion stops at three things, none of which you have to think about: a full window (16 operations), a second texture operand, and any spatial or filter operation. Those take passes of their own, because they read the image differently: a spatial operation moves the coordinate before sampling, and a filter reads a whole neighbourhood.

Blur has no radius limit. A wide blur costs passes, not quality: past what one pass can sample, the pass repeats rather than spreading its taps.

Platform support#

This runs everywhere OneJS does, WebGL included, because it draws with fragment shaders rather than compute shaders. WebGL2 has no compute shaders at all, so a compute-based pipeline could not run in a browser.

Gotchas#

load reads from Resources. The path is relative to a Resources folder and carries no extension, so "art/portrait", not "Assets/Art/portrait.png". Paths that work with <Image src> do not work here.

Values are not clamped. Targets are floating point, so multiply(2) on white really does give you 2.0, which is what makes the maths composable. Call saturate() before handing a result somewhere that expects 0..1.

outline needs to know where the shape is. A loaded sprite keeps its shape in alpha; sdf and noise put theirs in rgb and leave alpha at 1. The default reads alpha, so pass "luminance" for the generated sources or you get an empty ring.

image.load("sprite").outline(4, black)                  // alpha
image.sdf(256, 256, "heart").outline(4, red, "luminance") // rgb