What the painters were given

Workflow: crop renders, checkpoints, speed (stream 3, branch workflow)

notes/workflow.md · 466 lines · Notes

Given to 11 painters:

Workflow: crop renders, checkpoints, speed (stream 3, branch workflow)

Goal: a fast feedback loop, so a painter can afford tiny details. Before this branch, the only way to see a detail at full resolution was a whole 3200px render (friedrich_moonrise_valley: 107–145 s here, under a load average of 20–130 from the other streams), and every run repainted every stage.

What landed:

  1. Crop renders: --crop x0,y0,x1,y1 (units) paints only that window, at the run's full resolution. They are fast, and close to a whole render (not exact; the reasons are below).
  2. Stages and checkpoints: --ckpt saves the complete canvas state after every stage and --resume <stage> starts from one. Resuming is bit-exact, and stale checkpoints are refused.
  3. Speed, with the output unchanged: an exact per-column memo for 1-D profiles (cuts mask cost in crops) and a dependency-ordered tile scheduler in place of phase barriers.
  4. Not done: --frames (the "watch Claude paint" time-lapse).

All timings below were taken on the shared 10-core machine with the other four streams running (load average 17–130). They are wall-clock times and noisy; compare within a row, not across rows.

The fast loop

cargo paint friedrich_moonrise_valley -- --full --crop 280,440,460,580 --ckpt   # once, 14 s
cargo paint friedrich_moonrise_valley -- --full --crop 280,440,460,580 --resume mist
                                                                          # 1.6 s per iteration
moonrise, the two figures on the ledge time
whole canvas, 3200px 86–145 s (load 14–110)
crop 180×140 units (576×448 px) at 3200px, margin 40 13–14 s
the same crop, resumed from "mist" 1.6 s
whole canvas, 1000px preview 16–21 s
1000px preview, resumed from "mist" 1.1–2.7 s

Painter API

paintings/src/run.rs (module docs have the full list):

let o = Run::new("my_painting");
let mut rng = Rng::new(o.seed);
let mut c = o.canvas(|| st.prepare(o.width, 1.4, o.seed)); // loaded instead on --resume
let f = c.frame();                                          // whole-canvas frame for masks
let ridge = f.per_column(|x| h * 0.6 + 12.0 * n.get(x, 0.0)); // optional: exact memo
let sky = Mask::from_fn(f, |x, y| ...);                     // outside stages: always runs
if o.stage("sky", &mut c, &mut rng) {
    c.work(&sky, ...);                                      // skipped when resuming later
}
if o.stage("land", &mut c, &mut rng) { ... }
o.finish(&mut c, &mut rng, &Finish::aged(st.relief)); // or o.end(&mut c, &mut rng)

Crop renders: how they work

Frame gained a window: w × h are the pixels a buffer holds, at origin (x0, y0) in the whole full_w × full_h canvas. width() and height() stay whole-canvas units, and all px↔unit conversions go through Frame::{index, ux, uy, whole, clip, whole_index}. For a whole canvas every formula reduces to the old arithmetic, so the golden fingerprint did not move.

Why a crop is close but not exact

A mark's look depends on the brush's history along its whole path: paint it picked up and ridges it ploughed ahead of itself before entering the window, and earlier strokes of the same tile's brush. That history depends on the canvas outside the window, which a crop doesn't have. Long strokes (grounds of 250–600 units, mist of 80–200, badger fusing) carry the most history. Short strokes, dabs, the rigger, figures, trees and cracks match well. Resolution of the numbers below: max/mean over RGB, in 8-bit levels.

moonrise figures crop vs the same region of a whole render mean pixels > 8 levels max
1000px, margin 12 1.47 4.5% 39
1000px, margin 40 1.11 2.5% 40
1000px, margin 80 0.86 1.8% 42
1000px, margin 160 0.56 0.9% 41
3200px, margin 40 1.34 4.5% 56
3200px, margin 80 1.01 3.0% 48

Engine test crop_matches_whole checks this: linen and ground are exact, brushwork is close, and the difference falls as the margin grows. For scale: the palette history bug (below) alone moved the whole painting by a mean of 1.6 levels. The 1000px preview and the 3200px render differ in kind: work accepts a stroke center by reading the mask at that point, masks at another resolution accept slightly different centers, and from there the RNG sequence and every later mark differ.

Evidence (all in [local path redacted], viewed through scripts/peek):

In a crop, c.sample(), c.pixels() and c.surface_um() see only the window (sample clamps to it).

Checkpoints

A bug found by exactness: Palette::mix depended on history

The first resume from "sky lay" differed from the straight run: 0.02% of pixels, by 1 level. The cause was that Palette::mix cached mixtures under a quantized OKLab key but searched for whichever color reached the bucket first. So a pile's mixture depended on which colors had been mixed before, and a resumed run with an empty cache mixed differently. It now searches the bucket's center, which makes the result a pure function of the key. This changes output: per-stroke piles move within one 1/400 OKLab bucket, moonrise moves by a mean of 1.6 levels, and it looks the same (compared side by side). The golden was re-recorded (commit 3ca1c69). palette.rs is stream 1's file. The change is one line in mix, and stream 1 should keep the property "mix is a pure function of its input".

Speed

Profile (macOS sample; moonrise at 1000px, and the 3200px ground stage): bristle::drag_on/exchange is about 85% of busy CPU. The rest is dry and settle, the Mixbox conversion and crack rastering. Rayon workers are idle most of the time. Phases hold only 1–6 tiles (long strokes make big tiles), the badger passes over the sky run 1–3 tiles at a time (the sky stage takes as long on 1 thread as on 10), and the single drags (clouds, wisps, 100+ spruces) are sequential. In a crop at 3200px, the whole-canvas masks dominated instead: Perlin noise in the painting's mask closures.

Wins taken, all with bit-identical output (golden unchanged; moonrise at 1000px byte-identical to the post-palette-fix render):

Before/after, moonrise:

before (main) after
1000px whole 18.4 s (load ~20) 15.9–16.7 s (load ~46)
3200px whole 107 s (load ~40); 145 s after the palette fix (load ~110) 86 s (load ~14), byte-identical to the post-fix render
3200px crop, figures — (not possible) 13.3 s
3200px crop, resumed from mist — 1.6 s

What I'd do next for speed, in order: (1) exact micro-optimizations in exchange, which needs line-level profiling; I didn't have line tables. (2) Masks that evaluate lazily outside a crop window, so crops stop paying for whole-canvas masks. (3) Anything faster for long badger passes changes the physics order, so it is a design decision, not a safe win.

Merge notes for the integrator

Known issues

After merging main (color, strokes, stipple, form, motifs)

Verification after the merge (1000px, margin 40, diff in 8-bit levels):

crop vs the same region of a whole render mean > 8 levels max
moonrise figures 280,440,460,580 1.43 5.1% 96
study_stipple sky, both passes 300,150,500,300 0.59 0.003% 10
study_stipple mist over the ridge 300,560,500,625 1.98 6.0% 28
study_stipple same, margin 250 (before the rng fork) 0.64 0.3% 13
study_stipple ridge only (--stop ridge) 0.005 0% 1
study_form boulder 150,100,350,250 0.63 2.1% 61
study_form ranges 650,450,850,600 0.60 0.35% 33
study_form cliff 100,450,300,600 0.19 0.007% 18

The mist stipple is margin-limited: a dip serves 16 touches, so touches outside the window affect the load of those inside it. study_form crops take 11–12 s against 31 s whole, because its form fields (SDFs per pixel) are computed over the whole canvas, like masks.

Resuming stays byte-identical: moonrise from "sky lay" (wet), "mist" and "figures", and study_stipple from "pass 1".

Fixes from amnesia round 2 (branch fixes-ux)

From the painters' friction lists (notes/amnesia2.md items 9, 10, 11, 12). Resume is still byte-exact (study_stipple at 400px: whole vs resumed from "pass 1", cmp identical); golden unchanged.

Stage names and flags (paintings/src/run.rs):

Closures (winter #13, coast #7, mountains #6): Fbm is Copy. Its octave tables are built once per (seed, octaves, persistence) and shared (leaked, about 1 KB per octave), and its output is unchanged (tested against the noise crate). Frame::per_column returns &'a impl Fn(f32) -> f32, a Copy reference: it is still called ridge(x), and its table lives for the rest of the program. A move closure that captures only these is Copy too, so one profile or noise can feed a mask and any number of color closures with no let n = &n;. Mask and Form are still not Copy (capture &mask).

Mask::roughen (winter #10, #16): amount and edge are now canvas units. The contour moves by about amount units (fbm, period units), and the new edge ramps over edge units. It works through the signed distance to the 0.5 contour, so a hard Shape mask roughens like a soft one. Pixels farther than amount + edge from the edge keep their value. The old behavior (mask-value units, a no-op on hard masks, edge = 1 turned 0 into 0.16 everywhere) is gone. No built painting used it. Test: mask::tests::roughen_moves_hard_edges_in_units (at two resolutions).

form.rs docs: Sdf::block's size is the whole extent (it spans c ± size/2; an example seats a tor on the turf line), and Sdf::ellipsoid's r is radii. Every constructor and Ridge builder now states its units. The docs also say Ridge reaches depth below its crest whatever stands in front of it (mountains #9).

Open bugs

Fixed: a gesture with a NaN point laid one dab and said nothing (coast #17)

Fixed after merging tip: Gesture::validate and Touch::validate reject non-finite numbers, and drag_on/touch_on (every painting path) panic with e.g. "Gesture point 10 is not finite: (505.5, NaN)". The history:

Symptom (coast painter): an 11-point U-shaped drag (a coil of rope, Tool::rigger(0.5)) laid nothing, while straight strokes nearby did.

Cause: the painter's points. paintings/fresh2/fresh2_coast.rs builds y = py + 1.3 + drop * a.sin().powf(0.8) for a = PI * k / 10. In f32, PI.sin() is −8.7e-8, so powf(0.8) is NaN at k = 10. The only half that painted (bottom to left end) was the only one without the last point. Engine side (bristle.rs, drag_on): the arc length total is NaN, nsteps = ((total / step).ceil() as usize).max(1) is 1 (NaN casts to 0), and (k * step).min(total) ignores the NaN. So the brush takes a single step at the start and lifts: 12 pixels at 1000px, against 106 for the same U with a finite last point. footprint doesn't catch it either, since f32::min/max skip NaN. Tool::validate checks the tool but nothing checks the gesture.

Fix to apply after merging tip (bristle.rs is that stream's file): in Canvas::drag (and touch/Touch, and in Canvas::work's planned gestures via footprint_checked), reject non-finite points the way an invalid Tool is rejected. Panic with the point's index and value, e.g. "Gesture point 10 is not finite: (505.5, NaN)". Also check that pressure, attack, release and swell are finite. Repro: crates/paint/tests/curved_drag_nan.rs. With the fix in, two tests remain: a_nan_point_is_an_error (the panic names "point 10", so the coil's only NaN is its last point) and the finite-points control.

Tests: two tiers (Round 6)

Source