Preference, Effort, and the Limits of Brain Reading

December 8, 2025 · 3 min read · Send your thoughts via twitter or mail.

Preference Discovery & Taste Modeling

Many “I can’t execute my vision” complaints are actually “I don’t know what I’ll want until I see it.”

• People overestimate the fidelity of their internal imagery; what feels like detailed vision is usually a loose pointer that only sharpens once externalized.
• No amount of neural decoding or behavior modeling can skip the need to actually perceive candidate artifacts.
• Discovering taste has irreducible sampling cost: preferences must be inferred from reactions to concrete variants, not from introspection.¹
• Because the full preference surface is exponentially large, the brain lazy samples it: generate, evaluate, refine. It doesn’t store a detailed global map.
• When your own taste is high uncertainty, improving what you generate matters more than how fast you can tweak it.

Taste can be operationalized as a context-dependent decision boundary over pairs of options: A > B in Context C.

[truism] Preferences behave like a patchwork of local, context-bound objectives, not a single stable global utility function.

[derived] Supposed aesthetic universals like the golden ratio are better understood as historically reinforced conventions than as fixed perceptual laws. Weak cross-cultural tendencies exist, but culture explains more variance.²

[speculative] Rater-fingerprinting: Learning rater embeddings and conditioning generation on them lets models inhabit specific taste profiles rather than collapsing to a bland mean.

Creative Effort & Search Geometry

The effective difficulty of a creative domain is mostly set by three knobs:

• Oracle quality: how fast and honest feedback is
• Gradient locality: how predictably tiny edits move quality
• Compositionality: how safely you can change one part without wrecking the rest

Different creative media, genres, and modalities vary mainly in feedback latency, noise, and compositional coupling. Not in the underlying search algorithm.

• “Levels” of work (macro, meso, micro) are not fundamental buckets; they emerge from how oracle latency and coupling change as you zoom in or out on the artifact.
• What we call “execution” is just exploration inside a narrow, already-committed neighborhood of the search space.

[truism] As you push a creative work toward your local optimum, effort grows **superlinearly** relative to perceived improvement.

[derived] High-end craft lives on narrow peaks where most small perturbations decrease quality.³

[derived] Finer precision always comes with a tax: as tolerated error shrinks, Fitts' Law and rate-distortion guarantees push more effort into slow, high-attention micro-corrections.

[truism] Creation plays out as nested generate→evaluate→refine loops at multiple resolutions, not as a clean "idea" → "execution" handoff.

[truism] On sharply peaked quality landscapes, progress relies more on cached motor/judgment heuristics than on single-step insight.

[derived] Tools that keep gradients local (per note, per token, per stroke previews) flatten last mile cost; tools that couple many dimensions explode it.^d

[derived] Modalities with wide basins and slow or subjective feedback (like prose) make life easy for the slop demons.

Brain–Computer Interfaces & Cognitive Limits

Conscious deliberation is bottlenecked to ~10 bits/second at behavioral output.⁵ BCIs tapping motor cortex can exceed this (vendor benchmarks, unverified claim 200+ bits/s), but they’re reading signals structured by motor tasks. Remove the task structure and you get noise. Sampling earlier neural layers can decode action preparations 200–500ms ahead, but this reads the output of selection, not raw intentions. You don’t bypass deliberation.

For self-initiated actions, there’s nothing coherent to decode before a threshold crosses.⁶ For goal-directed actions, motor and parietal cortex do encode structured plans, but these are action affordances shaped by task context, not intentions in the everyday sense.

[derived] Read-only BCIs can only decode already-legible intentions, not bypass the need to externalize vague mental content.

[truism] Motor actions provide natural gating and discrete tokens (clicks, keypresses, gestures). That gating is a feature, not a bug: It separates deliberate intent from continuous neural signals in a way raw neural readout cannot.⁷

[derived] Learning depends on slow biochemistry like protein synthesis. Even perfect BCIs cannot compress complex skill acquisition into fast "downloads". They can only steer where practice goes.

[speculative] For non-paralyzed creatives, marginal benefit of read-only BCIs looks small.