CloudBlogDocs

Deterministic Desloppification

Jakob Heuser

On October 8, Harry Solovay posted six coding traits for Effect: use Effect's modules over the built-ins, use Effect.retry with a Schedule instead of a hand-rolled loop, provide shared layers once at the entry point, and three more like them.

Traits like these are markdown files an agent reads on every run. The flaw in a decision model is it's still not absolute. Nobody wants a test that passes 80% of the time. My question in the thread was about order:

So I tried it, with every trait as a deterministic rule first and a model only for what the rules leave over.

Six traits, nine rules

I started from an empty repo with a blank CLAUDE.md, installed Taskless with npx @taskless/cli, and handed Claude the traits one at a time, asking for a rule with test cases for each. Six traits went in and nine ast-grep rules came out, because three of the traits were really asking for two things.

Then I wrote my own fixtures without looking at Claude's tests. src/bad.ts has 13 violations across all six traits, and every one was flagged. src/good.ts has the compliant version of each and came back clean.

Here's the retry trait in both files. The hand-rolled loop in bad.ts sleeps and catches until it gives up:

export async function t5a() {
  for (let i = 0; i < 5; i++) {
    try { return await client.run() } catch { await new Promise((r) => setTimeout(r, 2 ** i * 100)) }
  }
  throw new Error("gave up")
}

good.ts uses Effect.retry, with a bound on attempts and a filter on which failures get retried:

export const g5 = g4.pipe(Effect.retry({ times: 3, while: isTransient, schedule: Schedule.exponential("100 millis") }))

no-manual-retry flags the first because it's a loop that both sleeps and catches a failure. The second passes, and so does a polling loop in good.ts that sleeps without catching anything.

What is covered

Trait

Rules

Coverage

Effect modules over built-ins

prefer-effect-module-utilities, prefer-mutation-over-copy-reassign

Solid

(v) => v.id over ({ id }) => id

no-destructured-selector-callback

Solid

No Effect.gen wrapper without effectful work

no-unnecessary-effect-gen

Solid

Effect.promise directly at Promise boundaries

no-effect-promise-wrapper

Solid

Effect.retry and Schedule over manual loops

no-manual-retry, bounded-effect-retry

Solid, apart from custom sleep helpers

Shared layers at the outermost boundary

provide-layers-at-entry-point, no-reprovide-captured-service

Partial

Four of the six are fully covered, and they're the kind of rule teams usually leave in markdown. I got the retry case (mostly) covered, but layer traits go beyond what static analysis can do.

Testing assumptions

Before I wrote any fixtures, Claude listed the cases it didn't think its rules could catch. I put each one in src/probe.ts, and Claude was right about all of them. They don't need the same fix, though.

The first kind needs types. userRepo.find("42") gets flagged as an array method, because a rule that can't see types can't tell an Effect service from an array. This is the gap from Taskless, ty, and the Type-Shaped Gap, and the answer is the same: run a type checker next to the rules.

The second kind needs more than one file. A generic fromPromise helper gets flagged where it's defined, but a call site importing it from another file goes through. A retry loop that sleeps with a custom pause() helper also goes through, because sleeps are recognized by name and pause is defined somewhere else. Both are answerable by following a symbol to its definition, which is the "follow symbols" extension Harry mentioned in the thread, and it's what Taskless runtime rules are for. The ty post's runtime rule walks a TypedDict's base classes across files the same way. I didn't build one for this repo, so these two are still open.

The third kind needs intent, which no file or type can supply. The layer trait says to provide shared layers once, at the entry point. That depends on whether a layer is shared, and syntax can't tell you that.

Determinism first, model second

The repo has three Effect.provide calls outside the entry point:

// src/bad.ts
.pipe(Effect.provide(Database.layer))           // shared layer, provided per call
// src/probe.ts
q.pipe(Effect.provide(Database.layerFromEnv())) // shared layer, built by a call
// src/good.ts
g3.pipe(Effect.provide(Tenant.layer(tenantId))) // scoped to one tenant on purpose

The rule flags the first one. The other two look the same to a syntax tree, since each provides a layer built by a function call, and a layer built from an argument usually means local configuration. So the rule passes both. One is a violation and the other is the compliant example.

This is the narrowing I was asking Harry about. ripgrep finds every file that calls Effect.provide. An ast-grep capture pulls out each call outside the entry point, along with the layer expression and the tag it provides. A second capture reads what src/bin/main.ts already provides at the top, which is Database.layer. That's all deterministic, and it already settles probe.ts: the entry point provides Database, so providing it again is the violation.

What's left is Tenant.layer(tenantId). Nothing at the entry point provides Tenant, and whether it should is a question about how the program is meant to work. No amount of determinism answers that. The meaning, and whether this call violates it, has to be inferred from the code around it, and inference is what a model is for.

That one call goes to a decision model, along with one typed question: is this layer meant to be shared across the program, yes or no? A model like Jev is built for that shape of question, answering cheaply with very little variance between runs. LangChain's test was small (five cases, one agent), so it's early.

Out of a whole codebase, the model sees one call and one question. Cost is what decides whether a check survives in CI, and a model that only gets the leftovers is cheap enough to run on every pull request. Its answer shows up as a review hint on the PR, next to the rule failures.

Try it

The whole experiment is up on GitHub so you can play around with Taskless' rule generation.

git clone https://github.com/theCodeDrift/effect-traits-taskless
cd effect-traits-taskless
npm install
npx @taskless/cli check

To run one rule:

npx @taskless/cli check --rule no-manual-retry

The rules live in .taskless/rules/sg/, each with a .tests/ folder of cases that should and shouldn't match, like the one for no-manual-retry. The traits came from a screenshot, and the last one was cut off, so the end of the layer trait is my best reading of it.