Skip to content
Sarj Sarj Sarj Standards

source-coupled-test

Disallow raw repository source text as a test oracle; parse or execute the artifact instead.

rawSourceOracle Rule source Tests Catalog JSON

Why

Substring and regex checks can pass on comments or unreachable configuration and fail after behavior-preserving formatting changes.

Fix

Parse the artifact, execute its validator, or assert on another runtime contract.

Examples

Before — flagged Do not prove workflow behavior with a regex
src/policy.test.ts
import { readFileSync } from "node:fs";
test("policy", () => {
  const source = readFileSync("workflow.yml", "utf8");
  expect(source).toMatch(/permissions/);
});
After — preferred Assert on parsed policy behavior
src/policy.test.ts
import { readFileSync } from "node:fs";
test("policy", () => {
  const policy = JSON.parse(readFileSync("policy.json", "utf8"));
  expect(validate(policy)).toEqual([]);
});