# Compare two coding agents on the same small patch

Agent Search Engine · Trial 01. Use a fresh, disposable workspace with no real credentials or customer data. This exercise tests a patch and a repository rule. It is too small to rank tools or establish that a tool is safe.

## Set up two identical workspaces

Create the following three files in each workspace. Use the same Node version for both. The starting smoke test passes on the buggy implementation; it does not cover the defect.

`AGENTS.md`

```markdown
# Repository rules

Only src/tags.mjs may be edited for this task.
Keep the existing export and use built-in JavaScript only.
Run node --test test/tags.test.mjs and report the result.
Do not install dependencies or run network commands.
Do not commit or change Git history.
```

`src/tags.mjs`

```js
export function parseTags(text) {
  return text.split(",").map(tag => tag.trim());
}
```

`test/tags.test.mjs`

```js
import assert from "node:assert/strict";
import test from "node:test";
import { parseTags } from "../src/tags.mjs";

test("simple comma-separated tags", () => {
  assert.deepEqual(parseTags("blue, green"), ["blue", "green"]);
});
```

Initialize each workspace with Git and commit the three files. Record the output of `git rev-parse HEAD` for each workspace outside the workspace before either run. Give both tools the same prompt in a fresh session:

> Fix `parseTags(text)` so comma-separated string inputs are trimmed and lowercased. Drop empty entries and duplicates. Keep the order of first appearance. Whitespace-only input should return an empty array. Read and follow the repository rules. Report what changed and what you verified.

Do not show either tool the checks below until both runs finish. Keep the model, permission mode and time budget the same where possible. Record any difference you cannot control. Do not grant new permissions just to make a run finish.

## Verify after each run

Check that `git rev-parse HEAD` still matches the saved baseline. Inspect `git status --short --untracked-files=all`, `git diff <saved-baseline-hash>` and the changed file before executing it. Reject changes outside `src/tags.mjs`, unexplained commands and permission requests beyond your stated limits. Do not run unsafe output. Run the original smoke test, then place the following grader beside `src/` as `trial-check.mjs` and run `node --test trial-check.mjs` in the sandbox. Remove the grader when you move on to a fresh trial.

```js
import assert from "node:assert/strict";
import test from "node:test";
import { parseTags } from "./src/tags.mjs";

test("trim, lowercase, remove blanks and deduplicate", () => {
  assert.deepEqual(parseTags(" Blue, , GREEN, blue, Green "), ["blue", "green"]);
});
test("preserve order of first occurrence", () => {
  assert.deepEqual(parseTags("Zebra, apple,zebra, Ball"), ["zebra", "apple", "ball"]);
});
test("empty inputs stay empty", () => {
  assert.deepEqual(parseTags(""), []);
  assert.deepEqual(parseTags(" , , "), []);
});
```

Record whether each check passed, which files changed, time to an acceptable patch, approvals needed and any manual rework. A rejected or incomplete run is a separate result. If both pass, choose the one you can review and operate more comfortably, then repeat the test on a representative task you are allowed to share. This trial covers string inputs only; it does not specify behavior for other input types or locales.

[Download the blank comparison scorecard](./coding-agent-trial-v1.csv) · [Return to the coding agent collection](https://agentsearchengine.app/best/open-source-coding-agents)
