Skip to content
KiweelyKiweelyKiweely

The Workday tenant test: method

On this page

Kiweely team, November 2026. Status: v1, written before any run. Anything changed after a run starts is logged at the bottom with the reason, and the runs it affects are named.

The question

Workday consultants already paste Workday work into ChatGPT, Claude and Gemini. This test asks one thing of every answer: when you do exactly what it says in a real Workday tenant, does the work come out right? Not whether the answer reads well. Each task has pass checks that are run in the tenant itself, the same way for every contestant.

Where it runs

  • One Workday tenant: Kiweely’s Workday Development tenant (a WCP Development Preview tenant with Workday’s fictional sample company and generated sample workers). No customer tenant, no real people’s data.
  • Every object a contestant creates carries its own suffix (below), so contestants never touch each other’s work and every result can be inspected afterwards. Nothing is cleaned up.

The contestants

Lane Code Product and model How it is used
LaneChatGPT CodeCG (suffix TT-GPT) Product and modelOpenAI GPT-6.2 Astra, reasoning effort high How it is usedOpenAI’s own Codex CLI (0.169.0) signed in with a ChatGPT plan, codex --search exec, read-only sandbox with no network, run in an empty folder. Web search on. It cannot see the tenant.
LaneClaude CodeCL (suffix TT-CLA) Product and modelAnthropic Claude Opus 5.5, effort high How it is usedAnthropic’s own Claude Code CLI (2.1.329) on a Claude plan, claude -p --safe-mode --strict-mcp-config, only the WebSearch and WebFetch tools, run in an empty folder. It cannot see the tenant.
LaneGemini CodeGE (suffix TT-GEM) Product and modelGoogle Gemini 4 Neon (High) How it is usedGoogle’s own Antigravity CLI (agy 1.5.13) signed in with a Google account, agy -p, web search allowed, commands denied, run in an empty folder. Its file-writing tool was not blocked by the deny list: on T3 and T7 it saved files into its own empty folder (see the changes log). It cannot see the tenant. Gemini 4 Neon is the model Google AI Pro subscribers get in the Gemini app from 18 Nov 2026 (third-party reports; Google’s own help pages were not checked).
LaneWorkday’s own agent CodeWD Product and modelWorkday Self-Service Agent (“Agent Chat” in the tenant’s top bar) How it is usedOpened in the tenant as the same administrator account; the prompt typed into it. Included because it is actually available in this tenant (checked 19 Nov 2026: asked “Can you create a calculated field or a custom report in this tenant for me?”, it answered that it cannot and pointed to the Workday tasks; screenshot evidence/workday-agent-check-2026-11-19.png).
LaneKiweely Vine CodeKV Product and modelKiweely Vine (current Dev build) running Claude Opus 5.5 at High effort How it is usedA clean test computer with Vine installed, connected to the same tenant. Vine works in the tenant itself. Same model as the Claude lane, so the difference between CL and KV is the product, not the model.

Isolation. Each client runs with nothing from the computer it runs on: Codex from its own home folder holding only its sign-in, with MCP servers, connected apps, browser use, computer use, plugins, image generation, the shell tool and memories switched off; Claude Code in safe mode (no CLAUDE.md, memory, skills, plugins, hooks or MCP servers) with only web search and page reading; Antigravity from a home folder holding only its sign-in (no rules, memories, knowledge items or MCP servers). Before the runs, each client was asked to list its tools and any instructions it had: Claude listed WebSearch and WebFetch and no instructions; Gemini listed no rules or memories; Codex listed no MCP servers or apps (its remaining built-in tools run in a read-only, offline, empty folder). Every transcript is checked afterwards for any tool use other than web search, and any found is reported with the result.

Why the vendors’ command-line clients rather than the chat websites: they are each vendor’s own client on the same subscription a consultant pays for, they use the vendor’s own system prompt and web search, and every turn is saved as a full transcript we can publish. Their default system prompts lean towards software work; that is the vendors’ choice, applied equally to all three. None of them is given files, a terminal into the tenant or any tool beyond web search.

The prompt

Every contestant gets the same task text, word for word, written as a consultant would paste it. {S} is replaced by the lane’s suffix (TT-CG, TT-CL, TT-GE, TT-WD, TT-KV) and nothing else changes. The prompt says what a consultant would know about their client (the organization names, the people, the data to load). It does not say which Workday tasks, functions or web services to use: knowing that is the work.

No system prompt, persona or preamble is added by us.

How a chat answer is applied

A chat answer is applied exactly as written by an operator working in the tenant through Workday’s own screens, step by step:

  1. The operator follows the steps in order, typing the names, values, formulas and choices the answer gives.
  2. The operator knows how to move around Workday (search for a task by name, open related actions, fill a prompt, click OK, Done or Submit, approve an item in their own inbox) but adds nothing of their own: no field, function, step or value the answer did not give, and no fixing of the answer’s mistakes.
  3. Where the answer leaves a business choice to the user, the operator takes it from the task text. Where it offers alternatives, the operator takes the first. Where it gives a file (a spreadsheet, a query, app files), the operator makes it exactly as given.
  4. When a step cannot be done as written (a task, field or option that does not exist, or an error), the operator stops and sends a follow-up to the same conversation: Workday’s exact message, or a plain description of what the screen shows instead (“Step 4 says choose the function X. The function list has no X; it shows A, B, C.”). No hints. At most two follow-ups, then the run ends where it stands.
  5. Every action is logged with its time and a screenshot (runs/<task>/<lane>/apply/actions.jsonl, shots/).

The operator is an AI agent (Claude, driving a browser through Playwright), not a person, so its applying time is recorded but is not a human time.

Workday’s agent (WD) is used the same way: the prompt is typed into Agent Chat. If it does the work, its result is checked. If it gives steps, they are applied as above. If it declines, that is the result.

How Vine runs

A clean test computer, a fresh Vine installation, Vine connected to the tenant (the same administrator account). A tester agent playing the consultant sends the prompt word for word as its first message, then only: answers Vine’s questions with facts from the task text, approves the tenant actions Vine asks to take, and says nothing about how to do the work in Workday. The whole session is filmed and Vine’s own logs are kept. The run ends when Vine says the work is done (or the tester gives up). Vine gets no follow-ups in the chat sense: it sees Workday’s errors itself, which is the point of the product.

Scoring

After each run, a checker who did not apply the answer runs the task’s pass checks in the tenant (Workday’s own query language, WQL, through Workday’s API, and Workday’s own screens). The checks are written below before any run and are the same for every lane.

  • Pass: every check passed. Recorded separately as first answer (no follow-up), or after N follow-ups.
  • Partial: the work exists but at least one check failed. Which checks passed is recorded.
  • Fail: nothing usable in the tenant, or the contestant declined.

Fairness: if a chat answer works, it passes. A check that could not run is recorded as not run, never as passed or failed.

Time

  • Chat lanes: the model’s answer time (from sending to the last word, each turn) and the operator’s applying time, separately.
  • Vine: from the first message to Vine saying it is done, from the run’s own timestamps and film.
  • WD: answer time, plus applying time if it gave steps.

What is recorded

runs/<task>/<lane>/: prompt.txt; transcript.md and the client’s raw output (raw/), every turn with its timestamps; apply/ (the operator’s log and screenshots); check.json (each check, its evidence and result). Vine runs keep run.json, the film and Vine’s logs from the test computer. results.md holds the one table, built only from check.json files.

The tasks

The suffix rule applies to every name. Sample data is the tenant’s own; nothing is created to make a check easier.

T1. Calculated field: service band

Prompt

In our Workday tenant I need a calculated field on Worker called “Service Band {S}” that we can use in custom reports. It should return one of these text values based on completed years of service up to today: “Under 1 year”, “1 to 3 years”, “3 to 5 years”, “5 years or more”. Use the worker’s Continuous Service Date, and if that is blank use their Hire Date. Exactly 1, 3 or 5 completed years belongs to the higher band (so a worker with exactly 3 completed years is “3 to 5 years”). Please give me what I need to build it.

(For Vine and WD the last sentence reads “Please build it in our Development tenant.” The same change applies to every task: chat lanes are asked for what they need to build it, Vine and WD are asked to do it.)

Pass checks

  1. A calculated field named “Service Band {S}” exists, saved, on the Worker business object, and is reportable (Workday exposes it to WQL on a worker data source).
  2. For every active employee in the tenant (205 on 19 Nov 2026), its value equals the band computed independently from the worker’s Continuous Service Date (Hire Date when blank) and the check date. Pass needs 100% agreement; the count of disagreements is recorded.
  3. Boundary: workers whose completed years are exactly 1, 3 or 5 on the check date (if any exist) are in the higher band. If none exist on the day, recorded as not run (no worker is altered to make one).

Note on the data: on 19 Nov 2026 every active employee in the tenant has a Continuous Service Date, so the Hire Date fallback is never exercised by the data. The checker reads each lane’s saved definition and records whether the fallback is there, but check 2 can only test the Continuous Service Date path.

T2. Custom report: organization roster

Prompt

Please create a custom report in our Workday tenant called “Org Roster {S}”. When someone runs it, it should ask them to pick a supervisory organization and then list the active workers in that organization and all of its subordinate organizations. Columns: Worker, Employee ID, Supervisory Organization, Job Profile, Location, Hire Date, Manager. Sort it by Hire Date, earliest first. Our main test organization is “HCM1119 Outfitters”. Please give me what I need to build it.

Pass checks

  1. A saved custom report “Org Roster {S}” exists and runs with a supervisory organization prompt.
  2. Run for HCM1119 Outfitters: the set of workers equals the active workers in HCM1119 Outfitters and every organization below it (from WQL on the organization hierarchy), no more, no fewer.
  3. All seven columns are present with correct values for five workers sampled by the checker.
  4. Rows are in Hire Date order, earliest first.
  5. Run for a second organization (HCM1119 IT): the result changes to match that organization.

T3. EIB load: new cost centers

Prompt

We need to load these new cost centers into our Workday tenant with an EIB, effective 1 January 2026. Each one should be active. Please give me what I need to do the load.

Cost Center Code Name
Cost Center Code{S}-4100 NameField Marketing {S}
Cost Center Code{S}-4110 NamePartner Marketing {S}
Cost Center Code{S}-4120 NameProduct Marketing {S}
Cost Center Code{S}-4200 NameCustomer Success {S}
Cost Center Code{S}-4210 NameRenewals {S}
Cost Center Code{S}-4300 NameData Platform {S}
Cost Center Code{S}-4310 NameSecurity Engineering {S}
Cost Center Code{S}-4320 NameDeveloper Tools {S}

Pass checks

  1. All eight cost centers exist in the tenant with exactly these codes and names.
  2. Each is active with effective date 1 January 2026.
  3. The load went through an EIB (an EIB integration event exists and completed), as asked.

T4. Business process change: a To Do on Change Job

Prompt

In our Workday tenant, on the Change Job business process for our top-level company organization, add a To Do step for the HR Partner called “Confirm workspace move {S}” that comes after the manager’s approval. The business process must still work for Change Job events started after the change. Please give me what I need to make the change.

Pass checks

  1. The Change Job business process definition for the top-level company organization has a To Do step assigned to HR Partner with the To Do name “Confirm workspace move {S}”.
  2. The step comes after the manager’s approval step in the definition’s order.
  3. The definition saved without errors and is the active definition (Workday’s “View Business Process Definition”).
  4. A Change Job started by the checker for one sample worker after the change reaches the “Confirm workspace move {S}” To Do for the HR Partner.

T5. Security: an auditors group

Prompt

In our Workday tenant, create a user-based security group called “Comp Auditors {S}” and add the user lmcneil (Logan McNeil) to it. The group should be able to view, but not change, worker compensation data: grant it View only on the domain “Worker Data: Compensation”. Make sure the change is actually in effect. Please give me what I need to make the change.

Pass checks

  1. A user-based security group “Comp Auditors {S}” exists with lmcneil as a member.
  2. The domain security policy for “Worker Data: Compensation” lists the group with View (Get) and not Modify (Put).
  3. The security policy change is activated (the group appears in the active policy, not only in a pending one).

T6. WQL: recent hires by organization

Prompt

Write me a WQL query I can run against our Workday tenant (through Workday’s WQL REST API) that returns, for each supervisory organization, the number of active employees hired on or after 1 January 2025, with the largest count first. Organizations with none can be left out.

(For Vine and WD: “…and run it in our Development tenant and give me the query and the result.”)

Pass checks

  1. The query, run exactly as written through Workday’s WQL REST API (/ccx/api/wql/v1/<tenant>/data), runs without error. If the answer gives no single runnable query, this fails.
  2. Its result equals the ground truth: per supervisory organization, the count of active employees with Hire Date on or after 2025-01-01, computed by the checker from the full worker list (every worker read, counted outside WQL).
  3. Sorted by count, largest first.

T7. A small Extend app: my team’s hire dates

Prompt

Build a small Workday Extend app called “Team Tenure {S}” for our Development tenant: one page that a manager opens from Workday and that lists their direct reports with each person’s name, job profile and hire date, newest hire first. Deploy it to our Development tenant (never promote it beyond Development). Please give me what I need to build and deploy it.

Pass checks

  1. An Extend app “Team Tenure {S}” (or with that name as its title) is deployed to the Development tenant.
  2. Its page opens from inside the tenant for Logan McNeil without an error.
  3. It lists exactly Logan McNeil’s direct reports (from WQL on the worker’s manager), with name, job profile and hire date matching the tenant, newest hire first.

T8. Integration: an outbound worker file

Prompt

Set up an outbound integration in our Workday tenant called “Worker Contacts {S}” that produces a CSV file of the active workers in the supervisory organization “HCM1119 IT”, with columns Employee ID, Legal Name, Work Email, Manager, Hire Date. Then launch it once so we get the file. Please give me what I need to set it up.

Pass checks

  1. An integration system “Worker Contacts {S}” exists in the tenant.
  2. It was launched and the integration event completed without errors.
  3. The output file is CSV and has exactly the five columns.
  4. Its rows equal the active workers of HCM1119 IT (WQL), with Employee ID, Work Email, Manager and Hire Date matching for every row.

Order of work

T1 runs first in all five lanes and is handed to the Kiweely team so the format can be checked; then T2 to T8.

Changes log

  • 20 Nov 2026, 06:18 UTC: while checking T5 for ChatGPT, the checker pressed OK on “Activate Pending Security Policy Changes” on the coordinator’s wrong instruction (it was believed to have a confirm screen; it has none), which activated whatever was pending at that moment. ChatGPT’s lane had already activated its own change at 06:09 UTC; Workday’s current security moment before the checker’s activation was that activation (described with the lane’s own words), so ChatGPT’s T5 result does not rest on the checker’s activation. No other lane had touched security yet. Checkers now only read that screen and cancel.

  • 20 Nov 2026, ~06:00 UTC: Gemini’s weekly plan quota ran out during T3’s first follow-up (“Individual quota reached ... Resets in 113h41m56s”, saved in runs/T3/GE/raw/turn-2.*); the team’s other Gemini sign-in is also used up. A quota error is not counted as a follow-up or a failure: Gemini’s lanes are applied as far as their first answers go, and any follow-up waits for the quota to reset (24 Nov 2026, about 23:40 UTC), then continues from where the lane stopped with the same text. Gemini’s results are reported only once those lanes finish.

  • 20 Nov 2026, ~06:00 UTC: the Gemini client’s deny list did not stop its file-writing tool. Its own logs show write_file calls on T3 (one CSV) and T7 (six app files), all inside its own empty folder; no command was run and nothing reached the tenant. A Gemini user would have those files beside the answer, so they count as part of Gemini’s answer: they are kept in runs/T3/GE/raw/files/ and runs/T7/GE/raw/files/ with the client’s log, and the operators use them as written. No other task’s Gemini folder holds any file.

  • 20 Nov 2026, before any T4 or T5 run: every lane changes the same Change Job definition (T4) and the same domain security policy (T5), so those lanes run one at a time, never at once, and each lane is checked before the next lane starts (on T5, one lane activating pending security changes would otherwise also activate an earlier lane’s pending change and credit it; on T4, a later lane’s edit could otherwise hide or break an earlier lane’s step). Order: ChatGPT, Claude, Workday’s agent, Vine, then Gemini (moved last once its quota ran out, so a Gemini lane paused mid-way can never leave a pending change for another lane to activate). A lane that comes later sees the earlier lanes’ steps in the definition; that is the same tenant every lane works in.

  • 20 Nov 2026, ~05:40 UTC: the operator rules said “do not delete anything”, meant to protect other lanes’ work. On T2 that stopped Claude’s lane from following its own answer (delete the half-built report it had just created and recreate it). The rule now allows deleting an object the lane itself created in that run when the answer says so, and that lane’s application was continued from where it stopped. On T6 the first operator skipped a lookup step Claude’s answer gave (run a query, paste the id it returns); the checker applied that step as written and recorded it in the lane’s report.

  • 19 Nov 2026, 20:50 PT: the first T1 to T3 answers from the three chat clients were thrown away before any was scored. The clients had been started with this computer’s own setup: Codex had extra tool servers (on T2 it tried a screen-control tool to look for an open Workday window), Claude Code loaded this computer’s user instructions, and Antigravity had code-index servers. All three were isolated as described above and every task was asked again. The thrown-away answers and the partial work two operators had started from them (fields named TT-CG, TT-CL, TT-GE in the tenant) are kept in runs/_void-v1-unisolated-clients/ and are not part of any result. The clean runs use new suffixes (TT-GPT, TT-CLA, TT-GEM) so nothing from the void runs can be mistaken for them.

  • 19 Nov 2026, before any T3 run: T3 no longer asks for a cost center hierarchy. The tenant’s 34 cost centers sit in no hierarchy that WQL or a task search shows, so the prompt would have named something a consultant could not point to. “Cost Center ID” became “Cost Center Code” (the Code field Workday shows).

  • 20 Nov 2026, T3 rerun for Vine: the rerun’s suffix is TT-KV2, because attempt 1 already created the eight TT-KV cost centers (they cannot be removed, and a rerun meeting its own earlier objects would not be the same task). The request is otherwise exactly as printed above. Vine’s run files now put the request last, after the tester’s acceptance notes, so nothing can follow it. The first messages of Vine’s T1, T2, T5 and T8 runs were read from their session logs: each is the request word for word, with nothing added.

  • 20 Nov 2026, before T3 for Vine (KV) was scored: Vine’s T3 request was not word for word. The tester’s first message (run tenant-test-t3/20261120T070818Z; its .claude jsonl; visible in the first frames of film.mp4) ends with two extra lines copied from the run’s idea file: “Acceptance (for the record only; checked by someone else afterwards): 1. kiweely received the request word for word as its first message. 2. kiweely said the work is finished, or said why it could not finish.” They give no how-to hint, but no other lane received them. This run is kept as evidence and is not the scored KV result for T3. T3 is asked again for Vine with the request exactly as printed above. If that rerun cannot finish by 26 Nov, when the tenant may expire, the first run is scored and its row says the request carried these two lines. T1 and T2 KV requests were checked the same way and are word for word. T6’s request is to be checked before T6 KV is scored.