Updated 23 October 2026.
A fast first draft is a weak business case. The useful question is whether AI helps someone complete a recurring project task with less total effort, acceptable errors and a result people can trust. That includes the work of preparing inputs, checking references, correcting omissions and getting approval.
A pilot makes that question answerable without turning a promising demonstration into a subscription commitment. Choose one task, record how it works today and decide what evidence would justify continuing. You may find a useful assistant, a narrower use than expected, or a sensible reason to stop.
For task-by-task prompts for status reports, RAG and RAID, see AI in Project Management (2026). This guide concentrates on evaluating the workflow around those tasks rather than repeating the prompts.
Define the benefit before choosing the tool
Start with a problem someone recognises. Perhaps collecting material takes too long, a reviewer repeatedly returns drafts, or people cannot find the evidence behind an update. Each problem needs a different test. Faster writing will not fix records nobody maintains.
Write the proposed benefit as a change in a completed task. For example, an illustrative pilot objective could be: prepare the same weekly update with less combined effort while keeping every material issue and preserving uncertainty. This describes an outcome that a reviewer can inspect.
Avoid using the number of generated documents as the success measure. More drafts can mean more reading, inconsistent versions and extra approval work. A useful result reaches its intended reader and supports the next conversation without someone reconstructing the source material.
Also ask whether a simpler change would solve the problem. A shared template, clearer ownership or a reminder rule may remove the same friction. The pilot should compare AI with a reasonable manual process, not with an unnecessarily disorganised one.
Choose one pilot with visible boundaries
A suitable first task repeats often enough to compare, uses permitted information and produces something a knowledgeable person can check. Keep the intended audience, output format and approval route stable while learning what the assistant changes.
A draft status report can be a practical starting point if the underlying records are maintained and approved for processing. Use the pillar’s status-report workflow and prompt as the drafting method. Keep publication with the project manager and measure the whole reporting cycle.
If your main problem is backlog preparation, use the Scrum and Kanban ceremony guide to choose a narrower activity. Do not mix reporting, refinement and knowledge search into one trial; you will struggle to explain which change produced the result.
Write a short pilot agreement covering the task, inputs, reviewer, output recipient and actions the tool may perform. Name who can suspend the trial. A drafting pilot does not need permission to send messages, change dates or update the live plan automatically.
Include a manual fallback from the beginning. If an outage prevents access to the assistant, someone must still be able to find the records and complete the task. A workflow that depends on a service for basic access may add operational fragility even when its drafts are useful.
Establish an honest baseline
Record how the task is completed without AI before judging the assisted version. Follow the work from collecting evidence to an approved output. Include the reviewer and anyone asked to clarify a contradiction or repair a missing fact.
Compare similar work. A quiet cycle with complete inputs is a poor comparison for a difficult cycle with late changes. Note the audience, input volume, unresolved questions and unusual events alongside each observation. Differences do not make a pilot useless, but they limit what you can conclude.
Use the same quality checks for both routes. A manual report can omit issues or contain unsupported wording too. The aim is to improve the task, not to assume either method starts without errors.
Keep setup effort separate from recurring effort so neither disappears. Creating a reusable input pack may be a one-off activity. Cleaning the pack every week is recurring work. Record both and state how long you reasonably expect to use the workflow before it changes again.
Do not quietly improve the manual process during the trial and attribute the whole benefit to AI. If introducing source IDs makes review easier, record that improvement in its own right. It may remain worthwhile even if you stop using the assistant.
Measure total effort, not generation speed
Use a simple working definition:
Total task effort = preparation + drafting + review + correction + approval and administration.
Preparation includes selecting permitted records, removing unnecessary personal information and resolving input conflicts. Drafting includes prompt adjustments and repeat attempts. Review includes opening references and checking the source records for omissions. Correction includes repairing wording and checking that the repair has not introduced another problem.
Count effort across people. Passing a draft to a specialist transfers work; it does not remove it. Record elapsed turnaround separately if waiting for review matters, because a short writing task can still miss its deadline while sitting in an approval queue.
Treat subscription and usage charges as costs alongside time. Include AI credits, usage-based pricing, required integrations and recurring administration. A pilot run by an enthusiastic expert may conceal support work that ordinary users will need later.
Avoid converting all saved minutes into cash automatically. Time released only creates a financial saving if the organisation can use it in a meaningful way. You can still value reduced overtime, more reliable turnaround or better preparation without presenting those benefits as money already recovered.
Copy this pilot scorecard
Complete the acceptance column before running the assisted task. The entries below are suggested tests to adapt, not measured results or universal thresholds. Keep a separate copy for each observation and retain the reviewed output where policy allows.
| Measure | Manual baseline | AI-assisted observation | Acceptance or stop rule | Evidence and owner |
|---|---|---|---|---|
| Task and input comparability | Audience, input condition, complexity | Note differences from baseline | Explain material differences before comparing | Pilot owner records context |
| Preparation effort | Time collecting and checking inputs | Include redaction and input packaging | No hidden recurring preparation work | Task owner’s time record |
| Drafting effort | Time to first reviewable draft | Include retries and prompt adjustments | Draft must meet the same purpose | Saved draft and task owner |
| Review and correction effort | Time checking and repairing | Include source opening and specialist help | Stop if checking outweighs useful benefit | Reviewer record |
| Factual accuracy | Errors found before issue | Invented facts and false source references | No unsupported material claim in issued output | Reviewer checks sources |
| Completeness and uncertainty | Missing issues and altered conditions | Omitted issues or forecasts made certain | Preserve every decision-relevant qualification | Source-to-output comparison |
| Reader usefulness | Questions or rework after receipt | Does the output support the next decision? | No loss of essential context | Recipient feedback |
| Recurring cost | Existing software and administration | Subscription, credits, integration support | Stay within the agreed cost envelope | Budget and service owner |
| Control and resilience | Approval and fallback route | Access, export and outage test | Suspend on unauthorised exposure or action | System owner evidence |
| Overall decision | Current route remains available | Keep, revise or stop, with reasons | Named decision-maker accepts the evidence | Dated decision record |
Do not collapse this into one average score. A lower drafting time cannot compensate for unauthorised disclosure. A favourable usefulness rating cannot erase a false commitment. Keep mandatory controls separate from benefits that can be traded against cost.
Price errors by consequence
An awkward sentence and an invented approval are different failures. Record what was wrong, how it was detected and what could happen if it escaped review. Use categories that lead to action rather than a single count of mistakes.
For example, in an illustrative report, an assistant changes “testing could start after access is confirmed” into “testing starts on Monday”. The correction itself may be quick. The potential consequence is a recipient allocating people against a commitment nobody made. Preserve the conditional language and record the failure as consequential.
False source references deserve their own check. A citation may look convincing but lead to an unrelated passage, an obsolete version or nothing at all. Open the reference and verify the claim it is supposed to support. Then scan the inputs for issues the draft did not mention.
An overconfident tone can conceal missing evidence even where individual sentences sound plausible. Ask the reviewer whether the overall impression matches the actual position. A reassuring conclusion built from selective facts can mislead without containing an obvious invented number.
Do not invent a financial probability for every possible failure. Where impact cannot be priced credibly, state it in plain terms and treat it as a constraint. Accountable judgement is more useful than a precise-looking calculation built on guesses.
Agree success, revision and stop criteria
Success means repeatable improvement across representative work, with acceptable review effort and controls. Agree which improvements matter: lower total effort, clearer evidence or fewer corrections. State what must remain true even when the draft looks good.
A revision decision suits a specific, fixable problem. Perhaps the input pack lacks dates or the prompt hides unresolved questions. Change one part of the workflow, record the change and compare again. Endless prompt tuning without a useful result is itself a cost.
Stop when checking consistently costs more than completing the task manually, consequential errors are difficult to detect, or users cannot reliably distinguish drafts from approved outputs. Suspend immediately if information is exposed without authorisation or the assistant performs an action outside its agreed remit.
Give the pilot an agreed review point and an owner. Avoid continuing simply because setup took effort. That effort is already spent; the decision concerns the future work and cost of keeping the process.
Include the costs that appear after the demo
Privacy review, access configuration and integration maintenance belong in the evaluation. Check retention, training use and data residency for the actual service configuration. Approval to use one product does not automatically permit every category of project information.
Ask how prompts, outputs and evidence leave the platform. Vendor lock-in can come from inaccessible history, proprietary workflows or a process that only one administrator understands. Test data export and retain enough instructions to perform the task elsewhere.
Usage can also change as the workflow grows. Repeat attempts, connected retrieval and multiple reviewers may affect consumption. Confirm the current charging rules and what happens when credits or usage limits are reached. Use the software buyer’s guide for a structured vendor trial.
When NOT to use AI
Do not submit confidential or personal data without approval for the specific use. If removing sensitive context makes the task meaningless, choose another task rather than pretending the remaining input is representative.
Keep decisions requiring accountable judgement, negotiation or acceptance of consequences with the authorised people. AI can organise approved evidence, but it cannot agree a trade-off with a supplier or take responsibility for a delivery commitment.
Do not generate from unreliable inputs and call the result an improvement. Fix conflicting records first, or make the unresolved conflict explicit. Stop using the assistant where checking costs more than doing the work, especially when reviewers lack the expertise to detect plausible errors.
Keep legally or contractually binding text within the appropriate drafting and approval process. A successful internal-summary pilot does not justify using the same workflow to issue commitments or interpret contractual obligations.
FAQ
Does a quicker draft prove that AI is worth paying for?
No. Compare preparation, drafting, review, correction and administration together, then include charges and the consequences of errors.
What should I pilot first?
Choose one recurring, reviewable task with permitted inputs and a named reviewer. A draft report may suit a team with reliable underlying records.
What if the pilot does not save time?
Keep it only if another explicit benefit justifies the cost and controls hold. Otherwise revise the workflow or stop, retaining any useful process improvements.
Put It Into Practice
Pick one thing from this article to try on your next project. If you need a starting point, Resources has free status report, action log and decision log templates. For a difficult conversation, try the People Field Guide.
Get new practical guides by email →
About Paulius
I’m Paulius Petravicius, a project manager with a background in business and data analysis. I share practical ways to make everyday project work a little easier. Read my story →