I wanted to experiment with building an end-to-end Playwright test with AI. I started with a single feature on one of my own projects: the pricing period toggle a guest would use to check rates based on the dates of their intended stay.
The workflow ended up looking something like this.
1. Explore the feature
Instead of manually clicking through the page and writing notes (exploratory testing), I had the AI interact with the page, take snapshots, and document what it observed.
As we reviewed the output together, we found a couple of genuine issues, including pricing content disappearing for two of the periods. We also ended up checking the HTML to confirm whether the behaviour was truly happening or whether the AI was making assumptions.
That was probably the first lesson, the AI still needed someone to sanity check its report.
2. Generate the test plan
Once we'd explored the feature, we turned the observations into a proper test plan. That became the reference point for everything afterwards. Every future test had to map back to something that existed in the plan.
That made reviewing the generated code much easier, because I always knew where a scenario had come from.
3. Generate the Playwright automation
From the approved plan, we generated:
- a reusable base page
- a rates page object
- a Playwright spec
At that point the code looked reasonable, but I didn't really trust it yet.
4. Review the generated code
This was where my QA experience mattered most. I wasn't looking for syntax errors, I was looking for the kinds of things I'd normally comment on during a code review.
- Were the locators robust?
- Were the assertions actually checking the right behaviour?
- Was there duplicated logic that belonged in the page object instead?
- Were the tests independent?
A couple of weak assertions came out of that review, even though the tests were passing.
5. Prove the tests are reliable
The first execution passed: 17 tests, 17 passes.
But I reran the suite repeatedly, initially I hit failures, but they turned out not to be flaky tests after all, I was running too many parallel browsers against a small live website. Once I fixed the way I was running the suite, the tests became stable. There was still one intermittent network assertion left, which I marked for further investigation.
Final thoughts
At the end of the afternoon, what I realized was that what took most of my time wasn't spent writing code, it was spent reviewing the observations, the test plan, the generated Playwright code, the execution results, and that takes time, experience and skill.
Turns out the typing was the easy part, while the human QA thinking was the work.