Background

Wasimil operates a hotel booking and management platform utilized by hotel owners, front desk staff, and guests. The platform is highly dynamic, with daily feature releases and a need for robust testing to ensure seamless user experiences and minimal production issues. Previously, Wasimil relied on Playwright for automation testing, but faced significant challenges with maintenance and reliability.

Challenges

• High Maintenance Burden: Over 50% of Playwright tests were broken at any given time, leading to frequent false positives and a lack of trust in automation results.
• Resource Drain: Engineers spent a disproportionate amount of time fixing and maintaining automation scripts rather than developing new features.
• Limited Accessibility: Playwright required coding expertise, making it difficult for non-technical QA team members to contribute effectively.

"Previously, when we used Playwright, half the tests were broken. And no engineers found any value in those playwright automation tests because they don't know if it was a false positive or an actual regression."
- Tom, CTO, Wasimil

Solution: Migration to Harness AI Test Automation

Faced with these challenges, Wasimil sought a solution that would reduce maintenance time, increase accessibility, and support daily releases. The team adopted Harness AI Test Automation for end-to-end testing.

Results

• Reduced Bug Count: The overall number of production bugs decreased, especially critical ones, allowing the team to focus on feature delivery rather than firefighting.
• Resource Optimization: Engineering resources previously spent on bug fixes and test maintenance were reallocated to feature development.
• Improved Release Cadence: The ability to release daily reduced team exhaustion and increased deployment flexibility.
• Accessibility: The web-based, no-code interface empowered non-technical QA team members and unified the testing approach for all engineers.

"When we use Harness AI Test Automation, we are now able to release every day because we have the automation that does the checking, and the QA and the other developers can do other things other than smoke testing."
Bima, Lead QA Engineer, Wasimil

Key Learnings

• AI Test Automation Reduces Maintenance: Harness AI’s ability to adapt to UI changes, which significantly reduces the time spent updating test scripts.
• Accessibility Drives Adoption: The ease of use and unified framework increased collaboration between QA and development teams.
• Quality and Velocity: Lower bug counts and faster release cycles improved both product quality and team morale.

"With AI Test Automation, we could ship features, not just bug fixes. We don't wanna spend 30 to 40% of our engineering resources on fixing bugs because we can't be proud to ship bug fixes to our customers. Right? And they expect features, not bug fixes."
- Tom, CTO, Wasimil

Next Steps

• Increase Developer Contributions: Encourage more developers to contribute to test automation, leveraging the accessible interface.
• Monitor and Optimize Failure Rates: Continue tracking and reducing test failure rates, especially those related to AI and selectors.

Conclusion

Wasimil’s adoption of Harness AI Test Automation transformed their testing process, reducing maintenance, increasing release velocity, and empowering both technical and non-technical team members. The platform’s AI-driven approach enabled daily releases, improved product quality, and allowed the team to focus on innovation rather than maintenance.

"The accessibility of Harness AI Test Automation to the average engineer is great. Not every engineer is familiar with Playwright. They always come with their own tool of choice. But what we need is one unified testing framework. That is accessible and easy for them to learn. They can use a simple web interface and figure out how to set up their selectors and the sequence that they need. And then run the test and see the results directly."
- Tom, CTO, Wasimil