TVNZ accelerated product innovation and improved customer insights with Harness Feature Management & Experimentation, transforming from limited manual experimentation to running multiple concurrent tests. Feature flags allowed them to minimize risk by toggling features on or off and gathering real customer feedback. Integration with Google Analytics and data pipelines saved engineering time while improving data tracking and attribution, enabling a truly data-driven product organization.
Summary
Discover how TVNZ transformed their experimentation process, scaling from one experiment per month to several continuous A/B tests with Split feature management and experimentation (acquired by Harness).
- Accelerated experimentation: TVNZ scaled their experimentation efforts, running multiple tests per day, improving their speed and efficiency along the way.
- Enhanced data-driven decisions: With Split’s real-time results and automatic statistical calculations, TVNZ gained granular insights into feature performance, so they could make smarter product decisions without hesitation.
- Improved feature rollouts: By using feature flags, TVNZ minimized risk by easily toggling features on/off and enabling targeted experiments for customer feedback.
- Streamlined data integration: Split's seamless integration with TVNZ's existing data pipelines and Google Analytics helped save engineering time and improved data tracking.
About TVNZ
Television New Zealand (TVNZ) is New Zealand’s state-owned, commercially funded broadcaster. Their media reach includes TVNZ 1, 2, and DUKE television channels, along with the TVNZ OnDemand streaming platform, connecting with millions of New Zealanders daily. As the digital landscape expanded, TVNZ sought a solution to enhance their product experimentation capabilities, scale innovation, and reduce the long-term maintenance costs of their in-house systems.
The Challenge
Slow, manual experimentation processes
TVNZ faced significant challenges with their in-house experimentation system, which required DevOps teams to manage feature flags and manual steps for A/B testing. These processes were slow, limiting their ability to iterate quickly on customer-facing features. Their goal was to move from running one experiment per month to several ongoing experiments, using a more automated and scalable system that would also provide higher statistical rigor.
The Solution
Faster, more scalable experimentation
TVNZ chose Split to build a reliable feature flag foundation, improve A/B testing, and automate their experimentation processes. With Split, the team could now easily toggle features on and off at the press of a button, reducing the risk of negative customer impacts.
- Streamlined experimentation: Split automated the A/B testing process, ensuring proper randomization and statistical calculations.
- Real-time insights: Product and engineering teams could view experiment results in real time, enabling faster decisions on feature rollouts or rollbacks.
- Dynamic targeting and experimentation: TVNZ used Split to segment users dynamically for personalized experiences, and adjust sample sizes and variations in real-time.
Nathan Wichmann, Product Manager at TVNZ OnDemand, highlights how Split allowed the team to move much faster in their experimentation and make data-driven decisions: “With Split, we’ve gone from running one experiment every few months to having so many experiments that we need to think carefully about sequencing and prioritization.”
Results
Scaling experimentation and innovation
The integration of Split into TVNZ’s development process resulted in measurable improvements:
- Faster experimentation: TVNZ transitioned from infrequent to continuous experimentation, dramatically speeding up their feedback loop.
- Increased experimentation cadence: The ease of creating and running experiments helped the team run multiple tests concurrently, achieving higher velocity in decision-making.
- Enhanced decision-making: With Split's real-time results and integration with their data pipeline, TVNZ could evaluate the impact of new features and quickly pivot when necessary.
- Feature debt reduction: TVNZ could safely revisit and re-assess old features, using Split to experiment with their removal or improvement.
- Business impact: The experiments helped the team measure and track product performance, directly linking innovation to business success.
“We use feature flags and run experiments a lot more because it’s easier now with Split. We can target a random sample size. We can increment the size of that sample really easily, and we can toggle it off immediately if there’s an issue.”— Mel De Soysa, Business Analyst, TVNZ
Future
Building a culture of experimentation
As TVNZ continues to scale their product and innovate, Split will be integral to maintaining their culture of experimentation. The team is able to test, measure, and refine product features while ensuring they align with key business metrics.
Nathan Wichmann explains, “Split has been really useful for us. We’ve been able to measure product impact and show that the ideas we’ve worked on have produced tangible business results.” TVNZ’s ongoing use of Split empowers them to continue evolving and making informed decisions for their products and customers.
