Testing Resilience

Diving Deeper into Software Testing Types

Discover the essentials of software testing—functional and non-functional—to improve app performance, reduce bugs, and boost user satisfaction. Learn about unit, integration, end-to-end, and performance testing, plus best practices for manual and automated approaches.

How to Build a Chaos Lab for Real-World Resilience Testing

Explore how to create a chaos lab to enhance reliability, run real-world failure scenarios, and drive continuous resilience in software delivery.

Understanding Object Dependency and How It Impacts Feature Access

Explore how object dependencies shape feature availability, streamline development, and influence release strategies for modern teams.

Overcome long-term dependencies with dynamic process monitoring

Discover how to tackle long-term dependencies, slow feature analysis, and streamline dynamic process monitoring on GitHub for optimized delivery.

Guide to incident management retrospectives for improvement

Learn how to run an effective incident management retrospective to uncover root causes, prevent recurrences, and optimize team performance.

What Is an Incident Management Team?

Explore how an Incident Management Team operates, the roles involved, and best practices for faster response, minimal downtime, and happier customers.

Understand software features and their impact on development

Learn how to define, prioritize, and measure features. Discover best practices, reduce risk with feature flags, and drive continuous improvements.

Epics and Features: The Cornerstone of Agile Success

Learn how epics and features power agile success. Ensure strategic alignment, sprint efficiency, and real value in every iteration.

How to Blend New Features with Old Features: A Detailed Guide

Learn proven strategies for blending new features with existing ones, ensuring product cohesion, user satisfaction, and long-term success.

What Is Feature Engineering? A Guide for Data-Driven Success

Uncover what feature engineering is, why it matters, and the essential techniques to drive impactful results in machine learning and analytics.

A guide to understanding Feature Driven Development principles

Learn how Feature Driven Development fosters efficient, high-quality software releases with real-world tips, best practices, and examples.

Mastering Feature Toggles in Java for Safer Releases

Discover how to use feature toggles in Java for safe, efficient code releases. Simplify feature flags, enhance collaboration, and reduce deployment risk.

GitLab Feature Flags: A Comprehensive Guide to Safer Deployments

Discover how GitLab Feature Flags enable controlled, incremental releases. Explore best practices for safer, data-driven deployments and faster innovation.

How feature flags enhance iOS app development efficiency

Learn how to leverage feature flags on iPhone for controlled rollouts, improved user experience, and safer releases in your iOS apps.

What is Incident Response?

Learn how incident response helps teams detect, manage, and resolve issues faster. Discover best practices, tools, and AI-driven approaches today.

What is the Software Release Life Cycle?

As Agile continues to shape how software is built and delivered, the Software Release Life Cycle has become faster, smarter, and more collaborative.

What is Fallback Value?

A fallback value is a predetermined option or default that is used when a desired or primary value is unavailable or cannot be determined.

What is a Flag Trigger?

An auto-kill webhook is designed to automatically terminate certain processes, tasks, or resources when triggered by predefined conditions.

What is Flag Dependency?

Flag dependencies are criteria that must be met before a particular flag can be set or activated within a system, software, or process.

What is a Winning Variation?

Winning variation refers to the version of an asset that outperforms other variations in achieving a specific goal.

What are Application Versions?

Application versions refer to releases or iterations of a software application, each representing a stage of development or enhancement.

What Is Feature Development?

With feature development, teams can successfully introduce new features that enhance the software product and provide value to users.

What is a Targeting Rule?

Targeting rules are predefined criteria or conditions to direct specific actions or resources towards particular targets within a system.

What is Deployment Automation?

Deployment automation enables teams to deploy faster, frequently, and with confidence so organizations can achieve more predictable outcomes.

What is a Modern Technology Stack?

A modern technology stack integrates various technologies and practices to address the complete lifecycle of an application.

What is a Configuration Flag?

Incorporating configuration flags into development, can improve deployment strategy and enhance testing capabilities for better experiences.

What Is a Cloud Deployment Model?

A cloud deployment model is a public, private, hybrid, or community cloud, chosen with an organization's needs and strategies in mind.

What is a Production Environment?

The production environment serves as the platform where users interact with the product in real-world scenarios.

What is Release Testing?

Release testing refers to the phase where a software product undergoes thorough testing to ensure its readiness for deployment to end-users.

What is Rolling Deployment?

A rolling deployment updates components or instances of a system incrementally to ensure continuous availability and minimal downtime.

What is Staging?

Staging refers to the process of deploying and testing software changes in an environment that closely resembles the production environment.

What is Release Management?

Release management is the strategic process of planning, coordinating, and controlling the deployment of software releases.

What is a Test Environment?

A test environment is a controlled setup used to conduct software testing where systems can be evaluated under specific conditions.

What is a Database Migration?

Database migration is the process of transferring data from one database system to another, involving changes in database schema or platform.

What are Observability Tools?

Observability tools are designed to provide visibility into the performance, health, and behavior of systems and applications.

What is a Shopping Cart?

A typical pattern in any e-commerce application is providing a visitor with a shopping cart to store products or services before checkout.

What are Monthly Active Users?

Monthly Active Users is a key performance indicator used to quantify the number of unique users who engage with a particular product.

What is a Web Session?

A web session is a way to organize interactions between a user and the application within a specific time frame. Sessions across pages or browser windows.

What is User Satisfaction?

A measurements of user satisfaction is the Net Promoter Score, which asks for a 1-10 rating of likeliness the user to give recommendations.

What is a Workflow Management?

Workflow management involves designing, executing, and monitoring workflows to enhance overall business processes.

What is Mock Testing?

Mock testing is a software testing technique where artificial components are created to replicate the behavior of real components.

What is Bayesian Statistics?

Bayesian statistics employs Bayesian probability theory to model and update uncertainties about hypotheses.

What is a Merge Conflict?

Merge Conflict Glossary: Learn about conflicts, resolution techniques, and essential terms to navigate collaborative coding seamlessly.

What are Page Views?

For any website or web application, one of the most valuable events you can track is the act of a user loading or viewing a page.

What is an Interaction?

The primary means of interaction with any web page are clicks, as those are the interactions that show the highest user intent.

What is a Rage Click?

We may not capture that the user is yelling at the screen, but a typical behavior that occurs is rage clicks - rapidly repeated clicks.

What is an Input?

Inputs account for a special class of interaction that prompts a user to either make a selection or provide text. These include entry fields, forms & more.

What is Page Load Performance?

Page load time refers to how long it takes for a page to show up on the screen. Modern browsers offer a Navigation Timing API.

What is an Error?

When those teams can combine feature data with those errors, they can achieve faster identification, response, and resolution.

What is a Conversion?

Through understanding and improving the conversion rate for your key workflows, the business can be successful.

What is a Treatment?

A treatment is the version of the UI (user interface) that the user sees. You can set a treatment in the UI to be on or off.

What is an Active User?

An application's active user count measures the number of unique users who have engaged with the application during a given period.

What is a Fixed Horizon?

Fixed Horizon is a statistical testing method where sample sizes and your experiment goals are defined in advance. Visit Harness to learn more.

What is a Pull Request?

A pull request is when you are requesting for someone in your org, like an engineer, to look at your code, review it, and pull the changes in.

What is a Main Branch?

The main branch is the original thread of code for a project’s codebase. The main branch of a project is considered the default branch.

What is a Segment?

Segments are used to organize targeted users into groups to facilitate setting up incremental product releases or experimentation.

What is Edge Computing?

One observable benefit of edge computing is an improvement in page loading time, but actually any type of internet communication can benefit.

What is a Feature Branch?

Feature branch is a copy of the main codebase, where an individual or team of software developers work on a new feature until it is complete.

What is Sequential Testing?

With Sequential Testing, you can confidently make informed decisions about your releases faster, since results are always valid.

What are Guardrail Metrics?

Guardrail metrics are business metrics designed to indirectly measure business value and provide alerts about erroneous results by defining boundaries.

What is a Power Analysis?

Power analysis is the process of estimating how many users you will need in order to detect an effect of a given size, or how small an effect

What is a False Negative?

In hypothesis testing, a false negative result indicates that the change being tested has not improved the key metric significantly when in fact, it has.

What is an Event Stream?

Event Streams as Telemetry: An event stream is a series of data points that flow into or out of a system continuously, rather than in batches.

What is a Change Advisory Board?

Change advisory board is a collective of representatives from different departments within a company who run change management processes.

What is a T-Test?

T-test is a type of hypothesis test which assumes the test statistic follows the t-distribution. Used to see differences between two groups.

What is False Discovery Rate?

What is a False Discovery Rate? Explore its role in understanding the accuracy of multiple hypothesis testing and how it differs from other measures.

What is a Data Pipeline?

Data pipeline automates the flow of data from one point to another. Defining how data is collected, and what schema it should be collected.

What is Configuration Drift?

A way to avoid the issues caused by configuration drift is to set up feature flags to test your code in production safely.

What is Simpson's Paradox?

When finding out if an admissions program is biased, a surgery is more successful, or an A/B test variant is superior, Simpson’s Paradox may come into play if the populations are not truly randomized.

What is a Kill Switch?

With a kill switch, when a feature breaks in production, you can turn it off immediately while your team analyzes the issue.

What is a Do No Harm Metric?

Do no harm metrics are metrics that teams use to ensure nothing bad is happening to their team's metrics due to a feature rollout.

What is a Type I Error?

A type I error is a type of statistical error where the hypothesis test gives a false positive result, when a perfect test would report a negative.

What is a Type II Error?

A type II error is a statistical error from a hypothesis test. A type II error occurs when a false null hypothesis is accepted.

What is Trunk-Based Development?

Trunk-Based Development is a software development model prioritizing continuous integration by committing to a central branch.

What is Statistical Significance?

Statistical significance is a measure of reliability for hypothesis testing. Identify genuine trends in data by separating experiment results from random noise.

What is False Positive Rate?

What is a false positive rate? It is a measure of accuracy for a test. Compare it with other accuracy metrics like the false discovery rate and sensitivity.

What is Testing in Production?

Testing in production is the process of continuing to test features that are deployed to the production environment by using staging to test your features.

What is Client Side Testing?

Client-side testing refers to any type of testing, A/B, multivariate testing or multi-armed-bandit testing occurring in the user’s browser or mobile device.

What is Server-Side Testing?

Server-side testing refers to any type of testing, A/B, multivariate testing or multi-armed bandit testing occurring on the web server.

What is Dark Launch?

Every product manager dark launches. Dark launch features are released to a subset of your users to enable testing before a wider deployment.

What is Hypothesis Driven Development?

Hypothesis driven development is the scientific method applied to software development.

What are Customer Experience Metrics?

We'll discuss some of the most commonly-used customer experience metrics and explain the benefits and drawbacks of each.

What is a Controlled Rollout Plan?

A controlled rollout allows the release of new features gradually, ensuring a good user experience before releasing them to larger groups.

What is Observability?

Observability for DevOps in feature monitoring is defined as the ability of the internal states of a system to be determined by its external outputs.

What is Progressive Delivery?

Progressive delivery uses feature flags to increase speed and decrease deployment risk, and uses a gradual process for rollout and ownership.

What is A/A Testing?

A/A testing involves running an A/B testing process with two identical versions in order to ensure the testing process is in working order.

What is a Multi-Armed Bandit?

A multi-armed bandit problem occurs when the benefits of each option are not known at the time of allocation, but can be discovered and resources reallocated.

What is A/B/n Testing?

A/B/n testing is the process of A/B testing with more than two different versions and differs from multivariate testing. The "n" refers to any additional tests.

What is Multivariate Testing?

Multivariate testing is a method of experimenting with different variations of elements to discover which variations will drive user behavior.

What is a Feature Rollout Plan?

A feature rollout plan allows the introduction of a set of new features to a group of your user base, often to a limited user set initially.

What is Mobile A/B Testing?

A/B testing for mobile apps is similar to standard software development A/B testing and includes app store A/B testing and mobile app A/B testing.

What is Usability Testing?

As a key part of the software development process, usability testing involves conducting real-world testing with a segment of your customer base.

Feature Flags in DevOps

Feature flags have been instrumental in progressing the DevOps philosophy to achieve continuous delivery of new, high-quality features.

What is A/B Testing?

A/B testing can be very similar, with only a change in button color, or very different, with a total change in the way a feature behaves.

Feature Flags

A feature flag is a dev tool used to safely activate or deactivate features for testing in production and experimentation. Learn about Split feature flags.

What is a Canary Deployment?

Learn the benefits of canary deployments in reducing risks, gathering feedback, and ensuring a smooth transition to production.

What is Chaos Engineering?

Learn how chaos engineering can help organizations build more robust and reliable systems by proactively uncovering vulnerabilities.

What are Feature Flags?

Learn how feature flags can help teams release new features incrementally, manage risk during deployments, and gather feedback from users in real-time

Testing & Resilience