Integrating Smoke Testing into Your CI/CD Pipeline: What DevOps Needs to Know | Harness Glossary

AI coding tools can increase code velocity, but how well they work still depends on keeping both throughput and stability. Smoke tests help keep things stable by finding "this deploy is fundamentally broken" failures early, before deeper suites and promotions make the problem worse.

The goal isn’t to test everything. It’s to add a fast “go/no-go” gate that validates the essentials in the environment you just deployed to.

Harness can help you operationalize that gate with consistent pipeline enforcement, standardized failure handling, and optional post-deploy verification. With Continuous Delivery and Continuous Integration, you can turn smoke tests from slow, manual steps into smart deployment gates.

What Is Smoke Testing?

Smoke testing in CI/CD is a short set of checks that answers one question: “Is this build healthy enough to keep moving through the pipeline?” It’s sometimes called build verification testing because it validates the essentials, fast.

Teams typically run smoke tests after a deployment (for example, to a staging environment) to confirm:

• The application starts successfully
  
• The service is reachable
  
• The most critical workflows still work
  
• Dependencies (like databases or downstream services) are available
  

They are a first-pass check that gives quick feedback and finds problems with deployment and configuration that other tests might miss.

The Purpose of Smoke Testing in DevOps and CI/CD

Smoke tests help protect your delivery process, especially when you deploy often and your environments change a lot.

What Smoke Tests Protect You From

Smoke tests are meant to catch problems that are costly if found too late:

• Broken startup and routing: The service doesn’t boot, readiness never goes green, or key routes return 404/500.
  
• Configuration and secrets issues: Missing env vars, invalid flags, expired credentials, or incorrect secret wiring.
  
• Dependency access problems: The app can’t reach the database, cache, queue, or a required downstream API.
  
• Unexpected packaging/runtime mismatches: The build is “green” but the runtime image is missing a dependency, migration, or asset.

Why This Matters In A CI/CD Pipeline

Smoke tests speed up delivery by stopping wasted effort.

• Block broken builds early. Stop unstable releases from advancing into QA, performance testing, or production.
  
• Smoke tests find problems at deployment that build-time tests can’t catch. They check the deployed system, like settings, secrets, routing, and connections, not just the code.
  
• Failing early saves time and resources. Running smoke tests right after deployment lets you find problems before spending more time in the pipeline.

The Outcomes Teams Get When Smoke Tests Are A Real Gate

When teams trust and always use smoke tests, they usually see these results:

• Cleaner promotions: Fewer “staging looked fine, prod fell over” moments because core paths are validated before promotion.
  
• Faster recovery: Failures surface earlier, with a smaller blast radius and a clearer rollback decision.
  
• More predictable pipelines: Less noise from downstream suites triggered by fundamentally broken deploys.

Smoke Testing Key Characteristics: What Makes a Smoke Suite "Good"?

A good smoke test suite is kept small on purpose. The goal is to get clear signals, not to cover everything.

Fast By Design (Minutes, Not Hours)

As a rule of thumb, smoke tests should be fast enough to run as a promotion gate, ideally in minutes, not hours. If they slow down delivery, they’ll stop being used (or stop being trusted).

To keep speed predictable:

• Keep the suite small (often 5–10 checks)
  
• Prefer API-level assertions over complex UI flows
  
• Run smoke tests in parallel when it doesn’t reduce determinism

Focused On Critical Paths

Smoke tests should focus on the most important parts:

• Auth / login
  
• A core API endpoint (read + write, if applicable)
  
• A core page or workflow that reflects real value (dashboard, checkout, search)
  
• Dependency connectivity (DB/cache/queue) when those are frequent sources of failure

Reliable And Repeatable

Smoke tests only work as gates if teams trust them. Make sure reliability is a top priority:

• Use deterministic assertions and stable test data
  
• Avoid brittle UI selectors when an API check will do
  
• Keep timeouts reasonable and error messages specific

Smoke Testing vs. Sanity Testing: Which One Do You Need (and When)?

Both smoke tests and sanity tests are "quick checks," but teams don't always use the same words to describe them. Microsoft says that people sometimes call smoke tests "sanity tests," "acceptance tests," or "build/release verification tests."

A practical way to separate intent (even if your team uses different labels):

• Smoke testing: “Is this build stable enough to keep moving?” Broad and shallow, meant to act as a gate after deploy.
• Sanity testing: “Did this specific change work?” Narrow and targeted, focused on recent edits or a hotfix.

Simple rule:

• Run smoke tests after deploy (staging, canary, or pre-promotion).
  
• Run sanity tests after targeted fixes when you want confidence in a specific area.

Types of Smoke Testing (What DevOps Teams Actually Use)

Most teams use a mix, then converge on what’s easiest to run reliably in CI/CD.

• Automated post-deploy smoke tests: Run right after a deploy to confirm the service is reachable and core paths work.
  
• API smoke tests: Fast HTTP checks with clear assertions (often the most stable signal).
  
• Minimal UI smoke tests: 1–2 high-value journeys where UI breakage is a real risk and can’t be validated via API.
  
• Pre-promotion smoke tests: A gate between environments (staging → prod).
  
• Production smoke tests (optional): Read-only checks that validate critical functionality without impacting users.

If you’re moving to progressive delivery, smoke tests are especially useful during canary steps; run them on the canary before increasing traffic.

And if you want deeper automated confidence, Harness Continuous Verification can validate post-deploy health signals (metrics/logs) using ML-based analysis.

When Should You Run Smoke Tests in Your CI/CD Pipeline?

Putting smoke tests in the right places at key pipeline gates gets the most value out of them while keeping costs low. If you run them too soon, you're testing an incomplete deployment. If you run them too late, you're wasting time and computing power on a release that was never possible.

Here are the placements that work well across most teams:

• Right after a deployment to staging (the default baseline). As soon as the release is in staging, run smoke tests. This catches problems with configuration, secrets, routing, and dependencies in the same place you make decisions about promotions.
• Before moving on to production, which is your "go/no-go" gate. Before you release something, think of smoke tests as the last test you need to pass. As part of the release flow, Microsoft's baseline architecture guidance says to run smoke tests and use failures to stop the pipeline and roll back when necessary.
• As part of either canary or progressive delivery. If you roll out slowly, do smoke tests on the canary first, and then do a slimmed-down version again when traffic picks up. The goal is to make sure that important paths work when routing, authentication, and dependencies are set up correctly.
• On temporary PR environments (optional, high-signal). If you create preview environments for each pull request, smoke tests are a quick way to check things before merging. Keep them small so they don't slow down the process of developers changing things.
• In production (with careful planning). Production smoke tests can be useful, but make sure they are read-only and don't cause any problems. Microsoft's basic architecture also talks about running smoke tests in production and going back to the last known good state if they fail.

How to Perform Smoke Testing

You don't need to completely change your pipeline to add useful smoke tests. Begin with a small, dependable set and let automation take care of the work.

Microsoft’s Engineering Fundamentals Playbook puts it well: smoke tests should cover only the most critical path and keep execution time and complexity to a minimum—because they’re meant to act as a gate.

Step 1: Choose 5–10 Checks That Represent “Success”

Pick checks that tell you whether the service is fundamentally alive in the environment you deployed to. A good starter set usually includes:

• Readiness: A readiness endpoint returns success (or Kubernetes readiness is true)
  
• Auth: A basic auth/token flow works (if your product requires login)
  
• One core read: The most-used API or page loads expected data
  
• One core write (optional): Only if you can do it safely with idempotent test data
  
• Dependency confidence: Database/cache/queue connectivity, when these are frequent failure points
  

If you’re not sure where to start, look at the last few incidents and ask, “What would have caught this within two minutes?”

Step 2: Pick the Right Level: HTTP, API, Minimal UI

Start with the lightest checks that still give you a trustworthy signal:

• HTTP reachability and status checks (seconds): Confirm the service responds and key routes aren’t broken.
  
• API assertions (minutes): Validate auth + a couple of core endpoints with clear assertions.
  
• Minimal UI checks (only when needed): 1–2 revenue- or mission-critical journeys that you can’t validate at the API layer.

A practical guideline: if a UI step is flaky, it doesn’t belong in smoke. Move it to a broader suite and keep smoke deterministic.

Step 3: Make Smoke Tests a Real Gate (Not a “Nice-to-Have”)

Smoke tests only help if failures actually stop the build from moving forward.

A clean pattern looks like this:

1. Deploy to staging
   
2. Run smoke tests
   
3. If smoke passes → continue to deeper suites or promote
   
4. If smoke fails → stop, capture artifacts, decide fix-forward vs rollback
   

You can enforce this gate with failure strategies (for example, abort, rollback, or mark failure) and control what happens when a step fails.

For CD pipelines, Harness also supports pipeline rollback strategies so teams can standardize what “safe failure” looks like when a gate fails.

Step 4: Make Failures Debuggable and Actionable

When smoke tests fail, speed matters. Your goal is to help someone fix the issue without having to re-run the entire deployment just to gain context.

Build these into your smoke stage:

• Capture request/response details (with secrets redacted)
  
• Upload artifacts on failure (logs, test output, screenshots for UI checks)
  
• Include “what to check next” in the assertion message (service logs, rollout status, dependency endpoints)
  
• Add correlation IDs so you can trace a single failing request across logs and traces

Best Practices for Smoke Testing That Won't Slow Releases

Smoke testing is most effective when it remains small, reliable, and focused.

• Keep smoke tests small and focused only on the most important paths. They are not meant to be a mini regression suite. Microsoft also recommends keeping smoke tests simple and focused on the critical path.
  
• Set a time limit for your smoke tests, such as 3–7 minutes, and stick to it. If tests take longer, treat it as a problem, since slow smoke tests can slow down delivery.
  
• Choose reliable checks instead of fragile UI automation. API-level checks are usually faster and more dependable.
  
• Make sure your tests are safe to run more than once. Use actions and test data that won’t cause problems if you need to retry.
  
• Treat flaky tests like production bugs. Test flakiness can become a big problem as it slows teams down with unreliable results.
  
• Don’t keep retrying tests until they pass. If you allow retries, keep them limited and make them visible. Flaky tests add confusion because failures may not be caused by code changes.
  
• Build in good security practices. Hide secrets, don’t log tokens, and keep production smoke tests read-only.
  
• Improve your process with automated verification. Use smoke tests as a quick gate, then check deeper health signals after deployment.

Smoke Test in Action: Real-World Examples for Your Software Product

It’s easiest to build smoke tests by starting with real user workflows and past failures your team has experienced.

Example 1: Web Application (Critical User Path)

Goal: confirm the app is usable end-to-end after deploy.

Minimal smoke checks:

• GET /health returns success and expected version metadata
  
• Login succeeds (via API if possible)
  
• A core page loads (dashboard, search, checkout start)
  
• A static asset or API call required by the page returns success

If you add UI checks, keep them minimal:

• Login → load dashboard → perform one primary action → logout

Example 2: API Service (Auth + Core Endpoints)

Goal: confirm the most important API surfaces are working.

Starter smoke checks:

• POST /auth/token returns a valid token
  
• GET /v1/me returns expected fields
  
• GET /v1/<core-resource> returns a 200 and a non-empty payload
  
• Optional safe write: POST /v1/<resource> with idempotency key → 200/201

Example 3: Kubernetes Deploy (Rollout + Readiness + Endpoint)

Goal: confirm the platform and workload are actually ready.

Common checks:

• kubectl rollout status deployment/<name> confirms rollout completes
  
• Pods are Ready and not crash-looping
  
• Service endpoint returns success
  

Kubernetes documents kubectl rollout status and other rollout commands as standard ways to check deployment progress.

For readiness/liveness configuration, Kubernetes provides guidance on readiness and startup probes (helpful in avoiding “it’s up but not ready” failures).

Example 4: Microservices (One End-To-End Path + Dependencies)

Goal: validate a single critical transaction across services.

Pick one path that matters (for example: create order → reserve inventory → charge payment → emit event). Then add just enough checks to prove:

• Each service in the path is reachable
  
• Dependencies are available
  
• The transaction completes with expected output

A Simple Smoke Testing Plan You Can Start This Week

You don’t need months to build a good smoke testing plan for CI/CD. Start by picking 5–10 checks that cover your most important user flows and API endpoints. Run these automated tests right after deploying to staging and before promoting to production. Your smoke suite should finish in under 5 minutes and stop the pipeline if it finds problems.

Make sure someone is responsible for maintaining the suite and set strict time limits to keep it focused. When you’re ready to add smoke testing with strong gates, look into how smart verification can improve your delivery pipeline.

TL;DR: Smoke tests aren’t about catching every bug — they’re about quickly answering one question: is this deployment safe to promote?

Explore Harness Continuous Delivery and Continuous Integration to start releasing with confidence.

CI/CD Smoke Testing: Frequently Asked Questions (FAQs)

These practical answers address the most common CI/CD smoke testing FAQ topics that come up when scaling deployment pipelines across enterprise environments.

How many smoke tests should we run?

Start with 5 to 10 important checks that show the service is working after it has been deployed. Add more only if they clearly help, and keep the suite small so it stays fast and works.

How long should a smoke suite take?

Aim for smoke tests that run in minutes, not hours. They should be fast enough to work as a gate without slowing down releases. Teams might skip them or stop trusting the results if they take too long.

Smoke tests vs. health checks: what’s the difference?

Health checks show that the service is working. Smoke tests show that the service works for key tasks right after it is set up.

Should smoke tests run in production?

You can run smoke tests in production, but make sure they are read-only, focused, and safe to run more than once. Microsoft says that the release process should include production smoke tests and a plan for rolling back if they fail.

Should smoke tests be API or UI?

Start with checks at the API level because they are quick and dependable. Add simple UI smoke tests only for cases where you can't use the API.

What causes flaky smoke tests (and how do we fix them)?

Timing problems, unstable environments, or outside dependencies are common causes of flaky tests. This makes the results hard to understand, which teams might ignore. To fix flakiness, use clear waits instead of sleeps, make sure your test data is stable, and keep your smoke tests small and reliable.

Where should smoke tests live (app repo vs separate repo)?

It's best to keep smoke tests in the same repo as the code they check so that they change with the code. If you use shared tools, put the framework in one place, but let each service team run their own tests.

Are smoke tests the same as end-to-end tests?

No. Smoke tests are intentionally small and fast, designed to validate that a deployment is fundamentally healthy. End-to-end tests are broader, slower, and validate full workflows across systems.