A healthy server does not mean a successful user
What has to be settled before the monitoring tool is the work a user must finish in your service, and the evidence that the work succeeded. A server can answer with CPU and memory to spare while a user cannot log in, cannot complete an order, or cannot use the file they uploaded. That service is not in a reliable state.
Infrastructure metrics are needed to explain why something broke. They cannot be the final definition of service reliability. Google SRE separates the symptom — "what is broken" — from the cause — "why is it broken". CPU saturation or a database connection error may be the cause, but the fact that the user could not complete important work comes first. An HTTP 200 returned alongside a wrong result also counts as a failure.[1]
So the design order runs from the critical user journey to the success outcome, the start, completion, and failure events, the SLI, the measurement point, the instrumentation required, and only then the tool. Choose the tool first and the dashboard fills with whatever is easy to collect — CPU, memory, request count, status codes. Start from the journey and it becomes visible which data is missing, where new instrumentation is needed, and what product or storage the work actually requires.
None of this makes status codes useless. A status code shows one step of request handling. The problem begins when that one step is read as the whole user outcome. In async work especially, accepted and completed have to stay apart. Putting a job on a queue is an intermediate checkpoint. The journey completes only once the worker has run, the external API has answered, the data is stored, and the result has reached the user.
- Login
- Signal mistaken for success: The auth API returned HTTP 200 / What the user experienced: Token storage or screen init failed, so the protected view never opened / Outcome to verify: The authenticated screen became genuinely usable
- Order and payment
- Signal mistaken for success: The payment request was approved / What the user experienced: The order ledger was not created, or the receipt never appeared / Outcome to verify: Payment, order, and display state completed consistently
- File upload
- Signal mistaken for success: The upload intake API succeeded / What the user experienced: The async conversion job failed and the file cannot be opened / Outcome to verify: The post-processed file became genuinely usable
- Search
- Signal mistaken for success: The search API responded normally / What the user experienced: A stale index or wrong filter hid the needed results / Outcome to verify: Results rendered and met accuracy and freshness conditions
Pick the critical journeys first
A critical user journey is the sequence of actions a user performs to obtain the core value of the service. In online commerce that may be product search, adding to cart, and completing a purchase. Each journey spans several components and requests, so one server log rarely decides success or failure on its own. Google SRE also says the activities that matter to users have to be identified before deciding what to measure.[1]
Not every screen and API needs to be critical. Narrow the scope with the six criteria below. Low frequency does not mean low importance: month-end settlement, administrative approval, and bulk data submission run rarely but hurt badly when they fail.
Record each selected journey in the fourteen fields of the User Journey Canvas.
The completion boundary has to be evidence that the state the user wanted came into being, not that the API responded. Conversely, a journey the user deliberately cancelled must not be counted as a service failure. What you include in and exclude from the denominator matters as much as the SLI itself.
- User value
- Would failure prevent the user from achieving the purpose of using the service?
- Harm
- Could failure cause hard-to-reverse financial, data, permission, or scheduling harm?
- Business importance
- Is it directly tied to revenue, contract delivery, or a core customer workflow?
- Dependencies
- Does it cross external authentication, payment, messaging, storage, or other systems?
- Recovery difficulty
- Does it require more than retrying, or need human confirmation?
- Reach
- Do many users rely on it, or is it infrequent but costly to fail?
Field | What to record |
|---|---|
| Journey Name | A user-recognizable verb phrase, for example: submit a file and check its processing result |
| Actor | User, role, or system performing the journey |
| User Goal | Final outcome the user wants |
| Start Boundary | Observable event marking the start |
| Completion Boundary | Event proving the user goal was met |
| Failure Boundary | Definitions of explicit failure, timeout, wrong result, and stuck intermediate state |
| User Harm | Actual harm caused by failure |
| Valid Attempt | Valid attempts included in the SLI denominator |
| Excluded Attempt | Conditions and reasons for exclusion, including user cancellation and invalid input |
| Critical Segments | User types, regions, devices, and tenants to examine separately |
| Dependencies | Internal and external systems the journey depends on |
| Outcome Owner | Team accountable for final-outcome correctness |
| Current Evidence | Start, completion, and failure events already available |
| Blind Spots | Unmeasured stretches and their impact |
Operations connected from observation to improvement — Respond using service indicators and alerts, then connect change records and incident reviews to improvements.
Turn the journey into outcome, event, and SLI
An SLI, or service level indicator, is a measurement of the quality of the outcome the service delivered to users. In practice it is easiest to reuse when written as the ratio of good events to all valid events: the user journey success rate is the number of journeys confirmed successful divided by the number of measurable valid journeys.
Google SRE recommends specifying the SLI first as an outcome that matters to users, then treating how to measure that outcome in real data as a separate implementation question. The same SLI can be implemented from server logs, a load balancer, synthetic tests, or client instrumentation, each with different accuracy, coverage, and cost.[2] OpenSLO likewise defines the SLI as how to read a metric from a data source and expresses good or bad separately from total. SLO targets, evaluation windows, and alert policies are separate objects from the SLI.[3]
No latency threshold or target success rate is set here. First measure start and end consistently and secure the distribution and raw counts. Which level to aim for and over what window belongs to the SLO policy that follows.
Write outcome, events, candidate SLIs, and blind spots for each journey. For login before work, the outcome is that an authenticated user can genuinely use the protected screen; the events are login_submitted → session_created → protected_view_ready; the blind spot is external auth succeeding while client initialisation fails. For order and payment completion, the outcome is payment, order ledger, and receipt state agreeing; the events are checkout_submitted → payment_confirmed → order_committed → receipt_visible; the blind spot is webhook failure and duplicate retry after approval. For file upload and processing, the outcome is that the file becomes genuinely usable rather than merely accepted; the events are upload_started → upload_accepted → processing_started → file_ready; the blind spot is job loss after enqueue, dead letters, and missing callbacks. For search, the outcome is results rendered that reflect the required data scope and freshness; the events are search_submitted → results_rendered plus index_watermark; the blind spot is an empty HTTP 200, a stale index, and missing datasets. For approval work, the outcome is that the right approver acted and the downstream state followed; the events are approval_requested → approval_recorded → downstream_state_updated; the blind spot is an approval record whose downstream sync failed.
Event names vary by organisation. What matters is linking start, intermediate step, and final outcome under one journey ID. Where retries occur, an idempotency key or correlation ID keeps the same attempt from being counted more than once.
SLI axis | Question it answers | What to measure |
|---|---|---|
| Success rate | Did the user finish the work that mattered | Completed valid journeys ÷ all valid journeys |
| Latency | How long did the user wait for the outcome | Time distribution from user start to real completion |
| Correctness | Was the completed outcome right | Outcomes meeting business rules and invariants ÷ outcomes checked |
| Freshness | Does the data the user saw reflect the state at that moment | Gap between the read time and the update time of source data, index, or aggregate |
Decide where to measure
The closer the measurement is to the user, the better it reflects real experience — and the higher the implementation cost and chance of missing data. Measuring inside the server is easier to collect reliably but blind to failures before the request arrived and to client-side rendering problems. Google SRE also explains that one SLI specification can be implemented through server logs, browser-based synthetic tests, or client instrumentation, each carrying different gaps.[2]
No single point decides everything. In general you need one point that proves the user outcome and one internal point that explains the cause. Which of logs, metrics, and traces to use is a separate signal-selection question.
Measurement point | Captures well | Misses easily | Fits |
|---|---|---|---|
| Client / real user monitoring | Rendering, user network, browser and app errors, true end-to-end latency | Failures where the app never started so the instrumentation never ran, blocked SDKs, sample bias | Screen usability and actual user experience |
| Edge / load balancer | Requests that reached the server, status codes, boundary latency, backend routing | Rendering, business outcome, async work after the server | A close approximation of request success and latency near the user |
| Application service | Explicit business success and failure, error reasons, processing stages | Failures before the request reached the service, client usability | Processing outcomes that carry business meaning |
| Async worker / queue | Job start, retry, completion, stalls, dead letters | The link between the first user action and the final rendering | Async journeys such as upload conversion, message delivery, aggregation |
| Business event / data state | Real business state such as order created, approval recorded, settlement closed | Whether the outcome reached the user's screen, and delay or loss of the event itself | Proving final completion and correctness |
| External dependency boundary | Results and latency of external auth, payment, and messaging calls | Internal storage or display failures after the external call | Cause classification and provider impact |
| Synthetic transaction | Repeated checks of reachability and key paths even without real traffic | Real user distribution, real data variety, some personalisation problems | Low-traffic services, full-outage detection, baseline end-to-end checks |
Five blind spots recur
First, classifying HTTP 200 as success on its own. A status code is a protocol result. If the body was wrong, only part of the data returned, or an error message was wrapped in a 200, the user failed. Google SRE classifies an HTTP 200 returned with wrong content as an implicit failure and says such errors need end-to-end verification.[1] Completion evidence outside the status code is therefore required — a business result code, a screen-ready event, a data invariant.
Second, confusing intake with completion in async requests. A 202 Accepted, a successful enqueue, or an issued job ID is not the completion of a journey. The same journey ID has to hold until the outcome the user needs exists. Decide too what happens when the completion event never arrives. Counting only explicit failure events lets perpetually pending work vanish from the statistics. A state whose outcome cannot be determined is not a success; expose it separately as unknown or stuck.
Third, removing external dependency failures from user metrics. Even when the external payment or auth provider failed, the user could not use the service, so the user journey SLI keeps it as a failure. Tag the cause as a dependency to separate it from internal problems. Removing it from the user outcome because a provider is at fault makes internal metrics look better while moving them away from real experience. Keeping an end-to-end user SLI and splitting cause through component and dependency metrics works better.[1]
Fourth, reading only the ratio on low traffic. With few attempts, one failure swings the ratio hard, and with no requests at all there is no chance to observe a failure. Google SRE notes that a user with very few requests can be availability-perfect by chance or fall very low from one failure.[1] Five practices follow on low traffic. Show the raw numerator and denominator alongside the ratio. Do not convert "no requests" into "healthy". Cover full outages of the main path with synthetic transactions. Do not mix real user outcomes and synthetic results into one denominator. Set targets and windows in a separate policy after reviewing enough data and the real user harm.
Fifth, treating unmeasured journeys as successes. Client event collection outages, sampling, wrong filters, schema changes, and missing correlation IDs all produce results you cannot determine. Counting the unobserved as successful makes an instrumentation outage look like a reliability gain. Coverage has to be checked separately from the SLI: the share of expected start events whose outcome can be linked, event collection lag, schema errors, and the count of unclassified outcomes.
Close instrumentation gaps as a list
An unchecked item does not translate straight into a product purchase list. Register first what data is needed and why as an instrumentation backlog. Then judge whether existing logs or events can answer it, whether an application change is required, or whether separate measurement infrastructure is needed.
Area | Checks |
|---|---|
| Journey definition | Actor and user goal are explicit. Start, completion, and failure boundaries are described as user outcomes rather than system components. Denominator exclusions such as user cancellation and invalid input are recorded with their reasons |
| Event linkage | Every valid attempt carries a unique journey or correlation ID. Retries and duplicate requests normalise into one journey. Failure, stall, and unknown-result events are distinguished from completion |
| Async processing | Intake and final completion are separate events. The journey ID travels across queue, worker, webhook, and callback. Lost jobs and dead letters are findable |
| External dependency | External call failures land in the user journey outcome. Provider failure and internal follow-up failure carry different cause codes |
| Correctness and freshness | Invariants beyond status codes verify business correctness. Creation, index, or aggregation timestamps or watermarks are recorded. Partial success over a subset of data is identifiable |
| Measurement coverage | A measurement point exists for failures that never reached the server. There is a fallback when client instrumentation did not run. A synthetic path covers low-traffic periods. Numerator, denominator, and unclassified counts appear alongside the ratio |
| Segmentation and ownership | Minimum segments exist so user, region, device, and tenant problems are not hidden. Required dimensions are collected without personal or secret data. High cardinality does not inflate aggregation and storage cost without bound. Outcome definition and instrumentation implementation each have an owner. A schema change triggers an SLI impact review. A break in measurement data is treated as an operational problem |
Hand the SLI on to the next decision
Having a user journey SLI is not a licence to attach alerts or deployment gates immediately. The meaning of the SLI and the quality of its data have to be handed over in a form other operational policies can use. The chain runs from the critical user journey to the user outcome, the start, completion, and failure events, the SLI specification, and the SLI implementation with coverage evidence — and from there branches into SLO targets and error budgets, alert conditions and routing, signal selection, the incident runbook, and deployment gates and rollback behaviour. OpenSLO likewise represents the SLI, the SLO target and window, and the alert policy as separate structures. That separation reads better as a contract that keeps measurement definition apart from operational decisions than as a format required by any one tool.[3]
The Handoff Contract has the five entries below. Each separates inputs to hand over from decisions owned by the next stage.
There is no reason to plug a generic target such as 99.9% in at this stage. A target is set by reviewing user expectation, harm, the measurement data you now hold, cost, and operational responsibility together.
- SLO·Error Budget
- Inputs to hand over: Journey definition, good/total events, measurement points, coverage, segments, outcome finalization. Decision at the next stage: Target, evaluation window, error budget and usage policy.
- Alert
- Inputs to hand over: Meaning of user harm, failure reason codes, data lag, low-traffic characteristics. Decision at the next stage: When to wake a person; Page/Ticket/Dashboard; suppression and deduplication.
- Deployment Gate
- Inputs to hand over: Release-correlated SLI, before/after comparability, instrumentation health. Decision at the next stage: Deployment stop, approval, exception conditions and policies.
- Signal Selection
- Inputs to hand over: Failure stretches to investigate and measurement points. Decision at the next stage: Which logs, metrics, traces, and profiles to place where.
- Incident / Rollback
- Inputs to hand over: User impact, failing journeys, affected segments. Decision at the next stage: Mitigation, recovery, communication, rollback or roll-forward actions.
Tools come after the measurement contract
Once the journeys and SLIs are defined, the questions you put to a monitoring tool change. Can it collect events at the measurement points you need? Can it link client, server, async work, and external dependencies into one journey? Can it compute the numerator and denominator of good and valid events without losing either? Does it refuse to hide instrumentation gaps and unknown outcomes? Can it separate user, region, device, and tenant problems to the depth you need? Can it tie a deployment version to a user outcome? Can the measurement definition be reviewed and moved outside that one product?
Where those questions have no answer, a feature-rich tool adds dashboards rather than certainty. Saying a service is reliable does not mean the server is on. It means users start the work that matters and get the right outcome within the time they need. Infrastructure metrics let you find why that outcome degraded. What counts as reliability is decided by the user journey.
