COD Risk Scoring: Deciding Which Orders Get Friction

Most brands running Cash on Delivery apply their controls uniformly. Everyone gets the confirmation message, or nobody does. The advance is on for all COD orders or off for all of them. This is the one decision where uniformity is clearly wrong, because the orders are not alike: a repeat buyer in a lane you deliver reliably and a first-time buyer on a thin address in a lane that returns a quarter of what you send are the same row in your dashboard and completely different bets.

A risk score fixes that. Not because scoring is sophisticated, but because it lets you answer a narrower and much more useful question than “is COD risky”: which of today’s COD orders should I treat differently, and how?

Friction is a budget, and you are already overspending it somewhere

Start from the premise that every control is a cost, not a safeguard.

A verification message costs a per-message fee, a delay to dispatch while you wait for an answer, and a small share of orders that would have delivered fine but now get cancelled because the customer read a confirmation request as a reason to reconsider. An advance payment costs you the orders that abandon rather than pay it. A prepaid-only policy on a weak pincode costs you that pincode’s revenue. A slower courier with better COD performance costs you delivery days, which has its own return cost.

So the question is never “should we verify.” It is “on which orders is the expected saving from verifying larger than what verifying costs.” That ratio varies enormously across your order book, and a score is simply the instrument that sorts orders by it.

This also explains why uniform policies feel like they stop working. A store that puts a confirmation message on every COD order spends most of its message budget and most of its dispatch delay on orders that were never going to come back, and the arithmetic looks mediocre. The same spend, concentrated on the worst-scoring fifth of orders, looks completely different.

The signals are already in your order table

You do not need a data vendor to start. Four groups of signals sit in Shopify and your courier reports already, and between them they carry most of what is predictable.

Order and cart shape. Order value relative to your own average — the tails matter in both directions, since unusually large COD orders carry more loss per failure and unusually small ones often carry more casual intent. Quantity of a single size or variant, which in apparel frequently means a customer who intends to keep one and refuse the rest. Whether a discount code was used, and which one; codes from broad paid campaigns behave differently from codes given to an email list. Time of order, which is a weak signal on its own and a useful one in combination. Whether the cart was assembled over a session or added and checked out in under a minute.

Address and phone quality. The mechanical checks covered in address quality: pincode consistent with city and state, a digit present in the address line, address length above your own floor, phone number of a plausible shape. These are cheap to compute and they are the signals least likely to be unfair to a customer, because they describe the data rather than the person.

Buyer history. The strongest group, and the most underused. Has this phone number ordered before, and what happened? Has this address delivered successfully before? Has either previously produced a refusal or a return? A phone number with two prior returns behind it is worth more than every other signal combined. A phone number with three successful deliveries should move an order to the bottom of your friction queue regardless of what the rest of the score says — and most stores never check, because the lookup is on phone and address strings rather than on a customer record.

Destination lane. Pincode-level and region-level delivery outcomes from your own courier data, which you need anyway for courier allocation. Lane is a property of the shipment, not the customer, and it belongs in the score because it changes the expected cost of shipping the order unverified.

Four groups, perhaps a dozen fields. That is a complete first-generation score.

What Shopify’s own risk tooling does and does not cover

Shopify has risk machinery built in, and it is worth being precise about what it addresses, because the overlap with COD is smaller than the name suggests.

Shopify’s Help Center documentation on fraud analysis states that “Fraud analysis includes fraud indicators and, for eligible orders, a fraud recommendation”, and that “the fraud recommendation indicates whether an order has a low, medium, or high risk of a chargeback due to fraud.” The indicators it lists are payment-instrument signals — whether the card passes Address Verification System checks, whether the correct Card Verification Value was supplied, whether the customer’s location matches the payment method, unusual device or network activity, and attempts with more than one card. The documentation also notes a scope limit: “Fraud analysis works with online credit card orders that Shopify can verify.”

Read that carefully. The thing being predicted is a chargeback, and the evidence is drawn from a card transaction. A COD order has no card, no AVS result, no CVV, and cannot produce a chargeback. The failure mode you care about — a parcel that is shipped, refused at the door, and returned — is a different event with different causes, and Shopify’s documentation does not claim to predict it.

What Shopify does provide is somewhere to put your own assessment. Its Admin GraphQL API documents an OrderRiskAssessment object described as “A risk assessment for an order”, carrying a riskLevel and facts, where “Fact values are specific to the provider”, plus a provider field that is “The app that provided the assessment, null if the assessment was provided by Shopify.” The documented risk levels are HIGH, MEDIUM, LOW, NONE and PENDING. The Help Center adds that indicators and recommendations from fraud-prevention apps can be viewed alongside Shopify’s own, and that “When an order has assessments from multiple providers, select the provider whose assessment you want to review.”

The practical consequence: a COD risk score is a thing you or an app computes, and the platform’s contribution is the place to surface it so your fulfilment team sees it on the order rather than in a spreadsheet. Do not treat the built-in fraud recommendation as a COD signal. Equally, do not ignore it — on your prepaid orders it is doing a job your COD score is not.

Build it as points, not as a model

The instinct is to train something. Resist it for the first two quarters, for three reasons: you will not have enough labelled failures to train on until you have run COD at volume for a while, you cannot explain a model’s output to a support agent who has to make a call, and you cannot audit a model for the kinds of unfairness that get a brand into trouble.

Use additive points instead. Each rule adds or subtracts; the total lands in a band.

A workable starting shape, with weights you will change within a month:

Subtract for successful history — a delivered order on this phone or address, more for several. This is the single most important line in the table, and most scores omit it because they are built as fraud detectors rather than as friction allocators.

Add heavily for a prior refusal or return on this phone or address. Add heavily for an address that fails a mechanical check. Add heavily for a lane whose COD return rate is in your worst decile.

Add moderately for order value far above your average, for multiple units of one variant, for a first-time buyer with no history at all, and for a high-discount code from a broad acquisition campaign.

Add slightly for the weak signals — odd-hour orders, a one-minute session, a free-text address with no landmark. Keep these small. Their job is to break ties, not to decide cases.

Write the table down somewhere a human can read it, in a spreadsheet if nothing else, and version it. The moment you cannot reconstruct why an order scored what it did three weeks ago, you have lost the ability to improve the score.

Three bands, three different actions

A score with no attached action is a report. The bands are the product.

Low — ship as is. No message, no call, no delay. The discipline here is keeping this band large, because every order you move out of it is an order you are taxing. If low is not the clear majority of your COD volume, your thresholds are wrong, not your customers.

Medium — confirm, then ship. One verification attempt on the cheapest channel that works, escalating only on silence, in the sequence set out in WhatsApp vs SMS vs IVR. The decision this band needs made in advance is the no-response policy: ship anyway, or hold. For most stores, shipping a medium-band order after a silent verification attempt is correct, because silence is weak evidence and holding costs a dispatch day.

High — change the terms. Not “verify harder.” Ask for an advance or offer prepaid with an incentive, as covered in how much COD advance to collect and COD to prepaid conversion. The point of this band is that a conversation does not fix it — the economics of shipping this parcel unsecured are bad, so either the terms change or you accept a known loss rate on a small, deliberately chosen slice.

Cancellation is not a band. Keep it as an exception, used on a handful of orders a month, with a named person responsible. Teams that cancel on a score drift upward in how readily they do it, and the cost of that drift is invisible because cancelled orders leave no return to count.

Calibration is the whole job

The weights do not matter much. The thresholds do, and they are set from your own outcomes.

Score your last ninety days of COD orders retrospectively, with the outcome already known but not used as an input. Then build one table: score band on one axis, actual outcome on the other. You want to see the return rate rise monotonically as the score rises. If it does not — if your middle band returns at the same rate as your low band — the score is not separating anything and you should stop and look at which single signal does carry separation. Usually it is buyer history, and usually the rest of the table is decoration.

Then set thresholds by cost, not by percentile. For each band you can compute, from your own numbers, the expected loss of shipping unverified: the return rate times the cost of a return — forward freight, return freight, handling, and the capital locked in stock for the round trip. Compare that against the cost of the control: message fees plus the dispatch delay plus the share of verified orders that cancel. The threshold belongs where those two cross.

Run it as a holdout for the first month. Let a random tenth of medium-band and high-band orders through untreated, and compare. Without a holdout you cannot distinguish a score that works from a season that was going to be good anyway, and you will end up defending a number you cannot support.

Closing the loop

The score has to eat its own results, and this is where most implementations quietly die.

Write the score and the band onto the order record at the moment of scoring, not computed later on demand. When the parcel resolves — delivered, refused, returned, undelivered-and-recovered — you need the score as it stood at decision time, or you cannot measure anything. Re-scoring with today’s rules against last month’s orders will flatter you every time.

Once outcomes are attached, two things become possible. You can refresh the weights monthly on real labels. And you can maintain the one list that pays for all of this: addresses and phone numbers with repeat bad outcomes, which should route straight to the high band. Keep that list bounded and expiring — a single refusal eighteen months ago should not follow a customer forever, and a customer who later takes three deliveries has earned their way out.

Where this goes wrong

Scoring people instead of orders. Build the score from order and shipment properties and from this store’s own transaction history with this phone or address. Keep protected and quasi-protected attributes out of it entirely — and note that in India some geographic signals correlate closely with community, so a rule that penalises a locality rather than its measured delivery performance is both unfair and a worse predictor. Lane performance from your own courier data is a legitimate operational input. “This kind of area” is not.

Letting the score reach the customer. Nothing in the score belongs in a message. A customer told they have been flagged responds badly and reasonably. The band changes what you ask for, not how you speak.

Thresholds that drift upward. Friction is satisfying to add, and each addition looks locally justified. Review band volumes monthly against the same chart you set them from, and treat a growing high band as a problem to investigate rather than a sign of vigilance.

Scoring an order you have already shipped. The score is only worth computing inside the window where you can still act on it — between order capture and the label. If the score arrives after packing, you have built a report, and you should either move it earlier or stop paying for it.

Treating a high score as proof. A score is a distribution, not a verdict. A meaningful share of your high band would have delivered perfectly. That is expected and it is priced in; it only becomes a problem if you start cancelling.

Where to start

Pull ninety days of COD orders with their final delivery outcome. Add three columns: has this phone ordered before, did it deliver, did it ever come back. Sort by that alone and look at the return rate in each group. For most stores this single split does more work than the rest of the score put together, and it takes an afternoon.

Then add the mechanical address checks and one lane column from your courier reports. Score the ninety days retrospectively, build the band-versus-outcome table, and set two thresholds. Run the three bands by hand for a fortnight with a holdout before automating anything — doing it by hand is what tells you which band your team actually struggles to action, which is almost never the one you expect.

COD King handles the acting-on-it part: it scores and confirms COD orders before they are packed, so the medium band gets its verification attempt and the high band gets an advance or prepaid request inside the window where either still changes the outcome. The signal table and the thresholds are yours, and they are the part worth your own time.