How to weight nine factors without over-fitting to your last good deal

The moment a scoring model’s weights are editable, someone will want to edit them the week after a great deal closes. It’s a completely understandable instinct — you just watched a specific kind of lead turn into revenue, so surely the model should value that kind of lead more highly. It’s also, most of the time, the wrong move. One deal is an anecdote. A weighting model is supposed to encode a pattern.

Where the default weights come from

A sensible starting point spreads 100 points across nine factors, roughly in proportion to how much each one predicts fit:

• Role / decision authority — 20
• Industry match — 15
• Services-to-needs alignment — 15
• Company size fit — 10
• Geography match — 10
• Technology relevance — 10
• Outsourcing probability — 10
• AI / digital-transformation fit — 5
• Contact-information completeness — 5

Role gets the largest share because a perfect industry match with the wrong decision-maker still isn’t a workable lead. Contact completeness gets the smallest, because it affects how easily you can act on a lead, not whether the lead is good. That ordering is a starting hypothesis, not a law — but it should take more evidence to move than a single closed-won deal usually provides.

The over-fitting trap, in practice

Over-fitting a weighting model looks reasonable while it’s happening. You close a deal with a mid-size logistics company, notice the win came from a lead with so-so industry match but a great personal relationship, and quietly nudge weights toward whatever that lead had a lot of. Do that after every deal and within a quarter the model isn’t describing your Ideal Customer Profile any more — it’s describing a composite of your last five wins, several of which may have closed despite the model, not because of it.

A model tuned to explain your best deal in hindsight will not find your next one. It will find leads that resemble the exception, not the pattern.

Signs you’re over-fitting rather than genuinely improving the model:

• You can point to the single deal that motivated a weight change, but not a batch of them
• The change moves a factor by more than a couple of points, more than once a quarter
• Nobody has looked at what the change does to leads that didn’t close, only the one that did
• You’re adjusting weights instead of asking whether the ICP definition itself is wrong

Change weights on evidence, not on memory

The honest way to revisit weighting is to look at a batch of outcomes, not a single one — a cohort of leads that closed against a cohort that didn’t, across a full outreach cycle or more, not a single message thread you happen to remember vividly. If technology relevance genuinely correlates with faster deals across dozens of leads, raise it a little. If it doesn’t, leave it, no matter how good the story attached to one specific win sounds in a Monday stand-up.

Because weights and the qualification threshold are editable in one place, rescoring the whole list against a revised model costs nothing beyond a moment’s thought — no developer, no migration. That ease is a feature, but it cuts both ways: nothing stops you from making a change on a hunch just because it’s cheap to do. Treat “cheap to change” and “wise to change” as two separate questions.

The threshold matters as much as the weights

It’s worth remembering that a 70% qualification threshold and a set of nine weights are two different levers, and conflating them muddies the diagnosis. If too few leads are qualifying, the honest first question is whether the threshold is set too high for your actual market, not whether the weights need reshuffling. If leads are qualifying but not converting, that’s a weighting or an ICP-definition problem, not a threshold problem. Adjusting the wrong lever to fix the wrong symptom is how a model ends up with six months of well-intentioned tweaks and no clearer story than when it started.

The discipline that actually pays off

Nine weights summing to 100 is a small, legible model on purpose — small enough that a human can hold the whole thing in their head and sanity-check a change before making it. That’s worth protecting. The model earns its keep by being a stable, examinable description of who you sell to, revised occasionally on real evidence — not a diary of your most recent win, redrawn every few weeks.