← All posts

Writing

Testing System One Models for Smarter CRM Lead Routing

It started with this question:

Can CRM lead routing handle edge cases better without turning into an increasingly complicated tree of rules?

Most CRMs can already route leads using things like geography, company size, language, source, or round-robin assignment.

For straightforward leads, that works well.

The problem appears when the lead does not fit neatly into one field.

A 40-person company might normally be treated as SMB, but what if the deal involves SSO, SCIM, legal review, security requirements, and more than a thousand external users?

A static rule sees:

Employee count: 42
    ↓
SMB
    ↓
SMB Account Executive

But the actual buying motion may look much closer to enterprise.

That is the kind of edge case I wanted to explore.

The problem wasn't really routing

At first, it would be easy to frame this as:

Can AI route leads better than CRM rules?

But that is too broad.

Static routing is actually very good when the decision can be expressed clearly.

For example:

Germany
    ↓
DACH Account Executive

or:

North America
+ 700 employees
    ↓
Enterprise Account Executive

There is no reason to replace deterministic logic when deterministic logic already describes the decision well.

The interesting question is what happens when several signals conflict.

Small company
+ enterprise security requirements
+ complex procurement
+ large external deployment

Which signal should matter most?

That is where the routing problem starts becoming less about matching fields and more about interpreting the situation.

Exploring System One models

That led me to experiment with lightweight decision models, specifically Laya and Jev.

I wanted the smallest possible proof of concept:

Lead
    ↓
Same routing criteria
    ↓
┌──────────────┬──────────────┬──────────────┐
│ Static Rules │     Laya     │     Jev      │
└──────────────┴──────────────┴──────────────┘
    ↓
Compare against expected route

The models receive the same lead information and the same descriptions of the available sales representatives.

For example, instead of writing dozens of conditions for an enterprise AE, the routing criteria might say:

Best for North American enterprise or unusually complex deals involving Salesforce migration, SSO/SCIM, security review, procurement, regulated buyers, or large multi-team rollouts.

The decision model then has to determine which representative best fits the lead.

If no representative is clearly appropriate, it can send the lead to manual_review.

Building a static baseline first

Before testing either model, I created a simple CRM-style routing tree.

Something like:

DACH → Anna

France → Sophie

APAC → Priya

Latin America → Luis

North America:
    500+ employees → Maya
    75–499 → Luis
    Under 75 → Jordan

This gave the experiment a baseline.

Then I created twelve realistic leads.

Some were intentionally easy.

Others were designed around the situations where static rules usually become awkward.

One example was a 42-person SaaS company.

On company size alone, it looks like a straightforward SMB account.

But the lead also needed:

  • SAML SSO
  • SCIM
  • a security review
  • a negotiated DPA
  • roughly 1,200 contractor accounts

The expected route was the enterprise AE.

The static system still routed it to SMB.

That failure was useful.

It showed exactly what I wanted the PoC to test: whether a decision model could recognize the buying motion behind the fields rather than simply following the most obvious field.

Then the results became interesting

I ran the same twelve leads repeatedly across all three approaches.

After 20 runs, the result was:

Jev
239 / 240
99.58%
Average: 11.95 / 12 per run

Laya
132 / 240
55.00%
Average: 6.60 / 12 per run

Static Rules
120 / 240
50.00%
Average: 6.00 / 12 per run

The static baseline behaved exactly as expected.

It consistently handled the simple cases and consistently missed the edge cases.

Laya was much less stable. Some runs were noticeably better than others, and changing the evaluation conditions exposed how sensitive its decisions could be.

Jev was different.

Across the 240 routing decisions, it missed only one.

That does not mean the problem is solved.

The benchmark is still tiny, and the same twelve leads were reused across multiple runs.

The next meaningful test would be a much larger unseen dataset with more ambiguous and adversarial routing situations.

But as a proof of concept, the signal was strong enough to keep exploring.

What I actually learned

The interesting part was not that a model produced a higher score.

It was that the architecture of the routing system could potentially change.

Instead of encoding every exception:

IF employee_count < 75
AND security_review = true
AND SSO = true
AND SCIM = true
AND external_users > 1000
THEN enterprise

the system could keep deterministic rules for obvious cases and introduce semantic decision-making where the context becomes messy.

Something closer to:

Simple decision
    ↓
Static rule

Context-heavy decision
    ↓
Semantic router

Low confidence / no clear fit
    ↓
Manual review

That feels much more practical than trying to replace the entire routing system with AI.

Why I explored this in the first place

There was also a very practical reason behind the experiment.

I was looking at a potential client problem and wanted to understand whether I could contribute something beyond implementing another workflow inside an existing CRM.

The goal was not to show up saying:

"We should add AI."

It was to be able to say:

"Here is a specific failure mode in the current routing approach. I built a small experiment around it, compared three approaches, and here is what happened."

Even if the final solution ends up using mostly traditional rules, the experiment helps clarify where those rules are sufficient and where they start becoming expensive to maintain.

The next step

The current PoC is intentionally small.

The next version should answer a harder question:

Does semantic routing continue to outperform static rules when the model sees a much larger set of completely unseen leads?

That means expanding the dataset, introducing more conflicting signals, measuring consistency across option ordering, and looking closely at the cases where the models disagree.

For now, the main takeaway is simpler:

CRM lead routing is easy when the lead fits the rules. The interesting problem is what happens when the lead makes sense to a human but not to the routing tree.

That is where System One-style decision models may be useful.

Github repo for this PoC: https://github.com/jclumingkit/jev-poc