How to Dedupe a Lead List Properly (And Why Exact-Match Isn't Enough)

Every lead list has duplicates. If you have scraped more than one search, pulled from more than one source, or simply run the same search twice a month apart, you have the same businesses in your file more than once.

The instinct is to select the column, hit Remove Duplicates, and move on. That catches the easy cases and leaves most of the real ones behind, because genuine duplicates almost never have identical text.

Here is how duplicates actually appear in business data, and what to match on instead.

What a Real Duplicate Looks Like

Exact-match deduplication assumes the same business produces the same string every time. It doesn't.

The same plumber can appear as "Smith & Sons Plumbing", "Smith and Sons Plumbing Ltd", and "Smith & Sons Plumbing - Emergency Plumber Leeds". Three strings, one business, zero exact matches. Meanwhile a chain with fourteen branches gives you fourteen rows with the identical name — and those are not duplicates at all, they are fourteen separate locations you may genuinely want to contact.

So the problem cuts both ways. Match too loosely and you delete real leads. Match too strictly and you email the same person twice.

Match on Identifiers First, Text Last

There is a hierarchy of reliability, and you should work down it rather than starting with the name.

The strongest signal is a stable identifier from the source — a Google place ID, a Companies House number, anything issued by the system the data came from. If two rows carry the same identifier they are the same record, full stop, regardless of what the names say.

Next is the normalised domain. Two businesses sharing a website are almost always one business, provided you have normalised the URL properly first: protocol stripped, www stripped, trailing slash removed, tracking parameters cut. Without that normalisation you are back to exact-match text comparison wearing a different hat.

Phone number is a reasonable third, but only after normalising to a consistent format. 0113 249 6000 and +441132496000 are the same number and no naive comparison will tell you so.

Business name should be your last resort, not your first move.

When You Do Match on Names, Anchor Them to Something

Fuzzy name matching on its own is dangerous. "Bright Dental" and "Brite Dental" might be the same practice with an inconsistent listing, or two unrelated practices in different cities.

The fix is to require a second axis of agreement. Match fuzzy names only when the postcode also matches — ideally the full postcode, at minimum the outward code. That combination is strong: two businesses with near-identical names in the same postcode district are almost certainly one business. The same two names in Leeds and Southampton are two businesses.

This is also why rows with no postcode need special handling. If you cannot anchor a fuzzy name to a location, don't fuzzy match it at all — fall back to exact matching for those rows only. It is better to leave a duplicate in than to delete a lead you can never recover.

Chains, Franchises, and the Branch Problem

Multi-location businesses are the case that breaks most deduplication rules.

Fourteen branches of the same estate agency share a name, often share a domain, and sometimes share a head office phone number. Every heuristic above says "duplicate". Whether they actually are depends entirely on what you're doing: if you're pitching a national marketing contract, one head office contact is what you want. If you're selling something each branch buys locally, all fourteen are legitimate.

The practical answer is to let location break the tie. Same name plus same domain plus different postcode equals different branch, and keep both. Then decide at the campaign level whether to contact one per domain or all of them.

Dedupe Non-Destructively

Whatever rules you settle on, do not delete anything.

Deduplication is a judgement call executed at scale, which means it will occasionally be wrong, and a deleted lead is gone. Mark duplicates instead — flag them, group them under a primary record, or filter them out of the export while leaving them in your database. You keep the ability to review the decision and reverse it.

This is also the right ordering point. Clean your fields first — normalise phones, strip URL junk, trim name suffixes — then dedupe, then export. Deduplicating before normalisation means matching on messy strings, which is how you got here. There is more on that sequence in the guide to exporting a clean lead list to CSV, and the UK phone formatting guide covers the normalisation step in detail.

Doing It Automatically

LeadSnipe dedupes on the way into your lead list and again at export. Saved leads are matched on place ID, website, and phone as they arrive, and the export engine runs a second non-destructive pass — place ID first, then fuzzy name plus postcode, with rows lacking a postcode only ever matched exactly. Nothing is deleted; duplicates are simply left out of the file.

If you're building a list from scratch, the Google Maps scraping guide covers the sourcing step.