You need a PLAN to deal with duplicates
Here's the thing about duplicate records: No matter what you do, you'll never get rid of them completely, because humans are human. Whether it's a staff member creating a duplicate record because they forgot to check if the record exists, or it's a customer who doesn't want to take the time to reset his password and just creates another record with a different email address, duplicates are going to happen.
And because duplicates are inevitable, you have to have a plan for dealing with them. Here are some suggestions:
- Create data integrity reports that will help you identify potential duplicate records, and run those reports consistently. ("Consistently" means at least once a month, if not more often.) And make sure someone (or several someones) on staff has responsibility for cleaning up this data.
- Make it everyone's job on staff to identify potential duplicates whenever they are using the database and make sure you have a clear process for how those duplicate records are reported. And make sure someone (or several someones) on staff has responsibility for cleaning up this data.
- And did I mention to make sure someone (or several someones) on staff has responsibility for cleaning up this data?
Managing duplicate records is a journey, not a destination. No matter how good your technology is, like weeds in a garden, duplicate records are going to appear. So you have to have a plan for dealing with them, not just now, but always.
![]()
Wes's Wednesday Wisdom Archives
Longfellow and data management
Longfellow and data management “We judge ourselves by what we feel capable of doing while […]
Do you really need all that historical data?
Do you really need all that historical data? A question I’ll often get from my […]
AI actually requires thinking
AI actually requires thinking “I don’t think AI introduces a new kind of thinking. It […]
It’s not the mistakes, but how you respond
It’s not the mistakes, but how you respond Recently a client was complaining about a bug that […]
The hidden costs of bad data
The hidden costs of bad data Nobody likes bad data, and presumably we’re all working […]
Don’t let your customers edit their names online!
Don’t let your customers edit their names online! This issue came up recently and I […]
Once is an accident, twice is coincidence, three times is a pattern.
Once is an accident, twice is coincidence, three times is a pattern. We’ve probably all […]
“Every association does this.”
“Every association does this.” One of the most significant values I bring to my clients […]
Trust your gut
Trust your gut When I help associations with selection of a new technology system (e.g., […]
“People more frequently require to be reminded than informed.”
“People more frequently require to be reminded than informed.” “People more frequently require to be […]
