You need a PLAN to deal with duplicates
Here's the thing about duplicate records: No matter what you do, you'll never get rid of them completely, because humans are human. Whether it's a staff member creating a duplicate record because they forgot to check if the record exists, or it's a customer who doesn't want to take the time to reset his password and just creates another record with a different email address, duplicates are going to happen.
And because duplicates are inevitable, you have to have a plan for dealing with them. Here are some suggestions:
- Create data integrity reports that will help you identify potential duplicate records, and run those reports consistently. ("Consistently" means at least once a month, if not more often.) And make sure someone (or several someones) on staff has responsibility for cleaning up this data.
- Make it everyone's job on staff to identify potential duplicates whenever they are using the database and make sure you have a clear process for how those duplicate records are reported. And make sure someone (or several someones) on staff has responsibility for cleaning up this data.
- And did I mention to make sure someone (or several someones) on staff has responsibility for cleaning up this data?
Managing duplicate records is a journey, not a destination. No matter how good your technology is, like weeds in a garden, duplicate records are going to appear. So you have to have a plan for dealing with them, not just now, but always.
![]()
Wes's Wednesday Wisdom Archives
Is that meaningless data?
Is that meaningless data? I’m not a big quotes guy, but one of the few […]
Be aware of unintended consequences
Be aware of unintended consequences I’ve written before that every decision involves a trade-off. When […]
Positive change is harder to see
Positive change is harder to see Humans are wired to see negative change because we […]
MVP: Minimum Viable Product
MVP: Minimum Viable Product In product development there is a concept known as MVP, or […]
You always need a reason for collecting data
You always need a reason for collecting data When you ask for data from someone […]
If you’re unhappy, speak up!
If you’re unhappy, speak up! My clients will often ask me something along the lines […]
Does it advance the mission?
Does it advance the mission? Because associations are mission-driven, everything you do should be seen […]
How should you start a new data project?
How should you start a new data project? When you’ve got a new data project […]
A Data Integrity Report…for Reports!
I’ve written elsewhere about the value of data integrity reports. But one of the most […]
Simple rules for complex systems
Simple rules for complex systems I first heard the phrase “simple rules for complex systems” […]
