“We run the report three times and get three different results.”

"We run the report three times and get three different results."

"We run the same report three times and get different results."

In my 21 years of consulting, I've probably heard this (or something similar) dozens, if not hundreds, of times.

What the user is usually saying is that when they run two different reports or queries that are called the same thing (e.g., "All Members"), the result set is different. The perception is that the data is bad or the software has a bug. But what's really happening is they are running two different reports, with different selection criteria behind it. While both reports are labeled "All Members," they are actually using different criteria, and thus getting different results.

This problem is widespread among associations of all sizes. The challenge is twofold:

  1. We need our queries and reports to be completely transparent. It has to be absolutely clear to the user what data the report is looking at and outputting.
  2. The "catalog" of reports and queries needs to be curated and cleaned up consistently and frequently.

As data managers, we need to be better at both of these things.

A timeless challenge of data management is the perception that the system or the data (or both!) is bad. When staff believes that running the same report three times produces different results, you've got a BIG problem on your hands. You have to work at keeping your queries and reports as clean as the data itself.

Wes's Wednesday Wisdom Archives

Eventually automated emails get ignored

April 14, 2021

Eventually automated emails get ignored During the discovery and development phase of implementing a new …

Eventually automated emails get ignored Read More »

The challenge the AMS providers see

April 7, 2021

The challenge the AMS providers see Over the years I’ve asked AMS providers what they …

The challenge the AMS providers see Read More »

Be careful about “solutioning” too quickly

March 31, 2021

Be careful about “solutioning” too quickly One of the great things about software developers is …

Be careful about “solutioning” too quickly Read More »

What’s our next action?

March 24, 2021

What’s our next action? As you can imagine, I spend a lot of time in …

What’s our next action? Read More »

Don’t automate for the sake of automation

March 17, 2021

Don’t automate for the sake of automation I’m the laziest person in the world. I …

Don’t automate for the sake of automation Read More »

Don’t forget about what got better

March 10, 2021

Don’t forget about what got better Negativity bias is the tendency to focus on only …

Don’t forget about what got better Read More »

Give a little at a time rather than taking away

March 3, 2021

Give a little at a time rather than taking away I’m sure there’s research somewhere …

Give a little at a time rather than taking away Read More »

First, you gotta have the data

February 17, 2021

First, you gotta have the data When I work with clients on a new AMS …

First, you gotta have the data Read More »

Work on your relationship with your AMS vendor

February 10, 2021

Work on your relationship with your AMS vendor It is no coincidence that my most …

Work on your relationship with your AMS vendor Read More »

Dashboards for Data Integrity

February 3, 2021

Dashboards for Data Integrity I’ve written a bunch on data integrity reports. (Click here for …

Dashboards for Data Integrity Read More »

Scroll to Top