Be Aware of Selection Bias

Be Aware of Selection Bias

I wrote recently about the mistaken perception of older members not being tech savvy. (You can read that here.)

One of my astute readers, Will Slade at Protech, pointed out the following (I'm paraphrasing): "It's possible that the reason many associations perceive older members to be not tech savvy is that the older members will complain while the younger members will just walk away."

This is an important point! Which leads us to selection bias.

Selection bias occurs when we don't have a truly "random" selection of data points under consideration. In the case above, only the older people are complaining and so the assumption is "Our older members are having more trouble." This may or may not be true, but because they're the only ones we're hearing from, we're drawing conclusions based on incomplete data.

I believe associations (and ALL humans) make this mistake quite frequently. Between selection bias and recency bias, too often we're making decisions about design, programs, pricing, and more, based on faulty or incomplete data.

So be careful about how you're using the data you have on hand to make decisions. If the data is incomplete or poorly collected, it may steer you in the wrong direction!

Wes's Wednesday Wisdom Archives

Ask “What” not “Why”

June 3, 2020

Ask “What” not “Why” As a consultant, one of my favorite questions to ask my …

Ask “What” not “Why” Read More »

Engagement is measured by the customer

May 27, 2020

Engagement is Measured by the Customer I’ve written a bunch about measuring engagement in the past. (Click …

Engagement is measured by the customer Read More »

Yes, you do have a process for that

May 13, 2020

Yes, you do have a process for that When I work with clients on pretty …

Yes, you do have a process for that Read More »

“Correct your mistakes before they become your habits”

May 6, 2020

“Correct your mistakes before they become your habits” Yet another great quote from James Clear: “Correct …

“Correct your mistakes before they become your habits” Read More »

Sometimes it’s a process issue

April 29, 2020

Sometimes it’s a process issue When I work with clients on developing a needs list …

Sometimes it’s a process issue Read More »

First ask: “How will I use this data?”

April 22, 2020

First ask: “How will I use this data?” Whether it’s during data conversion, adding a …

First ask: “How will I use this data?” Read More »

“We run the report three times and get three different results.”

April 15, 2020

“We run the report three times and get three different results.” “We run the same …

“We run the report three times and get three different results.” Read More »

How do you know if you’re making progress?

April 8, 2020

How do you know if you’re making progress? We all want to improve our data management, …

How do you know if you’re making progress? Read More »

Don’t move too quickly OR too slowly

April 1, 2020

Don’t move too quickly OR too slowly This may sound like a big “duh!” but I’ll …

Don’t move too quickly OR too slowly Read More »

“Your current habits are perfectly designed to deliver your current results.”

March 25, 2020

“Your current habits are perfectly designed to deliver your current results.” “Your current habits are …

“Your current habits are perfectly designed to deliver your current results.” Read More »

Scroll to Top