Be Aware of Selection Bias

Be Aware of Selection Bias

I wrote recently about the mistaken perception of older members not being tech savvy. (You can read that here.)

One of my astute readers, Will Slade at Protech, pointed out the following (I'm paraphrasing): "It's possible that the reason many associations perceive older members to be not tech savvy is that the older members will complain while the younger members will just walk away."

This is an important point! Which leads us to selection bias.

Selection bias occurs when we don't have a truly "random" selection of data points under consideration. In the case above, only the older people are complaining and so the assumption is "Our older members are having more trouble." This may or may not be true, but because they're the only ones we're hearing from, we're drawing conclusions based on incomplete data.

I believe associations (and ALL humans) make this mistake quite frequently. Between selection bias and recency bias, too often we're making decisions about design, programs, pricing, and more, based on faulty or incomplete data.

So be careful about how you're using the data you have on hand to make decisions. If the data is incomplete or poorly collected, it may steer you in the wrong direction!

Wes's Wednesday Wisdom Archives

Try flowcharting your processes

October 21, 2020

Try flowcharting your processes Working with a client recently on their membership join process reminded me …

Try flowcharting your processes Read More »

“I just want a system I don’t have to fight with.”

October 14, 2020

“I just want a system I don’t have to fight with.” I asked my client: …

“I just want a system I don’t have to fight with.” Read More »

Inertia Contributes to Bad Data

October 7, 2020

Inertia Contributes to Bad Data Without knowing anything about your organization or its data, I’d …

Inertia Contributes to Bad Data Read More »

What are you doing with new contacts?

September 30, 2020

What Are You Doing with New Contacts/ I was very interested to read in a …

What are you doing with new contacts? Read More »

Some Things Just Take Time

September 16, 2020

Some Things Just Take Time I learned recently that an elephant’s gestation period is 18 …

Some Things Just Take Time Read More »

Sometimes It’s the Least Bad Choice

September 7, 2020

Sometimes It’s the Least Bad Choice Just like in life, sometimes when we’re making technology …

Sometimes It’s the Least Bad Choice Read More »

Our Members Aren’t Tech Savvy

September 2, 2020

Our Members Aren’t Tech Savvy Having worked now in the association space for more than …

Our Members Aren’t Tech Savvy Read More »

Motion vs. Action

August 26, 2020

Motion vs. Action One key to successful data management is understanding the difference between motion …

Motion vs. Action Read More »

There is ALWAYS a Trade-off

August 19, 2020

There is ALWAYS a Trade-off I’ve written many times about trade-offs (you can read a …

There is ALWAYS a Trade-off Read More »

Little by little, a little becomes a lot

August 12, 2020

Little by little, a little becomes a lot “Little by little, a little becomes a …

Little by little, a little becomes a lot Read More »

Scroll to Top