Be Aware of Selection Bias

Be Aware of Selection Bias

I wrote recently about the mistaken perception of older members not being tech savvy. (You can read that here.)

One of my astute readers, Will Slade at Protech, pointed out the following (I'm paraphrasing): "It's possible that the reason many associations perceive older members to be not tech savvy is that the older members will complain while the younger members will just walk away."

This is an important point! Which leads us to selection bias.

Selection bias occurs when we don't have a truly "random" selection of data points under consideration. In the case above, only the older people are complaining and so the assumption is "Our older members are having more trouble." This may or may not be true, but because they're the only ones we're hearing from, we're drawing conclusions based on incomplete data.

I believe associations (and ALL humans) make this mistake quite frequently. Between selection bias and recency bias, too often we're making decisions about design, programs, pricing, and more, based on faulty or incomplete data.

So be careful about how you're using the data you have on hand to make decisions. If the data is incomplete or poorly collected, it may steer you in the wrong direction!

Wes's Wednesday Wisdom Archives

Why do we treat data management differently?

August 5, 2020

Why do we treat data management differently? A recent post on ASAE’s community read: “Looking …

Why do we treat data management differently? Read More »

Don’t Forget Your Speakers!

July 29, 2020

Don’t Forget Your Speakers! A phenomenon I’ve noticed over the years is that my clients will …

Don’t Forget Your Speakers! Read More »

90% of your data is never touched a second time

July 22, 2020

90% of your data is never touched a second time I heard recently on a …

90% of your data is never touched a second time Read More »

Newton’s First Law

July 15, 2020

Newton’s First Law Part of Newton’s First Law states that “…an object in motion stays …

Newton’s First Law Read More »

No System is Perfect

July 7, 2020

No System is Perfect While it may sound trite, it bears repeating that no data management …

No System is Perfect Read More »

Data Accretes

July 1, 2020

Data Accretes One of my very first jobs in the association world was managing the production …

Data Accretes Read More »

Don’t be so lazy…

June 24, 2020

Don’t be so lazy… One of the things that attracted me to the software world …

Don’t be so lazy… Read More »

Don’t try to do everything at go-live

June 17, 2020

Don’t Try to do Everything at Go-live When implementing a new AMS, there is a …

Don’t try to do everything at go-live Read More »

Make sure your system has broad support

June 10, 2020

Make sure your system has broad support An AMS vendor pointed out to me recently …

Make sure your system has broad support Read More »

You have to have the right people in place

June 10, 2020

You have to have the right people in place This may seem obvious, but I’ll …

You have to have the right people in place Read More »

Scroll to Top