x

x

Sunday, April 19, 2020

Shared Moment

She was his last customer of the day.

He watched her as she unloaded the
grocery cart. She watched him as he
 scanned her items past the register.


When he handed her the receipt they
both felt a spark as their fingers
brushed.


Their eyes met and they laughed
realizing  that it was a winter day,
with low humidity, and they were
boths standing on carpet.

Friday, February 14, 2020

Evaluating the Risk and Reward of your Statistical Analysis


Most of us out there who perform statistical analyses to guide them and their organizations to solve problems do not have advanced degrees in statistics.  We’ve attended classes at university, we’ve been to varying levels of Six Sigma training, or we’ve done self-study.  

But I think I am safe to say that one thing we all have learned is that statistically evaluating a set of data is complicated and rife with uncertainty. We choose statistical tools to use among many possible tools and numbers ‘pop’ out telling us if our hypothesis is correct or not. From that data, we proceed to either take an action or not take an action depending on the statistical results.

But how many of you finish with your analysis and wonder, what if my analysis is wrong? Did I have enough data?  Did I choose the proper statistical tool? Do I even know the proper statistical tool? Arghh!! (*)

Most of us who are in decision making roles that require analysis of data to determine choices are cautious people and risk averse.  But we had our training.  My ANOVA said that part A is better than part B so why ask more questions?

I suggest that after any statistical analysis and before taking an action based on that analysis, we ask two more questions.

  •      What is my confidence I am right?
  •      What is my risk of being wrong?


And I don’t mean the statistical definitions of ‘risk’ and ‘confidence. I mean just sit back and take a broad overview of your data, where it came from, how you evaluated it and ask yourself how strongly you feel your results are true and ask yourself to decide what the impact is on your customer if your analysis is wrong?

Then you can decide what to do?

But how?

I came up with a simple chart to help guide what action to take. I don’t know whether this is original or not, but here you go anyway.




Let’s look at each box in a little more detail.

1. Confidence of Being Right is HIGH, Risk of Being Wrong is LOW (green quadrant)

You’ve done your analysis. You’ve used multiple tools, did your “Practical / Graphical / Analytical” analysis and you feel very good that you’ve found something significant and the benefits are measurable..

You find that the cost to implement is acceptable and after some thought and study you realize that if you are wrong, the implications to the customer are minimal.  

So, you recommend to Do It.

2. Confidence of Being Right is HIGH, Risk of Being Wrong is HIGH (blue quadrant)

You’ve done your analysis. You’ve used multiple tools, did your “Practical / Graphical / Analytical” analysis and you feel very good that you’ve found something significant and the benefits are measurable.

However, you find that the cost to implement is very high or you find that the effect on the customer if you are wrong is high.

Maybe wait and collect some more data. Even if your are pretty certain about your results,more data might help convince management, your customer (and you).

3. Confidence of Being Right is LOW, Risk of Being Wrong is LOW (tan quadrant)

You’ve done your analyses. You’ve used multiple tools, did your “Practical / Graphical / Analytical” analysis. But you are still not certain if you’ve found something significant and you are not certain that the benefits are measurable. 

However, you find that the cost to implement is acceptable and after some thought and study you realize that even if you are wrong, the implications to the customer are minimal.

So, you can decide make the change. After all the risk of being wrong is low and cost to implement is also low. In parallel you might decide to find someone more experienced than you to check your work and see if they agree.


4. Confidence of Being Right is LOW, Risk of Being Wrong is HIGH (red quadrant)

You’ve done your analyses. You’ve used multiple tools, did your “Practical / Graphical / Analytical” analysis. But you are still not certain if you’ve found something significant. Maybe you’re uncertain if the tools you used apply to this data set. Maybe you are not certain if the data was collected properly. Or maybe you don't know if you have enough data.

You also see that the cost to implement is very high or you find that the effect on the customer if you are wrong is high.

You Don’t Do It.  You might return to this sometime if more data is collected or if something else changes. Or, you might decide to find someone more experienced than you to check your work and see if they have suggestions on how to become more confident


Please comment below if you’re experience is different or if you feel this is way off base.


* I suspect Doctors of Statistical Science also have these 'argghh' moments

Monday, May 13, 2019

Evaluating the Validity of Data Reported in Social Media and the Press


I was going to write a blog posting on this topic, but then I found this excellent article written by The Writing Center of the University of North Carolina.  Yes, maybe I wimped out, but this is really a good summary of how to look at data critically. 
   

However, just to reinforce a couple points.  

Don’t trust data because its quoted on one of your social media ‘friends’ posts. You may trust Bob, and Bob trusts Alex, who has always trusted Sanjay, who trusts Cindy, who trusts Cal, who has an agenda and is distorting the truth.

As the article points out, there are three ways to calculate the center of a data set (Mean, Median, Mode).  Often those with an agenda choose the one that helps to make their point the best.

Finally, I am always suspicious when a data set makes me say ‘Yes! That’s just what I thought. I knew I was right.”  

Am I falling for a biased study, because it matches my beliefs? Question yourself as much as you question others.

Monday, April 15, 2019

Outlier Identification

Its been a while since my last post.  But I can assure you that this post is not an outlier... or is it?

Identifying outliers in a data set is one of the most difficult tasks we face as problem solvers. Mostly because there are no definitive tests which absolutely identify whether a data point is unique or if it is a natural, expected part of the data set. 

Outlier identification reminds us that being a statistical practitioner requires more than a good handle on statistical tools and good knowledge of the process from which the data was collected. Outlier identification requires the ability to use one's mind to take in all this information and make the right decision. Well, at least not make the wrong decision.

The attached is a summary of some methods to look at outliers. It is not a complete compendium of the issue. Please comment below if you have other methods for outlier identification that you have used, or if you feel my presentation needs corrected or adjusted. 

Outlier Indentification

Friday, July 13, 2018

Does country radio play more male singers than female singers?


I admit that I do not listen to much country music but when I do, it always seemed to me that the playlists on country radio are almost entirely men.

But is this a fair claim? Can statistical analysis help us decide?

When I first started thinking about this I realized that it is probably not fair to compare radio playlist counts of men and women singers to the proportion of men and women in the population (50/50). I do know that this sounds odd.

We need to consider that if the proportion women in the business of recording country songs is less than 50%, then we cannot expect the number of records by female artists that are ‘available’ to be played to be 50% of all country records. If fewer women are recording, then fewer women are available to be played on the radio.

To try to find out the real proportion of men to women who are in the business as a country artist I searched on line and found two lists of  country artists which appeared to be independent of radio playtime.

In Wikipedia I found a list of “modern” country artists and I categorized these artists and groups by gender. The other list was from  Country Notes which has attempted to create a complete list of all country artists from the early days through today. This list even contains a few pop stragglers such as Lionel Ritchie and Olivia Newton John who veered into the country lane.

Note: Group acts are counted as male or female depending on the lead singer (so Sugarland is counted as Female and Old Dominion is counted as Male).

From these two lists the ratio of male country artists to female country artists is about the same. The wiki list shows a ratio of 66% men to 34% women and the Country Notes list shows a ratio of 63% men and 37% women.

Given an assumption that more women are in the business today than in the early days, I picked a ratio of 65% men to 35% women as the make-up of all Country artists “available” to be played. One can see that this is much different number than 50% men and 50% women.

Therefore, right or wrong, when we listen to country radio, we should expect to hear about 65% of the songs feature male singers and about 35% of the songs feature female singers.

A side issue of “why” less women are in the country music business is a deeper topic and may actually be the real point of the issue.  However,  this article is not going to address that. I am just going to look at the numbers.

Now that we know what our expected proportion of male singers to female singers is,  we can search the playlists from a sample of  country music radio stations and calculate the percentage of men and women being played on the radio. We would compare this observed proportion to our expected proportion. I also took data from the Billboard Top 50 for the years 2016 and 2017 as well.

But how do we determine if the observed percentage of men versus women on a playlist is significantly higher than expected? If we find that a station is playing 66% male singers and 34% female is that significant? Is 70% men versus 30% women significantly different from our expected 65/35 ratio? Is 80/20 significantly different?

One method that can help us decide is a statistical tool called the Chi-Squared analysis for categorical data. This tool takes data such as Yes/No, Male/Female, Democrat/Republican/Independent and helps us to calculated differences.

This tool is used often in sociology, medicine, and biology studies and evaluates the observed proportions and compares them to the expected proportions. It then lets us know if the observed is significantly different than expected. For instance Chi-Squared analysis has helped to answer questions such as is there a difference in the effectiveness of Drug A between men and women or between Adults and Children.





Of course caveats to the results of this particular study apply.

  • This study looks only at the numbers not at the reasons.
  • Data from only four country radio stations was collected. Also the data from these stations is for one specific day’s playlist.
  • I do not know how Billboard determines its “Top 50”.
  • Chi-Square tests are sensitive to sample size. With small samples sizes it is harder to show a significant deviation between observed and expected.
  • These results do not mean to imply any intentional bias by any organization or radio station. 
  • These results cannot explain ‘why’ more male singers are played than women.

 Example Chi-Square Calculation:




Results:

Of the two samples from a major song ranking service's Top 50 list, the Top 50 for 2017 did not show a difference between observed and expected and the 2016 Top 50 did show a significant difference may exist (more men on the list than women)

Of the five radio stations sampled, 3/4 showed a significant difference may exist (more men played on the radio than women)






So there you go. The answer to the question “Does country radio play more male singers than female singers?” is a resounding “maybe.”

Thursday, August 10, 2017

Capability and MSAs are NOT the same thing!

I have found that people often confuse MSA’s and Capability Studies.  Far too often, I hear the question ‘when will we run the capability study on the tester?’   And while I am sure that you few braves souls who read my blog do not fall into this trap, you might know of people who do. Maybe giving them this link will help.

MSA’s are for tests and gages. Capability studies are for the processes being measured. 

Or to state it another way, MSA's give us confidence that we can measure the capability of our process to produce parts to our customers specification.

One can talk about the 'capability' of a tester, but only when the word is being used in its classic sense, i.e. ' the extent of someone's or something's ability.'

Let's review....

Measurement System Analysis 


A measurement system is a collection of procedures, gages and operators that are used to obtain measurements. Measurement systems analysis (MSA) is used to assess the ability of a measurement system using the following statistical metrics;  stability, repeatability (test / re-test variation) and reproducibility (operator variation). 



The most common metric for an MSA is the Gage R&R value. This value is a ratio of the  variation due to the measurement error (repeatability and reproducibility) to the total variation of the system (including both part and measurement variation).



Gage R&R = Variation due to R and R / (Measurement + Part Variation)







Sometimes, one cannot find parts that demostrate part variation to use in the MSA. An example is in electronics with electrical testing (in-circuit tests or functional testing). These systems make hundreds of measurements and it is impractical to attempt to create or find part variation to use in the MSA

In these cases we usually run the MSA with ten or so parts off the line. In these cases, the part variation will be very low. Therefore the Gage R&R should be calculated as a percent of the tolerance spec range.



Gage R&R = Variation due to R and R / Tolerance Range







Capability Analysis



From Wikipedia…. “The process capability is a measurable property of a process to the specification, expressed as a process capability index (e.g., Cpk, Ppk, Cp, and/or Pp). 

The output of this measurement is usually illustrated by a histogram and calculations that predict how many parts will be produced out of specification



Two parts of process capability are: 1) measure the variability of the output of a process, and 2) compare that variability with a proposed specification or product tolerance.



Cp (or Pp)  = Spec Range / (6 x total system variation (std dev))




Capability studies assume that the measurement variation is low enough to not be a factor, and that the “total variation of the system’ is effectively due to process and part variation.  

Remember that to do a proper capability study we need a successful MSA,  a stable process, and an large enough sample size to be statistically significant (usually about 90 pieces). Getting a good capability study from prototype builds is difficult due to the (usually) small sample size.