If you were among the 300 people who attended the first-ever Minitab Insights conference in September, you already know how powerful it was. Attendees learned how practitioners from a wide range of industries use data analysis to address a variety of problems, find solutions, and improve business practices.

In the coming weeks and months, we will share more of the great insights and guidance shared by our speakers and attendees. But here are five helpful, challenging, and thought-provoking ideas and suggestions that we heard during the event.

Joel Smith of the Dr. Pepper Snapple Group used the assessment of different beers to show how applying the tools in Minitab can help a business move from raw Voice of the Customer (VOC) data to actionable insights. His presentation showed how to use graphical analysis and descriptive statistics to clean observational VOC data, and then how to use cluster analysis, principal component analysis, and regression analysis to make informed decisions about how to create a better product.  

Graphs are often part of a Minitab analysis, but a graph may not be the only way to visualize your results. Think about your audience and your communication goals when choosing and customizing your graphs, suggested Rip Stauffer, senior consultant at Management Science and Innovation. He showed examples of how the same information comes across very differently when presented in various charts, and when colors, thicknesses, and styles are selected carefully. Along the way, he also illustrated Minitab's flexibility in tailoring the appearance of a graph to fit your needs. 

We hear all the time about the impact of quality improvement methods on manufacturing. But what about using statistical analysis to boost sales? Andrew Mohler from global chemical company Buckman explained how training technical sales associates to use data analysis and Minitab has transformed the company's business. Empowering the sales team to help customers improve their processes has enabled the company to provide more value and to drive sales—boosting the bottom line.

In the quality improvement world, we tend to think that transforming an organization's culture so everyone understands the value of data analysis only brings benefits. But Richard Titus, a consultant and adjunct instructor at Lehigh University who has worked with Crayola, Ingersoll-Rand, and many other organizations, highlighted potential traps for organizations with a high level of statistical knowledge. These include trying to find data to fit favored answer(s); working as a "lone ranger" independent of a team; failing to map and measure processes; not selecting a primary metric to measure success; searching for a "silver bullet;" and trying to outsmart the process. 

T. C. Simpson and M. E. Rusak from Air Products illustrated how using a traditional P chart to monitor a transactional process can lead to problems if you have a large subgroup size. False alarms or failure to detect special-cause variation can result from overdispersion or underdispersion in your data when your subgroup sizes are large. You can avoid these risks with a Laney P' control chart, which uses calculations that account for large subgroups. Learn more about the Laney P' chart. 

Watch for more stories, tips, and ideas from the the Minitab Insights conference in future issues of Minitab News, and on the Minitab Blog.

I watched an old motorcycle flick from the 1960s the other night, and I was struck by the bikers' slang. They had a language all their own. Just like statisticians, whose manner of speaking often confounds those who aren't hep to the lingo of data analysis.

It got me thinking...what if there were an all-statistician biker gang? Call them the Nulls Angels. Imagine them in their colors, tearing across the countryside, analyzing data and asking the people they encounter on the road about whether they "fail to reject the null hypothesis."

If you point out how strange that phrase sounds, the Nulls Angels will know you're not cool...and not very aware of statistics.

Speaking purely as an editor, I acknowledge that "failing to reject the null hypothesis" is cringe-worthy. "Failing to reject" seems like an overly complicated equivalent to accept. At minimum, it's clunky phrasing.

But it turns out those rough-and-ready statisticians in the Nulls Angels have good reason to talk like that. From a statistical perspective, it's undeniably accurate—and replacing "failure to reject" with "accept" would just be wrong.

Hypothesis tests include one- and two-sample t-tests, tests for association, tests for normality, and many more. (All of these tests are available under the Stat menu in Minitab statistical software. Or, if you want a little more statistical guidance, the Assistant can lead you through common hypothesis tests step-by-step.)

A hypothesis test examines two propositions: the null hypothesis (or H0 for short), and the alternative (H1). The alternative hypothesis is what we hope to support. We presume that the null hypothesis is true, unless the data provide sufficient evidence that it is not.

You've heard the phrase "Innocent until proven guilty." That means innocence is assumed until guilt is proven. In statistics, the null hypothesis is taken for granted until the alternative is proven true.

That brings up the issue of "proof."

The degree of statistical evidence we need in order to “prove” the alternative hypothesis is the confidence level. The confidence level is 1 minus our risk of committing a Type I error, which occurs when you incorrectly reject the null hypothesis when it's true. Statisticians call this risk alpha, and also refer to it as the significance level. The typical alpha of 0.05 corresponds to a 95% confidence level: we're accepting a 5% chance of rejecting the null even if it is true. (In life-or-death matters, we might lower the risk of a Type I error to 1% or less.)

Regardless of the alpha level we choose, any hypothesis test has only two possible outcomes:

1. Reject the null hypothesis and conclude that the alternative hypothesis is true at the 95% confidence level (or whatever level you've selected).
    
2. Fail to reject the null hypothesis and conclude that not enough evidence is available to suggest the null is false at the 95% confidence level.

We often use a p-value to decide if the data support the null hypothesis or not. If the test's p-value is less than our selected alpha level, we reject the null. Or, as statisticians say "When the p-value's low, the null must go."

This still doesn't explain why a statistician won't "accept the null hypothesis." Here's the bottom line: failing to reject the null hypothesis does not mean the null hypothesis is true. That's because a hypothesis test does not determine which hypothesis is true, or even which is most likely: it only assesses whether evidence exists to reject the null hypothesis.

Hark back to "innocent until proven guilty." As the data analyst, you are the judge. The hypothesis test is the trial, and the null hypothesis is the defendant. The alternative hypothesis is the prosecution, which needs to make its case beyond a reasonable doubt (say, with 95% certainty).

If the trial evidence does not show the defendant is guilty, neither has it proved that the defendant is innocent. However, based on the available evidence, you can't reject that possibility. So how would you announce your verdict?

"Not guilty."

That phrase is perfect: "Not guilty"doesn't say the defendant is innocent, because that has not been proven. It just says the prosecution couldn't convince the judge to abandon the assumption of innocence.

So "failure to reject the null" is the statistical equivalent of "not guilty." In a trial, the burden of proof falls to the prosecution. When analyzing data, the entire burden of proof falls to your sample data. "Not guilty" does not mean "innocent," and "failing to reject" the null hypothesis is quite distinct from "accepting" it. 

So if a group of marauding statisticians in their Nulls Angels leathers ever asks, keep yourself in their good graces, and show that know "failing to reject the null" is not "accepting the null."

We hosted our first-ever Minitab Insights conference in September, and if you were among the attendees, you already know the caliber of the speakers and the value of the information they shared. Experts from a wide range of industries offered a lot of great lessons about how they use data analysis to improve business practices and solve a variety of problems.

I blogged earlier about five key takeaways gleaned from the sessions at the Minitab Insights 2016 conference. But that was just the tip of the iceberg, and participants learned many more helpful things are well worth sharing. So here are five more helpful, challenging, and thought-provoking ideas and suggestions that we heard during the event.

Everyone has personal goals they'd like to achieve, such as getting fit, changing a habit, or writing a book. Rod Toro, deployment leader at Edward Jones, explained how challenging himself and his team to apply Lean and Six Sigma tools to their personal goals has helped them better understand the underlying principles of quality improvement, personalized learning and gain deeper insights, and expanded their ability to apply quality methods in a variety of circumstances and situations. 

Minitab technical training specialist Scott Kowalski reminded us that when we test a hypothesis with statistics, "failing to reject the null" does not prove that the null hypothesis is true. It only means we don't have enough evidence to reject it. We need to keep this in mind when we interpret our results, and to be careful how we explain our findings to others. We also need to be sure our hypotheses are clearly stated, and that we've selected the appropriate test for our task!

We've all seen them in our data: those troublesome observations that just don't want to belong, lurking off in the margins, maybe with one or two other loners. It can be tempting to ignore or just delete those observations, but Larry Bartkus, senior distinguished engineer at Edwards Lifesciences, provided vivid illustrations of the drastic impact outliers can have on the results of an analysis. He also reminded us of the value in slowing down our assumptions, looking at the data in several ways, and trying to understand why our data is the way it is. 

When we need to assess how well an attribute measurement system performs, attribute agreement analysis is the go-to method—but Thomas Rust, reliability engineer at Autoliv, demonstrated that many more options are available. In encouraging quality practitioners to "break the attribute paradigm," Rust detailed four innovative ways to assess an attribute measurement system: measure an underlying variable; attribute measurement of a variable product; variable measurement of an attribute product; and attribute measurement of an attribute product.

More than anything else, what we took away from Minitab Insights 2016 was an even greater appreciation for the people who are using our software in innovative ways—to increase the quality of the products we use every day, to raise the level of service we receive from businesses and organizations, to increase the efficiency and safety of our healthcare providers, and so much more.

Watch for more stories and ideas from the the Minitab Insights conference in future issues of Minitab News, and on the Minitab Blog.

A reader asked a great question in response to a post I wrote about Pareto charts. Our readers typically do ask great questions, but this one turned out to be more difficult to answer than it first seemed.

My correspondent wrote: 

My understanding is that when you have count data, a bar chart is the way to go. The gaps between the bars emphasize that the data are not measured on a continuous scale. The Pareto chart puts the bars in decreasing size going from left to right. However, the bars now touch, even though the data scale has not changed. I'm just looking for some history or explanation as to why the bars in the Pareto chart touch, which seems to violate basic rules of effective graphing.

In case you're not familiar with all of this, here's a quick, mostly visual recap. A bar chart displays counts of categorical variables. Separating the bars emphasizes the data's categorical nature: 

Assume that for some measurable aspect of our business, we classify measurements from 1-10 as Critical, from 11-20 as Very Important, from 21-30 as Important, and so on. A histogram of integer data corresponding to the counts in the bar chart above looks like this:

The bars of the histogram touch because they represent continuous data. It makes sense that the bars abut each other, since there's no categorical "gap" between, say, 1 and 2.

Which brings us to the Pareto chart, whose bins show counts or frequencies of defects—categorical data. And yet, when you produce a Pareto chart in Minitab and most other packages, the bars touch...

The question had me scratching my head. I checked Minitab's built-in statistical glossary and help files, our web site, and then expanded my search for to some reliable statistics resources on the Web. No answer. So I went to my colleague-next-door's office and asked her why the bars on the Pareto chart touch, even though they represent counts or frequencies of categories. 

She didn't know, either, but she had a good idea who would know: Dr. Terry Ziemer, who as a senior statistician at Minitab during the 1990s directed development work in the area of industrial statistics. He later became a principal at the Six Sigma Academy and then founded Six Sigma Intelligence. I e-mailed Terry to find out why the bars on Minitab's Pareto chart touch. He quickly replied: 

I wish there was some big technical answer I could give you, but it was simply a design choice. At the time when I did program this, most of the example Pareto charts I looked at had the bars touching, and I agreed that (at least in my opinion) the chart looks better that way than it does when there are gaps between the bars. Since a Pareto chart is sort of a distribution graph for defect types, it did seem to make sense to make it more like a histogram, another distribution graph where the bars touch, than to make it look like a standard bar chart where you have the gaps.

So that's why the bars on a Pareto chart in Minitab touch: it was an aesthetic choice, and one that makes perfect sense if we see the Pareto chart as similar to a histogram, in that it shows you the distribution of defect types.  

If you're saying "But that's not a definitive answer," you're right. Unfortunately, there doesn't seem to be a definitive answer. Looking through the literature revealed advocates both for and against having the Pareto bars touch, but not much in the way of detailed rationales.

For example, The Practitioner's Guide to Statistics and Lean Six Sigma for Process Improvement by Mikel Harry et al. states on page 171, "The bars in a Pareto chart are arranged side-by-side (touching) in descending order from the left." Why the bars should touch, however, is left unexplained. Joiner Associates' Pareto Charts: Plain & Simple suggests "Having the bars touch makes it easier to judge the relative size or impact of the different parts of the problem." A design choice, again. 

On the other hand, page 112 of Statistical Reasoning for Everyday Life by Jeffrey Bennett et al., states "To make the Pareto chart, we put the bars in descending order of size...Because the categories are nominal, the bars should not touch." Clearly, there's still some debate among academics, and if you prefer your Pareto charts with space between bins, you'll find some support—but the touching bars, as implemented in Minitab, do appear to be the more popular option. 

Every day, thousands of people withdraw extra cash for daily expenses. Each transaction may be small, but the total amount of cash dispersed over hundreds or thousands of daily transactions can be very high. But every bank branch has a fixed cash flow, which must be set without knowing what each customer will need on a given day. This creates a challenge for financial entities. Customers expect their local bank office to have adequate cash on hand, so how can a bank confidently ensure each branch has enough funds to handle transactions without keeping too much in reserve?

A quality project team led by Jean Carlos Zamora and Francisco Aguilar tackled that problem at Grupo Mutual, a financial entity in Costa Rica.

When the project began, each of Grupo Mutual's 55 branches kept additional cash in a vault to avoid having insufficient funds. But without a clear understanding of daily needs, some branches often ran out of cash anyway, while others had significant unused reserves.

When a branch ran short, it created high costs for the company and gave customers three undesirable options: receive the funds as an electronic transfer, wait 1–3 days for consignment, or travel to the main branch to withdraw their cash. Having the right amount of cash in each branch vault would reduce costs and maintain customer satisfaction.

Using Minitab Statistical Software and Lean Six Sigma methods, the team set out to determine the optimal amount of currency to store at each branch to avoid both a negative cash flow and idle funds. The team followed the five-phase DMAIC (Define, Measure, Analyze, Improve, and Control) method. In the Define phase, they set the goal: creating an efficient process that transferred cash from idle vaults to branches that needed it most.

In the Measure phase, the team analyzed two years' worth of cash-flow data from the 55 branches. “Managing the databases and analyzing about 2,000 data points from each of the 55 branches was our biggest challenge,” says Jean-Carlos Zamora Mora, project leader and improvement specialist at Grupo Mutual. “Minitab played a very important part in addressing this issue. It reduced the analysis time by helping us identify where to focus our efforts to improve our process.” 

The Analyze phase began with an analysis of variance (ANOVA) for to explore how the banks’ cash flow varied per month. They used Minitab to identify which months were different from one another, and grouped similar months together to streamline the analysis. 

The team next used control charts to graph the data over time and assess whether or not the process was stable, in preparation for conducting capability analysis. To choose the right control chart and create comprehensive summaries of the results, the team used the Minitab Assistant.

The team then performed a capability analysis of each group’s current cash flow to determine whether customer transactions matched the services provided, and establish the percentage of cash used at each branch.

The analysis revealed that, in total, the vaults contained more than the necessary funds each branch needed to operate effectively, but excessive circulation of the money caused some to overdraw their vaults while others stored cash that was not utilized. 

“We found a positive cash balance at 95% of the branches,” says Zamora Mora. “The analysis showed the cash on hand to meet customer needs exceeded the requirements by over 200%, so we suddenly had lots of money to invest.” 

The analysis gave the team the confidence to move forward with the Improve phase: implementing real-time control charts that enabled management to check each branch’s cash balance throughout the day. Managers could now quickly move cash from branches with excess cash to those needing additional funds, and make more strategic cash flow decisions.

The team found that being able to answer objections with data helped secure buy-in from skeptical stakeholders. “Throughout this project, we encountered questions and situations that could have jeopardized our team’s credibility and our likelihood of success,” recalls Zamora Mora. “But the accuracy and reliability of our data analysis with Minitab was overpowering.” 

The changes made during the project increased cash usage by 40% and slashed remittance costs by 60%.The new process also cut insurance costs and shrank risks associated with storing and transporting cash. Overall, the project increased revenue by $1.1 million. 

To read a more detailed account of this project, click here. 

With another Halloween almost upon us, here's a look back at some of the posts we've written about this holiday specifically, and about various creepy things in general. I hope that you enjoy this roundup of 13 scary statistics posts...and that they won't keep you up at night!

As Halloween nears, you can customize your Minitab interface to match the season. Here's how...

I’m here to warn you about a ghoul that all statisticians and data scientists need to be aware of: phantom degrees of freedom...

Chi-square analysis has many applications in business and quality improvement. For example, let's say you manage three call centers and you want to know if solving a customer's problem depends on which center gets the call. To find out, you tally the...

What do zombies have to do with the 2-Sample Poisson Rate Test? Let’s say that your group of survivors is camping in an area that will soon be overrun by the undead zombie menace...

Data collection isn't always pretty. My lab worked with human cadavers. Usually not their entire body, but specifically their feet...

The 1982 film The Thing helped me get a handle on the Weibull distribution. In the film, Antarctic researchers encounter an alien with a truly frightening ability... 

On Halloween, I usually dress up, pass out candy, and watch "It’s the Great Pumpkin, Charlie Brown" on TV. But I’m trying to broaden my horizons to include YouTube. I want to use data...

That pattern in your time series plot...are you sure it's not a ghost? The trend that seems so evident might well be a phantom... 

It’s almost Halloween, so there’s lots to do. Of course, you have to plan your daily candy consumption to match the limits on free sugar recommended by the World Health Organization...

In “The Fall of the House of Usher,” the punctuation changes the rhythms of the story to create the intensity of the climax. Let’s look at these punctuation marks with Laney P’ Charts...

The National Retail Foundation (NRF) released the results of their Halloween Consumer Spending Survey last month. With Minitab, we can dig a little deeper into the data...

I’ve gone ghost hunting a half-dozen times over the past 3 years. Now, I’m a skeptic and not a paranormal enthusiast. But being skeptical about something does not preclude collecting data about it...

A dead salmon was placed in a functional magnetic resonance imaging (fMRI) machine and shown emotionally charged photographs. Analysis of that fMRI data showed evidence of brain activity...in the dead, frozen salmon. Uh-oh, I thought. If dead salmon can think, could they also become reanimated...?

Happy Halloween! 

If your work involves quality improvement, you've at least heard of Design of Experiments (DOE). You probably know it's the most efficient way to optimize and improve your process. But many of us find DOE intimidating, especially if it's not a tool we use often. How do you select an appropriate design, and ensure you've got the right number of factors and levels? And after you've gathered your data, how do you pick the right model for your analysis?

One way to get started with DOE is the Assistant in Minitab Statistical Software. When you have many factors to evaluate, the Assistant will walk you through a DOE to identify which factors matter the most (screening designs). Then the Assistant can guide you through a designed experiment to fine-tune the important factors for maximum impact (optimization designs). 

If you're comfortable enough to skip the Assistant, but still have some questions about whether you're approaching your DOE the right way, consider the following tips from Minitab's technical trainers. These veterans have done a host of designed experiments, both while working with Minitab customers and in their careers in before they became Minitab trainers. 

Performing exploratory runs before doing the main experiment can help you identify the settings of your process as performance moves from good to bad. This can help you determine the variable space to conduct your experiment that will yield the most beneficial results.                                             

In many cases, it's beneficial to choose a design with ½ or ¼ of the runs of a full factorial. Even though effects could be confounded or confused with each other, Resolution V designs minimize the impact of this confounding which allows you to estimate all main effects and two-way interactions. Conducting fewer runs can save money and keep experiment costs low.

Power is the probability of detecting an effect on the response, if that effect exists. The number of replicates affects your experiment's power. To increase the chance that you will be successful identifying the inputs that affect your response, add replicates to your experiment to increase its power.

Reducing defects is the primary goal of most experiments, so it makes sense that defect counts are often used as a response. But defect counts are a very expensive and unresponsive output to measure. Instead, try measuring a quantitative indicator related to your defect level. Doing this can decrease your sample size dramatically and improve the power of your experiment.  

Factorial designs let you take a comprehensive approach to studying all potential input variables. Removing a factor from the experiment slashes your chance of determining its importance to zero. With the tools available in statistical software such as Minitab to help, you shouldn't let fear of complexity cause you to omit potentially important input variables. 

Do you have any DOE tips to add to this list?

Pareto charts are a special type of bar chart you can use to prioritize almost anything. This makes them very useful in making sound decisions. For example, if you have several possible quality improvement projects, but not enough time or people to do them all now, you can use a Pareto chart to identify which projects have the most potential for making meaningful improvement.

Pareto charts look somewhat similar to regular bar charts. In their simplest form, you collect counts (for different categories of defects, for example). Then each bar is ordered according to size or frequency, so you can determine which categories comprise the "vital few" that you should care about, and which are the "trivial many" and therefore less worthy of your attention.

In the example below, taken from a Six Sigma healthcare project, you can see why Pareto charts are great for seeing where the largest gains might be made as you focus your improvement efforts.

Pareto charts are easy to understand and use. But, like any statistical tool, there are some things you need to keep in mind when you create and interpret the Pareto chart. Here are the top concerns to watch for so that you can get the most benefit from these simple but powerful tools. 

• If you only collect data from a brief period of time, you may reach incorrect conclusions. This is particularly true if your process is unstable. When the process is not in control, the causes may be unstable and the vital few problems may change from week to week. So collecting data for a single day may truly reflect your whole process. If your data are not reliable, or aren't truly representative of your population, your Pareto chart will give you a distorted picture of the distribution of defects and causes.
   
• On the other hand, you don't want to collect data over too long a time period, either. Data collected during long periods of time may include changes that affect counts or frequencies, but shouldn't be included as causes. Check the data for stratification, or changes in the distribution of frequencies or counts over time.
   
• Select the categories you will measure carefully. If your initial Pareto analysis does not yield useful results, make sure that your categories are meaningful and that your "other" category is not too large.
   
• Weighted Pareto charts can be particularly useful in many situations. But weighting criteria need to be selected with care. For example, if the expense of certain defects are higher than others, cost may be a more meaningful basis for prioritization than number of occurrences.
   
• Be clear about your ultimate goal, and choose your focus appropriately. Focusing on the problems that happen most often should cut the amount of rework that needs to happen. But focusing on problems with the highest cost should maximize an improvement project's financial benefits.
   
• Use common sense. The purpose of a conducting a Pareto analysis is to identify where you might get the most "bang for your buck" in quality improvement, but you shouldn't ignore small, easily solved problems until all larger problems are solved. 

Do you have any tips or suggestions for using Pareto charts effectively? Please share them in the comments! 

Minitab's LinkedIn group is a good place to ask questions and get input from people with experience analyzing data and doing statistics in a wide array of professions. For example, one member asked this question:

I am trying to create a chart that can monitor change by month. I have [last year's] data and want to compare it to [this year's] data...what chart should I use, and can I auto-update it? Thank you. 

As usual when a question is asked, the Minitab user community responded with some great information and helpful suggestions. Participants frequently go above and beyond, answering not just the question being asked, but raising issues that the question implies.  For instance, one of our regular commenters responded thus: 

There are two ways to answer this inquiry...by showing you a solution to the specific question you asked or by applying statistical thinking arguments such as described by Donald Wheeler et al and applying a solution that gives the most instructive interpretation to the data.

In this and subsequent posts, I'd like to take a closer look at the various suggestions group members made, because each has merits. First up: a simple individuals chart of differences, with some cool tricks for instant updating as new data becomes available. 

An easy way to monitor change month-by-month is to use an individuals chart. Here's how to do it in Minitab Statistical Software, and if you'd like to play along, here's the data set I'm using. If you don't already have Minitab, download the free 30-day trial version.

I need four columns in the data sheet: month name, this year's data, last year's data, and one for the difference between this year and last. I'm going to right-click on the Diff column, and then select Formulas > Assign Formula to Column..., which gives me the dialog box below. I'll complete it with a simple subtraction formula, but depending on your situation a different formula might be called for:

With this formula assigned, as I enter the data for this year and last year, the difference between them will be calculated on the fly. 

Now I can create an Individuals Chart, or I Chart, of the differences. I choose Stat > Control Charts > Variables Charts for Individuals > Individuals... and simply choose the Diff column as my variable. Minitab creates the following graph of the differences between last year's data and this year's data: 

Now, you'll notice that when I started, I only had this year's data through September. What happens when I need to update it for the whole year?  Easy - I can return to the data sheet in January to add in the data from the last quarter. As I do, my Diff column uses its assigned formula (indicated by the little green cross in the column header) to calculate the differences: 

Now if I look at the I-chart I created earlier, I see a big yellow dot in the top-left corner.

When I right-click on that yellow dot and choose "Automatic Updates," as shown in the image above, Minitab automatically updates my Individuals chart with the information from the final three months of the year: 

Whoa!  It looks like we might have some special-cause variation happening in that last month of the year...but at least I can use the time I've saved by automatically updating this chart to start investigating that! 

In my next post, we'll try another way to look at monthly differences, again following the suggestions offered by the good people on Minitab's LinkedIn group. 

A member of Minitab's LinkedIn group asked how to create a chart to monitor change by month, specifically comparing last year's data to this year's data. My last post showed how to do this using an Individuals Chart of the differences between this year's and last year's data.  Here's another approach suggested by a participant in the group. 

An individuals chart of the differences between this year's data and last year's might not be our best approach. Another approach is to look at all of the data together.  We'll put this year's and last year's data into a single column and see how it looks in an individuals chart. (Want to play along? Here's my data set.)

We'll choose Stat > Control Charts > Variables Charts for Individuals > Individuals... and choose the "2 years" column in my datasheet as the variable. Minitab creates the following I chart: 

Now we can examine all of the data sequentially and ask some questions about it. Are there outliers? The data seem remarkably consistent, but those points in December (12 and 24) warrant more investigation as potential sources of special cause variation. If investigation revealed a source for these data points that indicate these outliers should be disregarded, these outliers could be removed from the calculations for the center line and control limits, or removed from the chart altogether.

What about seasonality, or a trend over the sequence? Neither issue affects this data set, but if they did, we could detrend or deseasonalize the data and chart the residuals to gain more insight into how the data are changing month-to-month.  

Instead of an Individuals chart, one participant in the group suggested using an I-MR chart, which provides both the indiviudals chart and a moving-range chart.  We can use the same single column of data, then examine the resulting I-MR chart for indications of special cause variation. "If not, there's no real reason to believe one year was different than another," this participant suggests. 

Another thing you can do with most of the control charts in Minitab is establish stages.  For example, if we want to look for differences between years, we can add a column of data (call it "Year") to our worksheet that labels each data point by year (2012 or 2013).  Now when we select Stat > Control Charts > Variables Charts for Individuals > I-MR...we will go into the Options dialog and select the Stages tab.  

As shown above, we'll enter the "Year" column to define the stages. Minitab produces the following I-MR chart:

This I-MR chart displays the data in two distinct phases by year, so we can easily see if there are any points from 2013 that are outside the limits for 2012. That would indicate a significant difference. In this case, it looks like the only point outside the control limits for 2012 is that for December 2013, and we already know there's something we need to investigate for the December data.

For the purposes of visual comparison, some members of the Minitab group on LinkedIn advocate the use of a time series plot. To create this graph, we'll need two columns in the data sheet, one for this year's data and one for last year's.  Then we'll choose Graph > Time Series Plot > Multiple and select the "Last Year" and "This Year" columns for our series. Minitab gives us the following plot: 

Because the plot of this year's and last year's data are shown in parallel, it's very easy to see where and by how much they differ over time.

Most of the months appear to be quite close for these data, but once again this graph gives us a dramatic visual representation of the difference between the December data points, not just as compared to the rest of the year, but compared to each other from last year to this. 

Oh, and here's a neat Minitab trick: what if you'd rather have the Index values of 1, 2, 3...12 in the graph above appear as the names of the months?  Very easy!  Just double-click on the X axis, which brings up the Edit Scale dialog box. Click on the Time tab and fill it out as follows: 

(Note that our data start with January, so we use 1 for our starting value. If your data started with the month of February, you'd choose to start with 2, etc.)  Now we just click OK, and Minitab automatically updates the graph to include the names of the month:  

One thing I see again and again on the Minitab LinkedIn group is how a simple question -- how can I look at change from month to month between years? -- can be approached from many different angles.  

What's nice about using statistical software is that we have speed and power to quickly and easily  follow up on all of these angles, and see what different things each approach can tell us about our data. 

The line plot is an incredibly agile but frequently overlooked tool in the quest to better understand your processes.

In any process, whether it's baking a cake or processing loan forms, many factors have the potential to affect the outcome. Changing the source of raw materials could affect the strength of plywood a factory produces. Similarly, one method of gluing this plywood might be better or worse than another.

But what is even more complicated to consider is how these factors might interact. In this case, plywood made with materials obtained from supplier “A” might be strongest when glued with one adhesive, while plywood that uses material from supplier “B” might be strongest when you glued with a different adhesive.

Understanding these kinds of interactions can help you maintain quality when conditions change. But where do you begin? Try starting with a line plot.

Line plots created with Minitab Statistical Software are flexible enough to help you find interactions and response patterns whether you have 2 factors or 20. But while the graph is always created the same way, such changes in scale produce two seemingly distinct types of graph.

With just a few groups…the focus is on interaction effects. In the graph below, a paint company that wants to improve the performance of its products has created a line plot that finds a strong interaction between spray paint formulation and the pressure at which it’s applied.

An interaction is present where the lines are not parallel.

With many groups…the focus is on deviations from an expected response profile. (That's why in the chemical industry this is sometimes called a profile graph.) The line plot below shows a comparison of chemical profiles of a drug from three different manufacturing lines.

Any profile that deviates from the established pattern could suggest quality problems with that production line, but these three profiles look quite similar.

If you’re an experienced Minitab user, these examples may seem familiar. In its various incarnations, the line plot is similar to the interaction plot, to "Calculated X" plots used in PLS, and even to time series plots that appear with more advanced analyses. But the line plot gives you many more options for exploring your data. Here’s another example.

A line plot of the mean sales from a call center shows little interaction between the call script and whether the operators received sales training because the lines are parallel.

But because line plot allows us to examine functions other than the mean, we can see that there is, in fact, an interaction effect in terms of standard deviation. The lines are not parallel. For some reason, the variability in sales seems to be affected by the combination of script and training.

Creating a line plot in Minitab is simple. For example, suppose that your company makes pipes. You’re concerned about the mean diameter of pipes that are produced on three manufacturing lines with raw materials from two suppliers.

Because you’re examining only two factors­—line and supplier—a With Symbols option is appropriate. Use Without Symbols options when you have many groups to consider. Symbols may clutter the graph. Within these categories, you have your choice of data arrangement.

Choose Graph > Line Plot > With Symbols, One Y.
Click OK.

Now, enter the variables to graph. Note that Line Plot allows you to graph a number of different functions apart from the mean.

In Graph variables, enter 'Diameter'.
In Categorical variable for X-scale grouping, enter Line.
In Categorical variable for legend grouping, enter Supplier.
Click OK.

The line plot shows a clear interaction between the supplier and the line that manufacture the pipe. 

The line plot is an ideal way to get a first glimpse into the data behind your processes. The line plot resembles a number of graphs, particularly the interaction plots used with DOE or ANOVA analyses. But, while the function of line plots may be similar, their simplicity makes them an especially appropriate starting point.

It can highlight the variables and the interactions that are worth exploration. Its powerful graphing features also allow you to analyze subsets of your data or to graph different functions of your measurement variable, like standard deviation or count.

The language of statistics is a funny thing, but there usually isn't much to laugh at in the consequences that can follow when misunderstandings occur between statisticians and non-statisticians. We see these consequences frequently in the media, when new studies—that usually contradict previous ones—are breathlessly related, as if their findings were incontrovertible facts.

Similar, though less visible, misinterpretations abound in meeting rooms throughout the business world. When people who work with data and know statistics share their analyses with colleagues who aren't well-versed in the world of data, the message that gets received may be very different than the one the analyst tried to send.    

There are two equally vital solutions to this problem. One is encouraging and instilling greater statistical literacy in the population. Obviously, that's a big challenge that can't be solved by any one statistician or analyst. But we individuals can control the second solution, which is to pay more attention to how we present the results of our analyses, and enhance our sensitivity to the statistical knowledge possessed by our audiences. 

I've written about the challenges of statistical communication before, but I've been thinking about it anew after a friend sent me a link to this post and subsequent discussion about replacing the term "statistical significance."  I won't speculate on the likelihood of that proposal, but it felt like a good time to review some words or phrases that mean one thing in statistical vernacular, but may signify something very different in a popular context.

Here's what I came up with, presented in a tabular form:   

Can you add to my list? What statistical terms have complicated your efforts to communicate results?

Did you ever get a pair of jeans or a shirt that you liked, but didn't quite fit you perfectly? That happened to me a few months ago. The jeans looked good, and they were very well made, but it took a while before I was comfortable wearing them.

I much prefer it when I can get a pair with a perfect fit, that feel like I was born in them, with no period of "adjustment." 

So which pair do you think I wear more often...the older pair that fits me like a glove, or the newer ones that aren't quite as comfortable? You already know the answer, because I'll bet you have a favorite pair of jeans, too. 

So what does all this have to do with statistical software? Just this: if you can get statistical software that's perfectly matched to how you're going to use it, you're going to feel more comfortable, confident, and at ease when from the second you open it. 

We do strive to make Minitab Statistical Software very easy to use from the first time you launch it. Our roots lie in providing tools that make data analysis easier, and that's still our mission today. But we know a little bit of tailoring can make a garment that feels very good into one that feels great. 

So if you want to tailor your Minitab software to fit you perfectly, we also make that easy—even if you have multiple people using Minitab on the same computer. 

If you're like most people, you want software that gives you the options you want, when you want them. You want a menu has everything organized just the way you like it. And while we're at it, how about a toolbar that gives you immediate access to the tools you know you'll be using most frequently? 

We don't think that's too much to ask. 

In my job, I frequently need to perform a series of analyses on data about marketing and online traffic. It's easy enough to access those tools through Minitab's default menus, but one day I realized I didn't even need to do that—I could just make myself a menu in Minitab that includes the tools I use most frequently. 

Taking this thought from idea to execution was a breeze. I simply right-clicked on the menu bar and selected the "Customize" option. 

There are many more ways you can customize Minitab to suit your needs, including the creation of customized toolbars and individual profiles, which are great if you share your computer with someone who would like to have Minitab customized to their preferences, too. 

Let us know what you've done to customize Minitab so it fits you perfectly!

Opinions, they say, are like certain anatomical features: everybody has one. Usually that's fine—if everybody thought the same way, life would be pretty boring—but many business decisions are based on opinion. And when different people in an organization reach different conclusions about the same business situation, problems follow. 

Inconsistency and poor quality result when people being asked to make yes / no, pass / fail, and similar decisions don't share the same opinions, or base their decisions on divergent standards. Consider the following examples. 

Manufacturing: Is this part acceptable? 

Billing and Purchasing: Are we paying or charging an appropriate amount for this project? 

Lending: Does this person qualify for a new credit line? 

Supervising: Is this employee's performance satisfactory or unsatisfactory? 

Teaching: Are essays being graded consistently by teaching assistants?

It's easy to see how differences in judgment can have serious impacts. I wrote about a situation encountered by the recreational equipment manufacturer Burley. Pass/fail decisions of inspectors at a manufacturing facility in China began to conflict with those of inspectors at Burley's U.S. headquarters. To make sure no products reached the market unless the company's strict quality standards were met, Burley acted quickly to ensure that inspectors at both facilities were making consistent decisions about quality evaluations. 

Sometimes We Can't Just Agree to Disagree

The challenge is that people can have honest differences of opinion about, well, nearly everything—including different aspects of quality. So how do you get people to make business decisions based on a common viewpoint, or standard?

Fortunately, there's a statistical tool that can help businesses and other organizations figure out how, where, and why people evaluate the same thing in different ways. From there, problematic inconsistencies can be minimized. Also, inspectors and others who need to make tough judgment calls can be confident they are basing their decisions on a clearly defined, agreed-upon set of standards. 

That statistical tool is called "Attribute Agreement Analysis," and using it is easier than you might think—especially with data analysis software such as Minitab. 

What Does "Attribute Agreement Analysis" Mean? 

Statistical terms can be confusing, but "attribute agreement analysis" is exactly what it sounds like: a tool that helps you gather and analyze data about how much agreement individuals have on a given attribute.

So, what is an attribute? Basically, any characteristic that entails a judgment call, or requires us to classify items as this or that. We can't measure an attribute with an objective scale like a ruler or thermometer. The following statements concern such attributes:

• This soup is spicy.
• The bill for that repair is low. 
• That dress is red. 
• The carpet is rough. 
• That part is acceptable. 
• This candidate is unqualified. 

Attribute agreement analysis uses data to understand how different people assess a particular item's attribute, how consistently the same person assesses the same item on multiple occasions, and compares both to the "right" assessment. 

This method can be applied to any situation where people need to appraise or rate things. In a typical quality improvement scenario, you might take a number of manufactured parts and ask multiple inspectors to assess each part more than once. The parts being inspected should include a roughly equal mix of good and bad items, which have been identified by an expert such as a senior inspector or supervisor. 

In my next post, we'll look at an example from the financial industry to see how a loan department used this statistical method to make sure that applications for loans were accepted or rejected appropriately and consistently. 

Previously, I discussed how business problems arise when people have conflicting opinions about a subjective factor, such as whether something is the right color or not, or whether a job applicant is qualified for a position. The key to resolving such honest disagreements and handling future decisions more consistently is a statistical tool called attribute agreement analysis. In this post, we'll cover how to set up and conduct an attribute agreement analysis. 

A busy loan office for a major financial institution processed many applications each day. A team of four reviewers inspected each application and categorized it as Approved, in which case it went on to a loan officer for further handling, or Rejected, in which case the applicant received a polite note declining to fulfill the request. 

The loan officers began noticing inconsistency in approved applications, so the bank decided to conduct an attribute agreement analysis on the application reviewers.

Two outcomes were possible: 

1. The reviewers make the right choice most of the time. If this is the case, loan officers can be confident that the reviewers do a good job, rejecting risky applicants and approving applicants with potential to be good borrowers. 

2. The reviewers too often choose incorrectly. In this case, the loan officers might not be focusing their time on the best applications, and some people who may be qualified may be rejected incorrectly. 

One particularly useful thing about an attribute agreement analysis: even if reviewers make the wrong choice too often, the results will indicate where the reviewers make mistakes. The bank can then use that information to help improve the reviewers' performance. 

A typical attribute agreement analysis asks individual appraisers to evaluate multiple samples, which have been selected to reflect the range of variation they are likely to observe. The appraisers review each sample item several times each, so the analysis reveals how not only how well individual appraisers agree with each other, but also howl consistently each appraiser evaluates the same item. 

For this study, the loan officers selected 30 applications, half of which the officers agreed should receive approval and half which should be rejected. These included both obvious and borderline applications. 

Next, each of the four reviewers was asked to approve or reject the 30 applications two times. These evaluation sessions took place one week apart, to make it less likely they would remember how they'd classified them the first time. The applications were randomly ordered each time.

The reviewers did not know how the applications had been rated by the loan officers. In addition, they were asked not to talk about the applications until after the analysis was complete, to avoid biasing one another. 

You don't need to use software to perform an Attribute Agreement Analysis, but a program like Minitab does make it easier both to plan the study and gather the data, as well as to analyze the data after you have it. There are two ways to set up your study in Minitab. 

The first way is to go to Stat > Quality Tools > Create Attribute Agreement Analysis Worksheet... as shown here: 

This option calls up an easy-to-follow dialog box that will set up your study, randomize the order of reviewer evaluations, and permit you to print out data collection forms for each evaluation session. 

But it's even easier to use Minitab's Assistant. In the menu, select Assistant > Measurement Systems Analysis..., then click the Attribute Agreement Worksheet button:

That brings up the following dialog box, which walks you through setting up your worksheet and printing out data collection forms, if desired. For this analysis, the Assistant dialog box is filled out as shown here: 

After you press OK, Minitab creates a worksheet for you and gives you the option to print out data collection forms for each reviewer and each trial. As you can see in the "Test Items" column below, Minitab randomizes the order of the observed items in each trial automatically, and the worksheet is arranged so you need only enter the reviewers' judgments in the the "Results" column. 

In my next post, we'll analyze the data collected in this attribute agreement analysis. 

In the first part of this series, we saw how conflicting opinions about a subjective factor can create business problems. In part 2, we used Minitab's Assistant feature to set up an attribute agreement analysis study that will provide a better understanding of where and when such disagreements occur. 

We asked four loan application reviewers to reject or approve 30  selected applications, two times apiece. Now that we've collected that data, we can analyze it. If you'd like to follow along, you can download the data set here.

As is so often the case, you don't need statistical software to do this analysis—but with 240 data points to contend with, a computer and software such as Minitab will make it much easier. 

Last time, we showed that the only data we need to record is whether each appraiser approved or rejected the sample application in each case. Using the data collection forms and the worksheet generated by Minitab, it's very easy to fill in the Results column of the worksheet. 

The next step is to use statistics to better understand how well the reviewers agree with each others' assessments, and how consistently they judge the same application when they evaluate it again. Choose Assistant > Measurement Systems Analysis (MSA)... and press the Attribute Agreement Analysis button to bring up the appropriate dialog box: 

The resulting dialog couldn't be easier to fill out. Assuming you used the Assistant to create your worksheet, just select the columns that correspond to each item in the dialog box, as shown: 

If you set up your worksheet manually, or renamed the columns, just choose the appropriate column for each item. Select the value for good or acceptable items—"Accept," in this case—then press OK to analyze the data.  

Minitab's Assistant generates four reports as part of its attribute agreement analysis. The first is a summary report, shown below: 

The green bar at top left of the report indicates that overall, the error rate of the application reviewers is 15.8%. That's not as bad as it could be, but it certainly indicates that there's room for improvement! The report also shows that 13% of the time, the reviewers rejected applications that should be accepted, and they accepted applications that should be rejected 18% of the time. In addition, the reviewers rated the same item two different ways almost 22% of the time.

The bar graph in the lower left indicates that Javier and Julia have the lowest accuracy percentages among the reviewers at 71.7% and 78.3%, respectively. Jim has the highest accuracy, with 96%, followed by Jill at 90%.

The second report from the Assistant, shown below, provides a graphic summary of the accuracy rates for the analysis.

This report illustrates the 95% confidence intervals for each reviewer in the top left, and further breaks them down by standard (accept or reject) in the graphs on the right side of the report. Intervals that don't overlap are likely to be different. We can see that overall, Javier and Jim have different overall accuracy percentages. In addition, Javier and Jim have different accuracy percentages when it comes to assessing those applications that should be rejected. However, most of the other confidence intervals overlap, suggesting that the reviewers share similar abilities. Javier clearly has the most room for improvement, but none of the reviewers are performing terribly when compared to the others. 

The Assistant's third report shows the most frequently misclassified items, and individual reviewers' misclassification rates:

This report shows that App 9 gave the reviewers the most difficulty, as it was misclassified almost 80% of the time. (A check of the application revealed that this was indeed a borderline application, so the fact that it proved challenging is not surprising.) Among the reject applications that were mistakenly accepted, App 5 was misclassified about half of the time. 

The individual appraiser misclassification graphs show that Javier and Julia both misclassified acceptable applications as rejects about 20% of the time, but Javier accepted "reject" applications nearly 40% of the time, compared to roughly 20% for Julia. However, Julia rated items both ways nearly 40% of the time, compared to 30% for Javier. 

The last item produced as part of the Assistant's analysis is the report card:

This report card provides general information about the analysis, including how accuracy percentages are calculated. It also can alert you to potential problems with your analysis (for instance, if there were an imbalance in the amount of acceptable to rejectable items being evaluated); in this case, there are no alerts we need to be concerned about. 

The results of this attribute agreement analysis give the bank a clear indication of how the reviewers can improve their overall accuracy. Based on the results, the loan department provided additional training for Javier and Julia (who also were the least experienced reviewers on the team), and also conducted a general review session for all of the reviewers to refresh their understanding about which factors on an application were most important. 

However, training may not always solve problems with inconsistent assessments. In many cases, the criteria on which decisions should be based are either unclear or nonexistent. "Use your common sense" is not a defined guideline! In this case, the loan officers decided to create very specific checklists that the reviewers could refer to when they encountered borderline cases. 

After the additional training sessions were complete and the new tools were implemented, the bank conducted a second attribute agreement analysis, which verified improvements in the reviewers' accuracy. 

If your organization is challenged by honest disagreements over "judgment calls," an attribute agreement analysis may be just the tool you need to get everyone back on the same page. 

Have you ever wanted to know the odds of something happening, or not happening? 

It's the kind of question that students are frequently asked to calculate by hand in introductory statistics classes, and going through that exercise is a good way to become familiar with the mathematical formulas the underlie probability (and hence, all of statistics). 

But let's be honest: when class is over, most people don't take the time to calculate those probabilities—at least, not by hand. Some people even resort to "just making it up." Needless to say, we at Minitab are firmly opposed to just making it up.

The good news is that determining the real odds of something happening doesn't have to be hard work! If you don't want to calculate the probabilities by hand, just let a statistical software package such as Minitab do it for you. 

Let's look at how to compute binomial probabilities. The process we'll go through is similar for any of the 24 distributions Minitab includes.

We use the binomial distribution to characterize a process with two outcomes—for example, if a part passes or fails inspection, if a candidate wins or loses an election, or if a coin lands on heads or tails. This distribution is used frequently in quality control, opinion surveys, medical research, and insurance.

Suppose I want to know the probability of getting a certain number of heads in 10 tosses of a fair coin. I need to calculate the odds for a binomial distribution with 10 trials (n=10) and probability of success p=0.5.

To compute the probability of exactly 8 successes, select Calc > Probability Distributions > Binomial...

Choose “probability” in the dialog, then enter the number of trials (10) and the probability of success (0.5) for “event probability." If we wanted to calculate the odds for more than one number of events, we could enter them in a worksheet column. But since for now we just want the probability of getting exactly 8 heads in 10 tosses, choose the "Input Constant" option, enter 8, and press OK. 

The following output appears in the session window. It tells us that if we toss a fair coin with an 50% probability of landing on heads, the odds of getting exactly 8 heads out of 10 tosses are just 4%.

What if we wanted to know the cumulative probability of getting 8 heads in 10 tosses? Cumulative probability is the odds of one, two, or more events taking place. The word to remember is "or," because that's what cumulative probability tells you. What are the chances that when you toss this coin 10 times, you'll get 8 or fewer heads? That's cumulative probability.

To compute cumulative probabilities, select “cumulative probability” in the binomial distribution dialog.

The probability of 8 or fewer successes, is P(X ≤ 8) = 0.989258, or 98%:

We can also use Minitab to calculate a full table of probabilities. In the worksheet, enter all of the values of the number of successes in a column. For example, for a series of 10 tosses, you would enter 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Next we'll select Calc > Probability Distributions > Binomial... again, but this time choose “Input column” and select C1 instead of using the "Input constant." Specify a different column for storage and press OK.

The probabilities appear in column C2:

Suppose you want to see the distribution of these probabilities in a graph? Select Graph > Bar Charts..., then use the dialog box choose View Single. 

Just complete the dialog as shown:

When you press OK, Minitab produces this bar chart: 

If you need to know the precise value for a given number of events, just hover over that column and Minitab displays the details:

As you can see, using Minitab to check and graph the probabilities of different events is not difficult. I hope knowing this increases the odds that the next time you wonder about the likelihood of an event, you'll be able to find it quickly and accurately!

Statistics can be challenging, especially if you're not analyzing data and interpreting the results every day. Statistical software makes things easier by handling the arduous mathematical work involved in statistics. But ultimately, we're responsible for correctly interpreting and communicating what the results of our analyses show.

The p-value is probably the most frequently cited statistic. We use p-values to interpret the results of regression analysis, hypothesis tests, and many other methods. Every introductory statistics student and every Lean Six Sigma Green Belt learns about p-values. 

Yet this common statistic is misinterpreted so often that at least one scientific journal has abandoned its use.

Typically, a P value is defined as "the probability of observing an effect at least as extreme as the one in your sample data—if the null hypothesis is true." Thus, the only question a p-value can answer is this one:

How likely is it that I would get the data I have, assuming the null hypothesis is true?

If your p-value is less than your selected alpha level (typically 0.05), you reject the null hypothesis in favor of the alternative hypothesis. If the p-value is above your alpha value, you fail to reject the null hypothesis. It's important to note that the null hypothesis is never accepted; we can only reject or fail to reject it. 

Consider a typical hypothesis test—say, a 2-sample t-test of the mean weight of boxes of cereal filled at different facilities. We collect and weigh 50 boxes from each facility to confirm that the mean weight for each line's boxes is the listed package weight of 14 oz. 

Our null hypothesis is that the two means are equal. Our alternative hypothesis is that they are not equal. 

To run this test in Minitab, we enter our data in a worksheet and select Stat > Basic Statistics > 2-Sample T-test. If you'd like to follow along, you can download the data and, if you don't already have it, get the 30-day trial of Minitab. In the t-test dialog box, select Both samples are in one column from the drop-down menu, and choose "Weight" for Samples, and "Facility" for Sample IDs.

Minitab gives us the following output, and I've highlighted the p-value for the hypothesis test:

So we have a p-value of 0.029, which is less than our selected alpha value of 0.05. Therefore, we reject the null hypothesis that the means of Line A and Line B are equal. Note also that while the evidence indicates the means are different, that difference is estimated at 0.338 oz—a pretty small amount of cereal. 

So far, so good. But this is the point at which trouble often starts.

The p-value of 0.029 means we reject the null hypothesis that the means are equal. But that doesn't mean any of the following statements are accurate:

1. "There is 2.9% probability the means are the same, and 97.1% probability they are different." 
   We don't know that at all. The p-value only says that if the null hypothesis is true, the sample data collected would exhibit a difference this large or larger only 2.9% of the time. Remember that the p-value doesn't tell you anything directly about what you've seen. Instead, it tells you the odds of seeing it. 


2. "The p-value is low, which indicates there's an important difference in the means." 
   Based on the 0.029 p-value shown above, we can conclude that a statistically significant difference between the means exists. But the estimated size of that difference is less than a half-ounce, and won't matter to customers. A p-value may indicate a difference exists, but it tells you nothing about its practical impact.


3. "The low p-value shows the alternative hypothesis is true."
   A low p-value provides statistical evidence to reject the null hypothesis—but that doesn't prove the truth of the alternative hypothesis. If your alpha level is 0.05, there's a 5% chance you will incorrectly reject the null hypothesis. Or to put it another way, if a jury fails to convict a defendant, it doesn't prove the defendant is innocent: it only means the prosecution failed to prove the defendant's guilt beyond a reasonable doubt. 

These misinterpretations happen frequently enough to be a concern, but that doesn't mean that we shouldn't use p-values to help interpret data. The p-value remains a very useful tool, as long as we're interpreting and communicating its significance accurately.

It's one thing to keep all of this straight if you're doing data analysis and statistics all the time. It's another thing if you're only analyze data occasionally, and need to do many other things in between—like most of us. "Use it or lose it" is certainly true about statistical knowledge, which could well be another factor that contributes to misinterpreted p-values. 

If you're leery of that happening to you, a good way to avoid that possibility is to use the Assistant in Minitab to perform your analyses. If you haven't used it yet, the Assistant menu guides you through your analysis from start to finish. The dialog boxes and output are all in plain language, so it's easy to figure out what you need to do and what the results mean, even if it's been a while since your last analysis. (But even expert statisticians tell us they like using the Assistant because the output is so clear and easy to understand, regardless of an audience's statistical background.) 

So let's redo the analysis above using the Assistant, to see what that output looks like and how it can help you avoid misinterpreting your results—or having them be misunderstood by others!

Start by selecting Assistant > Hypothesis Test... from the Minitab menu. Note that a window pops up to explain exactly what a hypothesis test does. 

The Assistant asks what we're trying to do, and gives us three options to choose from.

We know we want to compare a sample from Line A with a sample from Line B, but what if we can't remember which of the 5 available tests is the appropriate one in this situation? We can get guidance by clicking "Help Me Choose."

The choices on the diagram direct us to the appropriate test. In this case, we choose continuous data instead of attribute (and even if we'd forgotten the difference, clicking on the diamond would explain it). We're comparing two means instead of two standard deviations, and we're measuring two different sets of items since our boxes came from different production lines. 

Now we know what test to use, but suppose you want to make sure you don't miss anything that's important about the test, like requirements that must be met? Click the "more..." link and you'll get those details. 

Now we can proceed to the Assistant's dialog box. Again, statistical jargon is minimized and everything is put in straightforward language. We just need to answer a few questions, as shown. Note that the Assistant even lets us tell it how big a difference needs to be for us to consider it practically important. In this case, we'll enter 2 ounces.

When we press OK, the Assistant performs the t-test and delivers three reports. The first of these is a summary report, which includes summary statistics, confidence intervals, histograms of both samples, and more. And interpreting the results couldn't be more straightforward than what we see in the top left quadrant of the diagram. In response to the question, "Do the means differ?" we can see that p-value of 0.029 marked on the bar, very far toward the "Yes" end of the scale. 

Next is the Diagnostic Report, which provides additional information about the test. 

In addition to letting us check for outliers, the diagnostic report shows us the size of the observed difference, as well as the chances that our test could detect a practically significant difference of 2 oz. 

The final piece of output the Assistant provides is the report card, which flags any problems or concerns about the test that we would need to be aware of. In this case, all of the boxes are green and checked (instead of red and x'ed). 

When you're not doing statistics all the time, the Assistant makes it a breeze to find the right analysis for your situation and to make sure you interpret your results the right way. Using it is a great way to make sure you're not attaching too much, or too little, importance on the results of your analyses.

A recent discussion on the Minitab Network on LinkedIn pertained to the I-MR chart. In the course of the conversation, a couple of people referred to it as "The Swiss Army Knife of control charts," and that's a pretty great description. You might be able to find more specific tools for specific applications, but in many cases, the I-MR chart gets the job done quite adequately.

When you're collecting samples of data to learn about your process, it's generally a good idea to group the sample data into subgroups, if possible. The idea is that these subgroups represent "snapshots" of your process. But what if you can't? Your process might have a very long cycle time, or sampling or testing might be destructive or expensive. Production volume may be too low. Or grouping your measurements might not feasibly capture variability for a given time. In many such instances, an I-MR chart is a good way to go. 

Let's take a closer look at this "Swiss Army Knife" control chart and see what it does and how it works. The I-MR chart, like other control charts, has three main uses: 

1. To monitor the stability of your process.
   Even the most stable process has variation in some amount, and attempts to "fix" normal fluctuations in a process may actually introduce instability. An I-MR chart can show you changes that should be addressed.
    
2. To determine whether your process is stable enough to improve.
   You generally don't want to make improvements to a process that isn't stable. That's because the instability keeps you from confidently assessing the impact of your changes. You can confirm (or deny) your process stability with an I-MR chart before you make improvements. 
    
3. To demonstrate process performance improvements.
   If your improvements had a big impact, how do you show your stakeholders and higher-ups? Before-and-after I-MR charts provide powerful visual proof. 

Now that we know what the I-MR chart can do, let's consider what it is. The I-MR is actually the combination of two different charts in a single presentation. The graph's top part is an Individuals (I) chart. It shows you the value of each observation (the individuals), and helps you assess the center of the process.

The graph at the bottom is called a Moving Range (MR) chart. It calculates the variation of your process using ranges of two (or more) successive observations, and plots them. 

The green line represents the process mean and process variation on the I and MR portions of the chart, respectively, while the red lines represent the upper and lower control limits. 

Suppose the chemical company you work for makes a custom solution, and you need to assess whether its pH value is consistent over time. You record a single pH measurement per batch. Since you are collecting individual samples rather than subgroups, the I-MR chart can help. 

You record pH measurements for 25 consecutive batches. To prepare that data for the I-MR chart, just enter those measurements, in order, in a single column of a Minitab worksheet. (You can download this data set here to follow along. If you don't already have Minitab Statistical Software, you can use the free trial.) 

Now select Stat > Control Charts > Variables Charts for Individuals > I-MR from the menu, and choose pH as the Variable. (If you have more than one variable you want to chart, you can enter more than one column here and Minitab will produce multiple I-MR charts simultaneously.) If you want to add labels, divide the data into stages, and more, you can do that in the "I-MR Options" subdialog.

Let's assume that we want to detect any possible special-cause variation. Click I-MR Options and select Tests. These tests highlight points that exceed the control limits and detect specific patterns in the data. In the dropdown menu, select "Perform all tests for special causes," and then OK out of the dialog.

Check the MR Chart First

After you press OK, Minitab generates your I-MR chart: 

It might seem counterintuitive, but you should examine the MR chart at the bottom first. This chart reveals if the process variation is in or out of control. The reason you want to check this first is that if the MR chart is out of control, the I-chart control limits won't be accurate. In that case, any unusual points on the I chart may result from unstable variation, not changes in the process center. But when the MR chart is in control, an out-of-control I chart does indicate changes in the process center.

When points fail Minitab's tests, they are marked in red. In this MR chart, none of the individual points fall outside the lower and upper control limits of 0 and 0.4983, respectively. In addition, the points also have a random pattern. That means our process variation is in control, and we're good to take the next step: looking at the I Chart.

Check the I Chart After the MR Chart

The individuals (I) chart shows if your process mean is in control. In contrast to the MR chart, this I chart shows evidence of potential nonrandom patterns. 

Minitab can perform up to eight different special-cause variation tests for the I chart. Problem observations are marked in red, and also display the number of the corresponding failed test.

This I chart shows that three separate observations failed two different tests. We can check the Session Window for more details about why Minitab flagged each point: 

Observation 8 failed Test 1, which means this observation was more than 3 standard deviations from the center line—the strongest evidence that a process is out of control. Observations 20 and 21 failed Test 5, which tests for a run of two out of three points with the same sign that fall more than two standard deviations from the center line. Test 5 provides additional sensitivity for detecting smaller shifts in the process mean.

This I-MR chart indicates that the process average is unstable and therefore the process is out of control, possibly due to the presence of special causes.

After looking at your data in the I-MR chart, you know there may be a problem that needs to be addressed. That's the whole purpose of the control chart! The next step is to identify and address the source of this special-cause variation. Until these causes are eliminated, the process cannot achieve a state of statistical control.

If you'd like to know more about the behind-the-scenes math that goes into this chart, check out my colleague Marilyn Wheatley's post about how I-MR control chart limits are calculated.

My colleague Cody Steele wrote a post that illustrated how the same set of data can appear to support two contradictory positions. He showed how changing the scale of a graph that displays mean and median household income over time drastically alters the way it can be interpreted, even though there's no change in the data being presented.

When we analyze data, we need to present the results in an objective, honest, and fair way. That's the catch, of course. What's "fair" can be debated...and that leads us straight into "Lies, damned lies, and statistics" territory.  

Cody's post got me thinking about the importance of statistical literacy, especially in a mediascape saturated with overhyped news reports about seemingly every new study, not to mention omnipresent "infographics" of frequently dubious origin and intent.

As consumers and providers of statistics, can we trust our own impressions of the information we're bombarded with on a daily basis? It's an increasing challenge, even for the statistics-savvy. 

The increased amount of information available, combined with the acceleration of the news cycle to speeds that wouldn't have been dreamed of a decade or two ago, means we have less time available to absorb and evaluate individual items critically. 

A half-hour television news broadcast might include several animations, charts, and figures based on the latest research, or polling numbers, or government data. They'll be presented for several seconds at most, then it's on to the next item. 

Getting news online is even more rife with opportunities for split-second judgment calls. We scan through the headlines and eyeball the images, searching for stories interesting enough to click on. But with 25 interesting stories vying for your attention, and perhaps just a few minutes before your next appointment, you race through them very quickly. 

But when we see graphs for a couple of seconds, do we really absorb their meaning completely and accurately? Or are we susceptible to misinterpretation?  

Most of the graphs we see are very simple: bar charts and pie charts predominate. But as statistics educator Dr. Nic points out in this blog post, interpreting even simple bar charts can be a deceptively tricky business. I've adapted her example to demonstrate this below.  

A city surveyed residents of two neighborhoods about the quality of service they get from local government. Respondents were asked to rate local services on a scale of 1 to 10. Their responses were charted using Minitab Statistical Software, as shown below.  

Take a few seconds to scan the charts, then choose which neighborhood's responses exhibit the most variation, Ferndale or Lawnwood?

Seems pretty straightforward, right? Lawnwood's graph is quite spiky and disjointed, with sharp peaks and valleys. The graph of Ferndale's responses, on the other hand, looks nice and even. Each bar's roughly the same height.  

It looks like Lawnwood's responses have the most variation. But let's verify that impression with some basic descriptive statistics about each neighborhood's responses:

Uh-oh. A glance at the graphs suggested that Lawnwood has more variation, but the analysis demonstrates that Ferndale's variation is, in fact, much higher. How did we get this so wrong?  

The answer lies in how the data were presented. The charts above show frequencies, or counts, rather than individual responses.  

What if we graph the individual responses for each neighborhood?  

In these graphs, it's easy to see that the responses of Ferndale's citizens had much more variation than those of Lawnwood. But unless you appreciate the differences between values and frequencies—and paid careful attention to how the first set of graphs was labeled—a quick look at the earlier graphs could well leave you with the wrong conclusion. 

Since you're reading this, you probably both create and consume data analysis. You may generate your own reports and charts at work, and see the results of other peoples' analyses on the news. We should approach both situations with a certain degree of responsibility.  

When looking at graphs and charts produced by others, we need to avoid snap judgments. We need to pay attention to what the graphs really show, and take the time to draw the right conclusions based on how the data are presented.  

When sharing our own analyses, we have a responsibility to communicate clearly. In the frequency charts above, the X and Y axes are labeled adequately—but couldn't they be more explicit?  Instead of just "Rating," couldn't the label read "Count for Each Rating" or some other, more meaningful description? 

Statistical concepts may seem like common knowledge if you've spent a lot of time working with them, but many people aren't clear on ideas like "correlation is not causation" and margins of error, let alone the nuances of statistical assumptions, distributions, and significance levels.

If your audience includes people without a thorough grounding in statistics, are you going the extra mile to make sure the results are understood? For example, many expert statisticians have told us they use the Assistant in Minitab 17 to present their results precisely because it's designed to communicate the outcome of analysis clearly, even for statistical novices. 

If you're already doing everything you can to make statistics accessible to others, kudos to you. And if you're not, why aren't you?