In an earlier post, I shared some great hidden helpers in Minitab Statistical Software that even many veteran users don't know about. Here are a few more!

Minitab’s Project Manager allows you to navigate, view, and manipulate various parts of your project. Right-clicking either the folders or their contents lets you access a variety of menus that allow you to manage Session Window output, graphs, worksheets, command language, and related project areas. You can also copy any or all analyses and graphs to Minitab’s built-in ReportPad to create reports and share your results with colleagues who may not have Minitab. You can also move ReportPad contents to a more powerful word processing program for further editing and layout. Press CTRL+I to access Minitab’s Project Manager.

To change the color of bars and symbols on a graph, double-click on any bar or symbol. Then use the Attributes tab to change the fill pattern, color, outline, and other aspects of how your graph appears.

To import data from Excel into Minitab, choose File > Open Worksheet, select Excel from Files of type and navigate to your Excel document. If you open a workbook with multiple sheets, each Excel sheet opens into a separate Minitab worksheet.

You can also click Options and Preview to specify the data to import and to ensure that it is formatted properly. Need to import a CSV or text file? You can import those and other data file formats using File > Open Worksheet.

Minitab’s Autofill capability lets you automatically repeat the value in a cell (or cells), or follow a pattern as you click and drag down a column or columns in a worksheet.

To repeat a value, select the cell, then hover your cursor on the lower right corner of the cell. When the cross appears, click and drag down the column to repeat the value in other cells.

To repeat a pattern of values rather than a single value, press CTRL as you drag, following the same convention as in Excel.

You can even use Autofill with custom lists. For example, with the worksheet active, choose Editor > Define Custom Lists to define “Mon Tues Wed Thu Fri Sat Sun” as a list, type one of the values in the first row of a column, and click and drag to fill in the list.

What features in Minitab do you find most helpful?  Let us know and we’ll share your favorites.   

We received the following question via social media recently:

I am using Minitab 17 for ANOVA. I calculated the mean and standard deviation for these 15 values, but the standard deviation is very high. If I delete some values, I can reduce the standard deviation. Is there an option in Minitab that will automatically indicate values that are out of range and delete them so that the standard deviation is low?

In other words, this person wanted a way to automatically eliminate certain values to lower the standard deviation.

Fortunately, Minitab 17 does not have the functionality that this person was looking for.

Why is that fortunate?  Because cherry-picking data isn’t a statistically sound practice. In fact, if you do it specifically to reduce variability, removing data points can amount to fraud.

So that raises a question: is it ever acceptable to remove data? The answer is yes. If you know, for a fact, that some values in your data were inappropriately attained, then it is okay to remove these bad data points. For example, if data entry errors resulted in a few data points from Sample A being entered under Sample B, it would make sense to remove those data points from the analysis of Sample B.

But you may encounter other suggestions for removing data. Some people will use a "trimmed" data set. This means you remove the top and bottom 1-2 samples. Depending upon what the data is, and how you plan to use it, this too can be fraud.

Some people will use the term "Data Cleansing." When they do this, they remove a few data points from a large data set. The end results tend to be minimal on data analysis. But when this changes the end results of an analysis, it again can amount to fraud.

The bottom line? If you don't know for certain that the data points are bad, removing them—especially to change the outcome of an analysis—is virtually impossible to defend.

Minitab 17 won't automatically delete values to make your standard deviation small. However, our statistical software does make it easy to identify potential outliers that may be skewing your data, so that you can investigate them. You can access the outlier detection tests at Stat > Basic Statistics > Outlier Test…

You can also look at specific statistical measures that indicate the presence of outliers in regression and ANOVA.

Of course, before removing any data points you need to make sure that the values are really outliers. First, think about whether those values were collected under the same conditions as the other values. Was there a substitute lab technician working on the day that the potential outliers were collected? If so, did this technician do something differently than the other technicians? Or could the digits in a value be reversed? That is, was 48 recorded as 84?

If you have just one factor in an ANOVA, try using Assistant > Hypothesis Tests > One-Way ANOVA… Outliers will be flagged in the output automatically:

You could then run the analysis again after manually removing outliers as appropriate.

You also can use a boxplot chart to identify outliers:

As you can see above, Minitab's boxplot uses an asterisk (*) symbol to identify outliers, defined as observations that are at least 1.5 times the interquartile range from the edge of the box. You can easily identify the unwanted data point by clicking on the outlier symbols so you can investigate further. After editing the worksheet you can update the boxplot, perhaps finding more outliers to remove.

While Minitab won't offer an automated "make my data look acceptable" tool, the software does make it easy to find specific data points that may take the results of your analysis in an in inaccurate or unwanted direction.

However, before removing any "bad" data points you should understand their causes and be sure you can avoid recurrence of those causes in the actual process. If the "bad" data could contribute to a more accurate understanding of the actual process, removing them from the calculation will produce wrong results. 

In his post yesterday, my colleague Jim Colton applied binary logistic regression to data on the current ebola virus outbreak in Guinea, Liberia, and Sierra Leone, and revealed that, horrific as it is, this outbreak actually appears to have a lower death rate than some earlier ones. 

He didn't address the potential for a global ebola pandemic, but over the last few days more than enough leading publications have featured extremely scary headlines about this extremely remote possibility. Less reputable organizations have promulgated even more exaggerated stories, usually with some ludicrous conspiracy angle ("they want it to spread!") thrown in for good measure.

Medical experts say that the risk of actually getting ebola is extraordinarily low, especially for people living outside of western Africa. The Boston Globe's Evan Horowitz did a particularly nice job illustrating just how minimal that risk is. 

The current outbreak is certainly newsworthy, but the ebola hysteria—a perjorative term I use very deliberately here—seems silly.

In the late 1990s I interviewed an epidemiologist while I was writing an article about a hantavirus outbreak in the southwestern U.S. I asked if hantavirus was one of the scarier diseases he studied. "As an epidemiologist, hantavirus and other exotic bugs don't scare me at all," he replied.

"The disease that keeps me up at night is the flu."

I thought he was joking. But as he explained the toll influenza has claimed throughout history, and how many lives the Spanish flu pandemic took less than 100 years ago, he shattered my misperception of the flu as a minor nuisance that had been defeated by modern science.

To him, every year the world doesn't lose millions of people to influenza seems like a momentary reprieve.

I haven't worried much about bugs like hantavirus and ebola since, although I still enjoy a good scare story like The Stand, 28 Days Later, or Contagion. 

On Tuesday, one of my buddies who's a bit of a hypochondriac called to ask what ebola precautions I was taking. He couldn't believe it when I said, "Nothing." Rather than argue with him, I started looking for a quick and dirty way to put the risk of an ebola epidemic into perspective.

The Boston Globe's story about the risk of ebola included a list of several horrible diseases, the average number of people an infected person spreads the disease to ("reproductive number"), and how long it takes a newly-exposed person to become infectious themselves ("generation time").

Using those numbers, I created a very basic worst-case scenario for the different diseases they cited. First I rounded the maximum reproductive number and the minimum generation time to integers. Then I extrapolated the spread over a 30-day period from a single person exposed to each disease. For each disease, I divided the 30-day period by the minimum generation time to obtain a number of periods, and simply totaled the number of new cases you'd expect to see in each period based on the reproductive number. 

So, if a disease with a reproductive number of 2 had 4 generation periods in 30 days, from a single person we could expect to see 1 + 2 + 22 + 23 + 24 = 31 total cases. 

Similarly, as shown in the second row in the table above, a malady with a reproductive number of 6 and 3 generation periods over 30 days would total 1 + 6 + 62 + 63 = 259 cases. 

The following table shows the rough totals I came up with for each disease cited in the Boston Globe article:

Of course, this quick and dirty extrapolation would never satisfy someone looking for the most accurate estimate of how these diseases could spread: this model only considers time and the reproductive number, which is itself a metric with some serious limitations.

All I wanted to do was show my friend that not worrying about an ebola pandemic in the U.S. is hardly a suicidal act.

The table above would seem to put the relative risk from ebola into perspective, but it's often useful to visualize the numbers. So I turned to one of my favorite quality tools, the Pareto chart, to illustrate my worst-case scenarios to my friend. 

Pareto charts rank "defects" based on frequency of occurrence, although you can also account for severity of the defect, the cost, or any other metric you like. To create a Pareto chart in Minitab, I opened Stat > Quality Tools > Pareto Chart...

I entered the column of disease names as "Defect" and entered "Worst Case" as the summarized value column. Minitab provided the Pareto chart shown below:

These data suggest that the epidemiologist I spoke with so many years ago had good reason to fear a flu epidemic. Of the diseases in the Boston Globe story, influenza infected the most people in my worst-case scenario, followed closely by cholera and measles.

Returning to the subject of today's headlines, you'll notice that "ebola" doesn't even appear on this chart—it's lumped in with SARS, dengue fever, rubella, and smallpox in the "Other" category, which combined accounts for less than 5% of the total number of infections calculated in my scenario. 

Clearly I'm no epidemiologist, but this quick exercise did serve its intended purpose with my friend. He's already planning to get a flu shot this year—his first ever—and he's much less concerned about exotic maladies that, however horrible, we're extremely unlikely to ever encounter.

Which makes perfect sense. Statistically speaking, we have much scarier bugs to worry about.

NOTE: This story reveals how easy it can be to identify important factors using the statistical method called Design of Experiments. It won't provide easy answers for making your own office's coffee any better, but it will show you how you can begin identifying the critical factors that contribute to its quality.

At their weekly meeting, her team gave Jill an ultimatum: Make the coffee better.

The office coffee was terrible. Drinking it was like playing a game of chicken with your taste buds. Jill’s practice was to let someone else get the first cup of the day; if gagging and/or swearing soon arose from the tester's cubicle, it was a particularly bad day for the coffee.

There were no good days for the office coffee.

Her team was right, Jill knew. Something had to change. But what was the real problem? Changing the wrong thing could, she thought with a shudder, make the coffee even worse.

She knew that some variables affecting the office coffee were fixed, and would not be easy to change. For example, the company wouldn’t purchase a new coffeemaker as long as the old one still worked, and they certainly wouldn’t buy gourmet coffee for everybody every day.

So Jill made of list of coffee-making variables that were under the team’s control, and came up with:

• Coffee brand
• Days since opening package of beans (0 days to 20 days)
• Type of water (tap or bottled)
• Duration of bean-grinding (1 minute or 4 minutes)
• The amount of ground coffee placed in the filter (1 cup vs. 1.25 cups)
• The number of cups in each pot (10 or 12)

But how could she determine which of these factors played a significant role in the bitterness of the coffee? She knew she needed to perform an experiment, but she wasn’t quite sure how to do it. So she turned to a trusted source of help with data analysis: The Assistant in Minitab 17. 

The Assistant's guidelines for planning and creating experiments helped her quickly determine that she needed to do a screening experiment:

However, having not done anything like this before, Jill wasn't entirely sure about how to proceed. So she clicked the Plan Screening Experiment box for some helpful guidelines:

She’d already identified the factors she needed to study, so she took some time to define the levels of those factors, then selected “Create Screening Design.” She completed the dialog box like this:

When she pressed OK, the Assistant created a datasheet containing her experimental design and even gave her the option to print out a data collection sheet.

It also provided a summary report that explained the experimental goal and the experiment’s strengths and limitations.

The experiment required Jill to make 12 pots of coffee with different combinations of factors.  After running the experiment and collecting the data about the bitterness of each experimental batch of coffee, it was time to analyze the results.

Jill returned to the Assistant, but this time she selected DOE > Analyze and Interpret... instead of Plan and Create...

She clicked “Fit Screening Model” in the Assistant’s decision tree:

And the Assistant analyzed the response data and produced all the output Jill needed to understand the results. The Report Card confirmed the data met statistical assumptions. The Diagnostic Report verified there were no unusual patterns in the results. The Effects Report illustrated the impacts of each factor.

Finally, the Summary Report made it easy to understand exactly which factors had a significant effect on the bitterness of the office coffee.

With the help of the Assistant, Jill now knew that the type of beans used, the number of cups brewed per pot, and the amount of grinding time the coffee beans received had a significant impact on the bitterness of the office coffee.

But just identifying these key factors wouldn’t resolve the team’s complaints. Now Jill needed to figure out how to use those factors to make the coffee more palatable.  Fortunately, the Assistant could help her with that, too...as we'll see in tomorrow's post. 

NOTE: This story will reveal how easy it can be to optimize settings using the statistical method called Design of Experiments, but it won't provide easy answers for making your own office coffee any better.

After her team’s ultimatum about the wretched office coffee, Jill used the design-of-experiments (DOE) tool in Minitab 17’s Assistant to design and analyze a screening study. Jill now knew that three of the factors she screened—the type of beans used, the number of cups brewed per pot, and the amount of grinding time the beans received—had a significant impact on the bitterness of coffee.

Now she needed to use those factors to make the coffee more palatable. 

Once again, she turned to the Assistant’s DOE tool.

This time, she selected “Create Modeling Design” in the Assistant’s decision tree.

She completed the dialog box as shown:

And the Assistant created a Minitab worksheet specifying the run order and variable settings for the experiment, and provided an option to print out sheets for data collection.

The experiment consisted of 16 runs, with each run being a pot of coffee prepared using varying combinations of factors. Jill asked all of the office’s coffee drinkers to sample and rate each brew. She then averaged all of the responses.  

Now it was time to analyze the data. After conducting the experiment and entering the data in the worksheet, Jill returned to the Assistant, this time to analyze the results:

The Assistant presented a decision tree that offered various analysis options. But it very easy for Jill to select and press the “Fit Quadratic Model” button, because The Assistant—which automatically recognized the model structure of the experiment—disabled the buttons for “Fit Linear Model” (which would not be appropriate for this data) and “Add Points for Curvature” (which was unnecessary since the experimental structure already included those points).

Before it could fit the model, the Assistant needed to know the goal: hitting a target, maximizing the response, or—as in this case—minimizing the response. Jill selected the appropriate option and clicked OK.

When Jill clicked OK, the Assistant analyzed the data and produced complete output for understanding the results. The Report Card confirmed the data met statistical assumptions, and the Diagnostic Report verified there were no unusual patterns in the results. The Effects Report illustrated the impacts of each factor, and the Summary Report clearly explained the bottom-line impact of the analysis.

But for Jill’s purposes, the most useful item was the Prediction and Optimization report, which detailed the optimal settings identified in the analysis, and also the top five alternative options, along with their predicted responses.

Based on the experiment’s results, Jill quickly wrote and posted new guidelines for preparing the office coffee. Steps 1 and 2 were:

1. Grind the whole coffee beans only for 1 minute.
2. Brew only 10 cups per pot of coffee.

Today, the office coffee may not rival the taste of that brewed by world-famous baristas, but it’s not so bad. And the cubicles no longer echo with sobbing and gagging sounds. 

Both the brew and Jill’s sales team are a lot less bitter, thanks to an experiment designed and analyzed with the Minitab 17 Assistant. 

Keeping your vehicle fueled up is expensive. Maximizing the miles you get per gallon of fuel saves money and helps the environment, too. 

But knowing if you're getting good mileage requires some data analysis, which gives us a good opportunity to apply one of the common tools used in Six Sigma -- the I-MR (individuals and moving range) control chart to daily life.   

Looking at your vehicle’s MPG data lets you see if your mileage is holding steady, declining, or rising over time. This data can also reveal unusual variation that might indicate a problem you need to fix.

Here's a simulated data set that collects 3 years’ worth of gas mileage records for a car that should get an average of 20 miles per gallon, according to the manufacturer’s estimates. However, the owner didn’t do any vehicle maintenance for the first two years he owned the car. This year, though, he’s diligently performed recommended maintenance.

How does his mileage measure up? And has his attention to maintenance in the past 12 months affected his car’s fuel economy?  Let’s find out with the Assistant in Minitab Statistical Software.

To create the most meaningful chart, we need to recall that a major change in how the vehicle is handled took place during the time the data were collected. The owner bought the car three years ago, but he’s only done the recommended maintenance in the last year.

Since the data were collected both before and after this change, we want to account for it in the analysis.

The easiest way to handle this is to choose Assistant > Before/After Control Charts… to create a chart that makes it easy to see how the change affected both the mean and variance in the process.

If you're following along with Minitab, the Maint column in the worksheet notes which MPG measurements were taken before and after DeWaggen started paying attention to maintenance. Complete the Before/After I-MR Chart dialog box as shown below:

After you press OK, the Assistant produces a Diagnostic Report with detailed information about the analysis, as well as a Report Card, which provides guidance on how to interpret the results and flags potential problems. In this case, there are no concerns with the process mean and variation.

The Assistant's Summary Report gives you the bottom-line results of the analysis.

The Moving Range chart, shown in the lower portion of the graph, illustrates the moving range of the data. It shows that while the upper and lower control limits have shifted, the difference in variation before and after the change is not statistically significant.

However, the car’s mean mileage, which is shown in the Individual Value chart displayed at the top of the graph, has seen a statistically significant change, moving from 19.12 MPG to just under 21 MPG. 

Control charts have been used in statistical process control for decades, and are among the most commonly accessed tools available in statistical software packages. The Assistant has made it particularly easy for anyone to create and see whether or not a process is within control limits, to confirm that observation statistically, and to see whether or not a change in the process results in a change in the process outcome or variation.

As for the data we used in this example, whether or not a 2 mile-per-gallon increase in fuel economy is practically as well as statistically significant could be debated. But since the price of fuel rarely falls, we recommend that the owner of this vehicle continue to keep it tuned up!

Halloween's right around the corner, so here's a scary thought for the statistically minded: That pattern in your time series plot? Maybe it's just a ghost. It might not really be there at all. 

That's right. The trend that seems so evident might be a phantom. Or, if you don't believe in that sort of thing, chalk it up to the brain's desire to impose order on what we see, even when it doesn't exit.  

I'm going to demonstrate this with Minitab Statistical Software (get the free 30-day trial version and play along, if you don't already use it). And if things get scary, just keep telling yourself "It's only a simulation. It's only a simulation."

But remember the ghost pattern when we're done. It's a great reminder of how important it is to make sure that you've interpreted your data properly, and looked at all the factors that might influence your analysis—including the quirks inherent in the statistical methods you used. 

We're going to need some random data, which we can get Minitab to generate for us. In many role-playing games, players use a 20-sided die to determine the outcome of battles with horrible monsters, so in keeping with the Halloween theme we'll simulate 500 consecutive rolls with a 20-sided die. Choose Calc > Random Data > Integer...  and have Minitab generate 500 rows of random integers between 1 and 20.  

Now go to Graph > Time Series Plot... and select the column of random integers. Minitab creates a graph that will look something like this: 

It looks like there could be a pattern, one that looks a little bit like a sine wave...but it's hard to see, since there's a lot of variation in consecutive points. In this situation, many analysts will use a technique called the Moving Average to filter the data. The idea is to smooth out the natural variation in the data by looking at the average of several consecutive data points, thus enabling a pattern to reveal itself. It's the statistical equivalent of applying a noise filter to eliminate hiss on an audio recording.  

A moving average can be calculated based on the average of as few as 2 data points, but this depends on the size and nature of your data set. We're going to calculate the moving average of every 5 numbers. Choose Stat > Time Series > Moving Average... Enter the column of integers as the Variable, and enter 5 as the MA length. Then click "Storage" and have Minitab store the calculated averages in a new data column. 

Now create a new time series plot using the moving averages:

You can see how some of the "noise" from point-to-point variation has been reduced, and it does look like there could, just possibly, be a pattern there.

Of course, a primary reason for doing a time series analysis is to forecast the next item (or several) in the series. Let's see if we might predict the next moving average of the die by knowing the current moving average.  

Select Stat > Time Series > Lag. In the dialog box, choose the "moving averages" column as the series to lag. We'll use this dialog to create a new column of data that places each moving average down 1 row in the column and inserts missing value symbols, *, at the top of the column.

Now we can create a simple scatterplot that will show if there's a correlation between the observed moving average and the next one. 

Clearly, there's a positive correlation between the current moving average and the next, which means we can use the current moving average to predict the next one.  

But wait a minute...this is random data!  By definition, you can't predict random, so how can there be a correlation? This is getting kind of creepy...it's like there's some kind of ghost in this data. 

Zoinks! What would Scooby Doo make of all this?  

Don't panic—there's a perfectly rational explanation for what we're seeing here. It's called the Slutsky-Yule Effect, which simply says an autoregressive time series (like a moving average) can look like patterned data, even if there's no relationship among the data points.  

So there's no ghost in our random data; instead, we're seeing a sort of statistical illusion. Using the moving average can make it seem like a pattern or relationship exists, but that apparent pattern could be a side effect of the tool, and not an indication of a real pattern. 

Does this mean you shouldn't use moving averages to look at your data? No! It's a very valuable and useful technique. However, using it carelessly could get you into trouble. And if you're basing a major decision solely on moving averages, you might want to try some alternate approaches, too. Mikel Harry, one of the originators of Six Sigma, has a great blog post that presents a workplace example of how far apart reality and moving averages can be. 

So just remember the Slutsky-Yule Effect when you're analyzing data in the dead of night, and your moving average chart shows something frightening. Shed some more light on the subject with follow-up analysis and you might find there's nothing to fear at all. 

Here's a shocking finding from the most recent ASQ Global State of Quality report: The higher you rise in your organization's leadership, the less often you receive reports about quality metrics. Only 2% of senior executives get daily quality reports, compared to 33% of front-line staff members.  

A quarter of the senior executives reported getting quality metrics on a monthly basis, at least. But just as many reported getting them only on an annual basis.  

This is simultaneously scary and depressing. It's scary because it indicates that company leaders don't have good access to the kind of information they need about their quality improvement initiatives. More than half of the executives are getting updates about quality only once a quarter, or even less. You can bet they're making decisions that impact quality much more frequently than that. 

It's depressing because quality practitioners are a dedicated, hard-working lot, and their task is both challenging and often thankless. Their efforts don't appear to be reaching the C-level offices as often as they deserve. 

Why do so many leaders get so few reports about their quality programs?

In fairness to everyone involved, from the practitioner to the executive, piecing together the full picture of quality in a company is daunting. Practitioners tell us that even in organizations with robust, mature quality programs, assessing the cumulative impact of an initiative can be difficult, and sometimes impossible. 

The challenges start with the individual project. Teams are very good at capturing and reporting their results, but a large company may have thousands of simultaneous quality projects. Just gathering the critical information from all of those projects and putting it into a form leaders can use is a monumental task. 

But there are other obstacles, too. 

• Teams typically use an array of different applications to create charters, process maps, value stream maps, and other documents. So the project record is a hodgepodge of files for different applications. And since the latest versions of some documents may reside on several different computers, project leaders often need to track multiple versions of a document to keep the official project record current. 
   
• Results and metrics aren’t always measured the same way from one team's project to another. If one team measures apples and the next team measures oranges, their results can't be evaluated or aggregated as if they were equivalent. 
   
• Many organizations have tried quality tracking methods ranging from homegrown project databases to full-featured project portfolio management (PPM) systems. But homegrown systems often become a burden to maintain, while off-the-shelf PPM solutions created for IT or other business functions don’t effectively support projects involving methods like Lean and Six Sigma. 
   
• Reporting on projects can be a burden. There are only so many hours in the day, and busy team members need to prioritize. Copying and pasting information from project documents into an external system seems like non-value-added time, so it's easy to see why putting the latest information into the system gets low priority—if it happens at all.

Given the complexity of the task, and the systemic and human factors involved in improving quality, it's not hard to see why many organizations struggle with knowing how well their initiatives are doing. 

But for quality practitioners and leaders, the challenge is to make sure that reporting on results becomes a critical step in every individual project, and that all projects are using consistent metrics. Teams that can do that will find their results getting more attention and more credit for how they affect the bottom line. 

This finding in the ASQ report caught my attention because it so dramatically underscores problems we at Minitab have been focusing on recently—in fact, last year we released Qeystone, a product portfolio management system for lean six sigma, to address many of these factors. 

Regardless of the tools they use, this issue—how to ensure the results of quality improvement initiatives are understood throughout an organization—is one that every practitioner is likely to grapple with in their career.  

How do you make sure the results of your work reach your organization's decision-makers?   

You need to consider many factors when you’re buying a used car. Once you narrow your choice down to a particular car model, you can get a wealth of information about individual cars on the market through the Internet. How do you navigate through it all to find the best deal?  By analyzing the data you have available.  

Let's look at how this works using the Assistant in Minitab 17. With the Assistant, you can use regression analysis to calculate the expected price of a vehicle based on variables such as year, mileage, whether or not the technology package is included, and whether or not a free Carfax report is included.

And it's probably a lot easier than you think. 

A search of a leading Internet auto sales site yielded data about 988 vehicles of a specific make and model. After putting the data into Minitab, we choose Assistant > Regression…

At this point, if you aren’t very comfortable with regression, the Assistant makes it easy to select the right option for your analysis.

We want to explore the relationships between the price of the vehicle and four factors, or X variables. Since we have more than one X variable, and since we're not looking to optimize a response, we want to choose Multiple Regression.

This data set includes five columns: mileage, the age of the car in years, whether or not it has a technology package, whether or not it includes a free CARFAX report, and, finally, the price of the car.

We don’t know which of these factors may have significant relationship to the cost of the vehicle, and we don’t know whether there are significant two-way interactions between them, or if there are quadratic (nonlinear) terms we should include—but we don’t need to. Just fill out the dialog box as shown. 

Press OK and the Assistant assesses each potential model and selects the best-fitting one. It also provides a comprehensive set of reports, including a Model Building Report that details how the final model was selected and a Report Card that notifies you to potential problems with the analysis, if there are any.

The Summary Report tells us in plain language that there is a significant relationship between the Y and X variables in this analysis, and that the factors in the final model explain 91 percent of the observed variation in price. It confirms that all of the variables we looked at are significant, and that there are significant interactions between them. 

The Model Equations Report contains the final regression models, which can be used to predict the price of a used vehicle. The Assistant provides 2 equations, one for vehicles that include a free CARFAX report, and one for vehicles that do not.

We can tell several interesting things about the price of this vehicle model by reading the equations. First, the average cost for vehicles with a free CARFAX report is about $200 more than the average for vehicles with a paid report ($30,546 vs. $30,354).  This could be because these cars probably have a clean report (if not, the sellers probably wouldn’t provide it for free).

Second, each additional mile added to the car decreases its expected price by roughly 8 cents, while each year added to the cars age decreases the expected price by $2,357.

The technology package adds, on average, $1,105 to the price of vehicles that have a free CARFAX report, but the package adds $2,774 to vehicles with a paid CARFAX report. Perhaps the sellers of these vehicles hope to use the appeal of the technology package to compensate for some other influence on the asking price. 

While these findings are interesting, our goal is to find the car that offers the best value. In other words, we want to find the car that has the largest difference between the asking price and the expected asking price predicted by the regression analysis.

For that, we can look at the Assistant’s Diagnostic Report. The report presents a chart of Residuals vs. Fitted Values.  If we see obvious patterns in this chart, it can indicate problems with the analysis. In that respect, this chart of Residuals vs. Fitted Values looks fine, but now we’re going to use the chart to identify the best value on the market.

In this analysis, the “Fitted Values” are the prices predicted by the regression model. “Residuals” are what you get when you subtract the actual asking price from the predicted asking price—exactly the information you’re looking for! The Assistant marks large residuals in red, making them very easy to find. And three of those residuals—which appear in light blue above because we’ve selected them—appear to be very far below the asking price predicted by the regression analysis.

Selecting these data points on the graph reveals that these are vehicles whose data appears in rows 357, 359, and 934 of the data sheet. Now we can revisit those vehicles online to see if one of them is the right vehicle to purchase, or if there’s something undesirable that explains the low asking price. 

Sure enough, the records for those vehicles reveal that two of them have severe collision damage.

But the remaining vehicle appears to be in pristine condition, and is several thousand dollars less than the price you’d expect to pay, based on this analysis!

With the power of regression analysis and the Assistant, we’ve found a great used car—at a price you know is a real bargain.

A few weeks ago my colleague Cody Steele illustrated how the same set of data can appear to support two contradictory positions. He showed how changing the scale of a graph that displays mean and median household income over time drastically alters the way it can be interpreted, even though there's no change in the data being presented.

When we analyze data, we need to present the results in an objective, honest, and fair way. That's the catch, of course. What's "fair" can be debated...and that leads us straight into "Lies, damned lies, and statistics" territory.  

Cody's post got me thinking about the importance of statistical literacy, especially in a mediascape saturated with overhyped news reports about seemingly every new study, not to mention omnipresent "infographics" of frequently dubious origin and intent.

As consumers and providers of statistics, can we trust our own impressions of the information we're bombarded with on a daily basis? It's an increasing challenge, even for the statistics-savvy. 

The increased amount of information available, combined with the acceleration of the news cycle to speeds that wouldn't have been dreamed of a decade or two ago, means we have less time available to absorb and evaluate individual items critically. 

A half-hour television news broadcast might include several animations, charts, and figures based on the latest research, or polling numbers, or government data. They'll be presented for several seconds at most, then it's on to the next item. 

Getting news online is even more rife with opportunities for split-second judgment calls. We scan through the headlines and eyeball the images, searching for stories interesting enough to click on. But with 25 interesting stories vying for your attention, and perhaps just a few minutes before your next appointment, you race through them very quickly. 

But when we see graphs for a couple of seconds, do we really absorb their meaning completely and accurately? Or are we susceptible to misinterpretation?  

Most of the graphs we see are very simple: bar charts and pie charts predominate. But as statistics educator Dr. Nic points out in this blog post, interpreting even simple bar charts can be a deceptively tricky business. I've adapted her example to demonstrate this below.  

A city surveyed residents of two neighborhoods about the quality of service they get from local government. Respondents were asked to rate local services on a scale of 1 to 10. Their responses were charted using Minitab Statistical Software, as shown below.  

Take a few seconds to scan the charts, then choose which neighborhood's responses exhibit the most variation, Ferndale or Lawnwood?

Seems pretty straightforward, right? Lawnwood's graph is quite spiky and disjointed, with sharp peaks and valleys. The graph of Ferndale's responses, on the other hand, looks nice and even. Each bar's roughly the same height.  

It looks like Lawnwood's responses have the most variation. But let's verify that impression with some basic descriptive statistics about each neighborhood's responses:

Uh-oh. A glance at the graphs suggested that Lawnwood has more variation, but the analysis demonstrates that Ferndale's variation is, in fact, much higher. How did we get this so wrong?  

The answer lies in how the data were presented. The charts above show frequencies, or counts, rather than individual responses.  

What if we graph the individual responses for each neighborhood?  

In these graphs, it's easy to see that the responses of Ferndale's citizens had much more variation than those of Lawnwood. But unless you appreciate the differences between values and frequencies—and paid careful attention to how the first set of graphs was labelled—a quick look at the earlier graphs could well leave you with the wrong conclusion. 

Since you're reading this, you probably both create and consume data analysis. You may generate your own reports and charts at work, and see the results of other peoples' analyses on the news. We should approach both situations with a certain degree of responsibility.  

When looking at graphs and charts produced by others, we need to avoid snap judgments. We need to pay attention to what the graphs really show, and take the time to draw the right conclusions based on how the data are presented.  

When sharing our own analyses, we have a responsibility to communicate clearly. In the frequency charts above, the X and Y axes are labelled adequately—but couldn't they be more explicit?  Instead of just "Rating," couldn't the label read "Count for Each Rating" or some other, more meaningful description? 

Statistical concepts may seem like common knowledge if you've spent a lot of time working with them, but many people aren't clear on ideas like "correlation is not causation" and margins of error, let alone the nuances of statistical assumptions, distributions, and significance levels.

If your audience includes people without a thorough grounding in statistics, are you going the extra mile to make sure the results are understood? For example, many expert statisticians have told us they use the Assistant in Minitab 17 to present their results precisely because it's designed to communicate the outcome of analysis clearly, even for statistical novices. 

If you're already doing everything you can to make statistics accessible to others, kudos to you. And if you're not, why aren't you?  

Isn't it great when you get a set of data and it's perfectly organized and ready for you to analyze? I love it when the people who collect the data take special care to make sure to format it consistently, arrange it correctly, and eliminate the junk, clutter, and useless information I don't need.  

You've never received a data set in such perfect condition, you say?

Yeah, me neither. But I can dream, right? 

The truth is, when other people give me data, it's typically not ready to analyze. It's frequently messy, disorganized, and inconsistent. I get big headaches if I try to analyze it without doing a little clean-up work first. 

I've talked with many people who've shared similar experiences, so I'm writing a series of posts on how to get your data in usable condition. In this first post, I'll talk about some basic methods you can use to make your data easier to work with. 

I'm not complaining about the people who give me data. In most cases, they aren't statisticians and they have many higher priorities than giving me data in exactly the form I want.  

The end result is that getting data is a little bit like getting food: it's not always going to be ready to eat when you pick it up. You don't eat raw chicken, and usually you can't analyze raw data, either.  In both cases, you need to prepare it first or the results aren't going to be pretty.

Here are a couple of very basic things to look for when you get a messy data set, and how to handle them.  

Frequently I get a data set that includes a lot of information that I don't need for my analysis. I also get data sets that combine or group information in ways that make analyzing it more difficult. 

For example, let's say I needed to analyze data about different types of events that take place at a local theater. Here's my raw data sheet:  

With each type of event jammed into a single worksheet, it's a challenge to analyze just one event category. What would work better?  A separate worksheet for each type of occasion. In Minitab Statistical Software, I can go to Data > Split Worksheet... and choose the Event column: 

And Minitab will create new worksheets that include only the data for each type of event. 

Minitab also lets you merge worksheets to combine items provided in separate data files. 

Let's say the data set you've been given contains a lot of columns that you don't need: irrelevant factors, redundant information, and the like. Those items just clutter up your data set, and getting rid of them will make it easier to identify and access the columns of data you actually need. You can delete rows and columns you don't need, or use the Data > Erase Variables tool to make your worksheet more manageable. 

What if you don't want to actually delete any data, but you only want to see the columns you intend to use? For instance, in the data below, I don't need the Date, Manager, or Duration columns now, but I may have use for them in the future: 

I can select and right-click those columns, then use Column > Hide Selected Columns to make them disappear. 

Voila! They're gone from my sight. Note how the displayed columns jump from C1 to C5, indicating that some columns are hidden:  

It's just as easy to bring those columns back in the limelight. When I want them to reappear, I select the C1 and C5 columns, right-click, and choose "Unhide Selected Columns." 

Data may arrive in a disorganized and messy state, but you don't need to keep it that way. Getting rid of extraneous information and choosing the elements that are visible can make your work much easier. But that's just the tip of the iceberg. In my next post, I'll cover some more ways to make unruly data behave.  

In my last post, I wrote about making a cluttered data set easier to work with by removing unneeded columns entirely, and by displaying just those columns you want to work with now. But too much unneeded data isn't always the problem.

What can you do when someone gives you data that isn't organized the way you need it to be?  

That happens for a variety of reasons, but most often it's because the simplest way for people to collect data is with a format that might make it difficult to assess in a worksheet. Most statistical software will accept a wide range of data layouts, but just because a layout is readable doesn't mean it will be easy to analyze.

You may not be in control of how your data were collected, but you can use tools like sorting, stacking, and ordering to put your data into a format that makes sense and is easy for you to use. 

Depending on how its arranged, the same data can be easier to work with, simpler to understand, and can even yield deeper and more sophisticated insights. I can't tell you the best way to organize your specific data set, because that will depend on the types of analysis you want to perform, and the nature of the data you're working with. However, I can show you some easy ways to rearrange your data into the form that you select. 

The data below show concession sales for different types of events held at a local theater. 

If we wanted to perform an analysis that requires each type of event to be in its own column, we can choose Data > Unstack Columns... and complete the dialog box as shown:

Minitab creates a new worksheet that contains a separate column of Concessions sales data for each type of event:

A similar tool will help you put data from separate columns into a single column for the type of analysis required. The data below show sales figures for four employees: 

Select Data > Stack > Columns... and select the columns you wish to combine. Checking the "Use variable names in subscript column" will create a second column that identifies the person who made each sale. 

When you press OK, the sales data are stacked into a single column of measurements and ready for analysis, with Employee available as a grouping variable: 

The following data appear in the worksheet in the order in which individual stores in a chain sent them into the central accounting system.

When the data appear in this uncontrolled order, finding an observation for any particular item, or from any specific store, would entail reviewing the entire list. We can fix that problem by selecting Data > Sort... and reordering the data by either store or item. 

What if you need to analyze information about the same items, but that were recorded on separate worksheets?  For instance, if one group was gathering historic data about all of a corporation's manufacturing operations, while another was working on strategic planning, and your analysis required data from each? 

You can use Data > Merge Worksheets to bring the data together into a single worksheet, using the Division column to match the observations:

You can also choose whether or not multiple, missing, or unmatched observations will be included in the merged worksheet.  

Making changes to the layout of your worksheet does entail a small investment of time, but it can bring big returns in making analyses quicker and easier to perform. The next time you're confronted with raw data that isn't ready to play nice, try some of these approaches to get it under control. 

In my next post, I'll share some tips and tricks that can help you get more information out of your data.

Everyone who analyzes data regularly has the experience of getting a worksheet that just isn't ready to use. Previously I wrote about tools you can use to clean up and elminate clutter in your data and reorganize your data. 

In this post, I'm going to highlight tools that help you get the most out of messy data by altering its characteristics.

Many problems with data don't become obvious until you begin to analyze it. A shortcut or abbreviation that seemed to make sense while the data was being collected, for instance, might turn out to be a time-waster in the end. What if abbreviated values in the data set only make sense to the person who collected it? Or a column of numeric data accidentally gets coded as text?  You can solve those problems quickly with statistical software packages.

Here's an instance where a data entry error resulted in a column of numbers being incorrectly classified as text data. This will severely limit the types of analysis that can be performed using the data.

To fix this, select Data > Change Data Type and use the dialog box to choose the column you want to change.

One click later, and the errant text data has been converted to the desired numeric format:

When this company collected data on the performance of its different functions across all its locations, it used numbers to represent both locations and units. 

That may have been a convenient way to record the data, but unless you've memorized what each set of numbers stands for, interpreting the results of your analysis will be a confusing chore. You can make the results easy to understand and communicating by coding the data. 

In this case, we select Data > Code > Numeric to Text...

And we complete the dialog box as follows, telling the software to replace the numbers with more meaningful information, like the town each facility is located in.  

Now you have data columns that can be understood by anyone. When you create graphs and figures, they will be clearly labelled.  

Dates and times can be very important in looking at performance data and other indicators that might have a cyclical or time-sensitive effect.  But the way the date is recorded in your data sheet might not be exactly what you need. 

For example, if you wanted to see if the day of the week had an influence on the activities in certain divisions of your company, a list of dates in the MM/DD/YYYY format won't be very helpful.   

You can use Data > Date/Time > Extract to Text... to identify the day of the week for each date.

Now you have a column that lists the day of the week, and you can easily use it in your analysis. 

These tools are commonly seen as a way to correct data-entry errors, but as we've seen, you can use them to make your data sets more meaningful and easier to work with.

There are many other tools available in Minitab's Data menu, including an array of options for arranging, combining, dividing, fine-tuning, rounding, and otherwise massaging your data to make it easier to use. Next time you've got a column of data that isn't quite what you need, try using the Data menu to get it into shape.

"Data! Data! Data! I can't make bricks without clay."
 — Sherlock Holmes, in Arthur Conan Doyle's The Adventure of the Copper Beeches

Whether you're the world's greatest detective trying to crack a case or a person trying to solve a problem at work, you're going to need information. Facts. Data, as Sherlock Holmes says. 

But not all data is created equal, especially if you plan to analyze as part of a quality improvement project.

If you're using Minitab Statistical Software, you can access the Assistant to guide you through your analysis step-by-step, and help identify the type of data you have.

But it's still important to have at least a basic understanding of the different types of data, and the kinds of questions you can use them to answer. 

In this post, I'll provide a basic overview of the types of data you're likely to encounter, and we'll use a box of my favorite candy—Jujubes—to illustrate how we can gather these different kinds of data, and what types of analysis we might use it for. 

At the highest level, two kinds of data exist: quantitative and qualitative.

Quantitativedata deals with numbers and things you can measure objectively: dimensions such as height, width, and length. Temperature and humidity. Prices. Area and volume.

Qualitative data deals with characteristics and descriptors that can't be easily measured, but can be observed subjectively—such as smells, tastes, textures, attractiveness, and color. 

Broadly speaking, when you measure something and give it a number value, you create quantitative data. When you classify or judge something, you create qualitative data. So far, so good. But this is just the highest level of data: there are also different types of quantitative and qualitative data.

There are two types of quantitative data, which is also referred to as numeric data: continuous and discrete. As a general rule, counts are discrete and measurements are continuous.

Discrete data is a count that can't be made more precise. Typically it involves integers. For instance, the number of children (or adults, or pets) in your family is discrete data, because you are counting whole, indivisible entities: you can't have 2.5 kids, or 1.3 pets.

Continuousdata, on the other hand, could be divided and reduced to finer and finer levels. For example, you can measure the height of your kids at progressively more precise scales—meters, centimeters, millimeters, and beyond—so height is continuous data.

If I tally the number of individual Jujubes in a box, that number is a piece of discrete data.

If I use a scale to measure the weight of each Jujube, or the weight of the entire box, that's continuous data. 

Continuous data can be used in many different kinds of hypothesis tests. For example, to assess the accuracy of the weight printed on the Jujubes box, we could measure 30 boxes and perform a 1-sample t-test. 

Some analyses use continuous and discrete quantitative data at the same time. For instance, we could perform a regression analysis to see if the weight of Jujube boxes (continuous data) is correlated with the number of Jujubes inside (discrete data). 

When you classify or categorize something, you create Qualitative or attribute data. There are three main kinds of qualitative data.

Binary data place things in one of two mutually exclusive categories: right/wrong, true/false, or accept/reject. 

Occasionally, I'll get a box of Jujubes that contains a couple of individual pieces that are either too hard or too dry. If I went through the box and classified each piece as "Good" or "Bad," that would be binary data. I could use this kind of data to develop a statistical model to predict how frequently I can expect to get a bad Jujube.

When collecting unordered or nominal data, we assign individual items to named categories that do not have an implicit or natural value or rank. If I went through a box of Jujubes and recorded the color of each in my worksheet, that would be nominal data. 

This kind of data can be used in many different ways—for instance, I could use chi-square anlaysis to see if there are statistically significant differences in the amounts of each color in a box. 

We also can have ordered or ordinal data, in which items are assigned to categories that do have some kind of implicit or natural order, such as "Short, Medium, or Tall."  Another example is a survey question that asks us to rate an item on a 1 to 10 scale, with 10 being the best. This implies that 10 is better than 9, which is better than 8, and so on. 

The uses for ordered data is a matter of some debate among statisticians. Everyone agrees its appropriate for creating bar charts, but beyond that the answer to the question "What should I do with my ordinal data?" is "It depends."  Here's a post from another blog that offers an excellent summary of the considerations involved. 

For more fun statistics you can do with candy, check out this article (PDF format): Statistical Concepts: What M&M's Can Teach Us. 

For a deeper exploration of the probability distributions that apply to different types of data, check out my colleague Jim Frost's posts about understanding and using discrete distributions and how to identify the distribution of your data.

One of my favorite bloggers about the application of statistics in health care is David Kashmer, an MD and MBA who runs and writes for the Business Model Innovation in Surgery blog. If you have an interest in how quality improvement methods like Lean and Six Sigma can be applied to healthcare, check it out. 

A while back, Dr. Kashmer penned a column called "How to Measure a Process When There's No Metric," in which he discusses how you can use the measurement systems analysis method called Gage R&R (or gauge R&R) to create your own measurement tools and validate them as useful metrics. (I select the term “useful” here deliberately: a metric you’ve devised could be very useful in helping you assess your situation, but might not meet requirements set by agencies, auditors, or other concerned parties.) 

I thought I would use this post to show you how you can use the Assistant in Minitab Statistical Software to do this.

Kashmer posits a scenario in which state regulators assert that your health system's ability to oversee residents is poor, but your team believes residents are well supervised. You want to assess the situation with data, but you lack an established way to measure the quality of resident supervision. What to do?

Kashmer says, "You decide to design a tool for your organization. You pull a sample of charts and look for commonalities that seem to display excellent supervision versus poor supervision."

So you work with your team to come up with a tool that uses a 0 to 10 scale to rate resident supervision, based on various factors appearing on a chart. But how do you know if the tool will actually help you assess the quality of resident supervision?  

This is where gage R&R comes in. The gage refers to the tool or instrument you're testing, and the R&R stands for reproducibility and repeatability. The analysis will tell you whether different people who use your tool to assess resident supervision (the gauge) will reach the same conclusion (reproducibility) and do it consistently (repeatability). 

We're going to use the Assistant in Minitab Statistical Software to help us. If you're not already using it, you can download a 30-day trial version for free so you can follow along.  Start by selecting Assistant > Measurement Systems Analysis... from the menu: 

Follow the decision tree...

If you're not sure about what you need to do in a gage R&R, clicking the more... link gives you requirements, assumptions, and guidelines to follow: 

After a look at the requirements, you decide you will have three evaluators use your new tool to assess each of 20 charts 3 times, and so you complete the dialog box thus: 

When you press "OK," the Assistant asks if you'd like to print worksheets you can use to easily gather your data:

Minitab also creates a datasheet for the analysis. All you need to do is enter the data you collect in the "Measurements" column:

Note that the Assistant automatically randomizes the order in which each evaluator will examine the charts in each of their three judging sessions. 

Now we're ready to gather the data to verify the effectiveness of our new metric for assessing the quality of patient supervision. Come back for Part 2, where we'll analyze the collected data!  

In my previous post, I showed you how to set up data collection for a gage R&R analysis using the Assistant in Minitab 17. In this case, the goal of the gage R&R study is to test whether a new tool provides an effective metric for assessing resident supervision in a medical facility.  

As noted in that post, I'm drawing on one of my favorite bloggers about health care quality, David Kashmer of the Business Model Innovation in Surgery blog, and specifically his column "How to Measure a Process When There's No Metric." 

In one scenario Kashmer presents, state regulators and hospital staff disagree about a health system's ability to oversee residents. In the absence of an established way to measure resident supervision, the staff devises a tool that uses a 0 to 10 scale to rate resident supervision. 

Now we're going to analyze the Gage R&R data to test how effectively and reliably the new tool measures what we want it to measure. The analysis will evaluate whether different people who use the tool (the gauge) reach the same conclusion (reproducibility) and do it consistently (repeatability).   

To get data, three evaluators used the tool to assess each of 20 charts three times each, and recorded their score for each chart in the worksheet we produced earlier. (You can download the completed worksheet here if you're following along in Minitab.)   

Now we're ready to analyze the data. 

Once again, we can turn to the Assistant in Minitab Statistical Software to help us. If you're not already using it, your can download a 30-day trial version for free so you can follow along. Start by selecting Assistant > Measurement Systems Analysis... from the menu: 

In my earlier post, we used the Assistant to set up this study and make it easy to collect the data we need. Now that we've gathered the data, we can follow the Assistant's decision tree to the "Analyze Data" option.  

Selecting the right items for the Assistant's Gage R&R dialog box couldn't be easier—when you use the datasheet the Assistant generated, just enter "Operators" for Operators, "Parts" for Parts, and "Score" for Measurements.   

Before we press OK, though, we need to tell the Assistant how to estimate process variation. When Gage R&R is performed in a manufacturing context, historic data about the amount of variation in the output of the process being studied is usually available. Since this is the first time we're analyzing the performance of the new tool for measuring the quality of resident supervision, we don't have an historical standard deviation, so we will tell the Assistant to estimate the variation from the data we're analyzing. 

The Assistant also asks for an upper or lower specification limit, or tolerance width, which is the distance from the upper spec limit to the lower spec limit. Minitab uses this to calculate %Tolerance, an optional statistic used to determine whether the measurement system can adequately sort good from bad parts—or in this case, good from bad supervision. For the sake of this example, let's say in designing the instrument you have selected a level of 5.0 as the minimum acceptable score.

When we press OK, the Assistant analyzes the data and presents a Summary Report, a Variation Report, and a Report Card for its analysis. The Summary Report gives us the bottom line about how well the new measurement system works.  

The first item we see is a bar graph that answers the question, "Can you adequately assess process performance?" The Assistant's analysis of the data tells us that the system we're using to measure patient supervision can indeed assess the resident supervision process. 

The second bar graph answers the question "Can you sort good parts from bad?" In this case, we're evaluating patient supervision rather than parts, but the Analysis shows that the system is able to distinguish charts that indicate acceptable resident supervision from those that do not. 

For both of these charts, less than 10% of the observed variation in the data could be attributed to the measurement system itself—a very good result.

I can't count the number of times I've heard people say that they can't gather or analyze data about a situation because "it can't be measured." In most cases, that's just not true. Where a factor of interest—"service quality," say—is tough to measure directly, we can usually find measurable indicator variables that can at least give us some insight into our performance. 

I hope this example, though simplified from what you're likely to encounter in the real world, shows how it's possible to demonstrate the effectiveness of a measurement system when one doesn't already exist.  Even for outcomes that seem hard to quantify, we can create measurement systems to give us valuable data, which we can then use to make improvements.  

What kinds of outcomes would you like to be able to measure in your profession?  Could you use Gage R&R or another form of measurement system analysis to get started?  

A while back, I offered an overview of process capability analysis that emphasized the importance of matching your analysis to the distribution of your data.

If you're already familiar with different types of distributions, Minitab makes it easy to identify what type of data you're working with, or to transform your data to approximate the normal distribution.

But what if you're not so great with probability distributions, or you're not sure about how or even if you should transform your data? You can still do capability analysis with the Assistant in Minitab Statistical Software. Even if you're a stats whiz, the Assistant's easy-to-follow output can make the task of explaining your results much easier to people who don't share your expertise. 

Let's walk through an example of capability analysis with non-normal data, using the Assistant.

For this example, we'll use a data set that's included with Minitab Statistical Software. (If you're not already using Minitab, download the free trial and follow along.) Click File > Open Worksheet, and then click the button labeled "Look in Minitab Sample Data folder." Open the dataset named Tiles. 

This is data from a manufacturer of floor tiles. The company is concerned about the flexibility of the tiles, and the data set contains data collected on 10 tiles produced on each of 10 consecutive working days.

Select Assistant > Capability Analysis in Minitab:

The Assistant presents you with simple decision tree that will guide you to the right kind of capability analysis:

The first decision we need to make is what type of data we've collected—Continuous or Attribute. If you're not sure what the difference is, you can just click the "Data Type" diamond to see a straightforward explanation.

Attribute data involves counts and characteristics, while Continuous data involves measurements of factors such as height, length, weight, and so on, so it's pretty easy to recognize that the measurements of tile flexibility are continuous data. With that question settled, the Assistant leads us to the "Capability Analysis" button:

Clicking that button brings up the dialog shown below. Our data are all in the "Warping" column of the worksheet.  The subgroup size is "10", since we measured 10 samples on each day. Enter "8" as the upper spec limit, because that's the customer's guideline.

Then press OK.

Uh-oh—the Assistant immediately gives us a warning. Our data don't meet the assumption of normality:

When you click "Yes," the Assistant will transform the data automatically (using the Box-Cox transformation) and continue the analysis. Once the analysis is complete, you'll get a Report Card that alerts you if there are potential issues with your analysis, a Diagnostic Report that assesses the stability of your process and the normality of your data, a detailed Process Performance Report, and a Summary Report that captures the bottom line results of your analysis and presents them in plain language.

The Ppk of .75 is below the typical industry acceptability benchmark of 1.33, so this process is not capable. Looks like we have some opportunities to improve the quality of our process!

Once we've made adjustments to the process, we can also use the Assistant to see how much of an impact those changes have had. The Assistant's Before/After Capability Analysis is just what we need:

The dialog box for this analysis is very similar to that for the first capability analysis we performed, but this time we can select a column of data from before we made improvements (Baseline process data), and a column of data collected after our improvements were implemented: 

Press OK and the Assistant will again check if you want to transform your data for normality before it proceeds with the analysis. Then it presents us with a series of reports that make it easy to see the impact of our changes. The summary report gives you the bottom line quickly. 

The changes did affect the process variability, and this process now has a Ppk of 1.94, a vast improvement over the original value of .75, and well above the 1.33 benchmark for acceptability.  

I hope this post helps you see how the Assistant can make performing capability analyses easier, and that you'll be able to get more value from your process data as a result. 

This week I'm at the American Society for Quality's World Conference on Quality and Improvement in Nashville, TN. The ASQ conference is a great opportunity to see how quality professionals are tackling problems in every industry, from beverage distribution to banking services. 

Given my statistical bent, I like to see how companies apply tools like ANOVA, regression, and especially designed experiments—particularly if they happen to be using the statistical software I like best. 

One of the most popular sessions involved a company whose products are instantly recognizable to almost everyone who's ever had (or been) a child: Crayola.

There's something about using crayons that brings out the imaginative kid in all of us, and as this session started I saw lots of smiles and even overheard some wistful recollections about "new crayon smell" from the row behind me.  

I also heard comments about the quality of Crayola's crayons compared to other brands, and I flashed back to my own childhood experiences: other crayons' tips weren't as strong, and if you pressed really hard they were much more prone to snapping in two. But Crayolas were always the best—if you wanted to break a Crayola, you needed to work at it!  

The conference room was packed with people who'd had similar experiences. We'd seen the results of Crayola's efforts to make the highest-quality crayons available, and now we wanted to learn more about how they did it.  We weren't disappointed. 

Speaking for Crayola were Bonnie Hall, the company's vice president for global quality and continuous improvement, and Rich Titus, a Lean Six Sigma Master Black Belt who has consulted with Crayola for several years. 

They talked about the history of Crayola, and the fact that they've been making crayons for more than 100 years from their headquarters and facilities in Bethlehem, Pennsylvania. They also shared a brief history of the company's Lean and Six Sigma initiatives, which kicked off in 2001 and have yielded great benefits. 

Then they walked participants through several examples of how Crayola has used data analysis and statistics to reduce waste, cut costs, and most important, maximize the quality of the crayons, markers, modeling materials and other art supplies they make. 

They talked about how Crayola followed a systematic process to improve the accuracy of its inventory system, following the DMAIC roadmap and applying tools like graphical analysis, ANOVA, Design of Experiments, and capability analysis. 

It's pretty cool to see and hear how statistics helps Crayola make sure they're delivering items that kids can count on, and it's really gratifying to know they trust Minitab's software to make the data analysis as easy and straightforward as possible. 

In their presentation today, Bonnie Hall and Rich Titus cited a couple of key attributes that they believe have made Crayola's quality program successful. 

• Minitab and Data Driven Problem Solving now the Norm at Crayola
• Lean and Six Sigma now part of Crayola Culture
• Senior-level managers are fully trained green and black belts, and do their own projects
• Executives and managers conduct regular project reviews for current projects

Crayola's quality improvement efforts have been a tremendous success, and the companies leaders were gracious enough to spend time with me and some other Minitab folks earlier this year to tell us more about about how data analysis has helped them compete and improve.

You can visit our web site to learn more about how Crayola is using statistics and data to maintain and enhance the quality of their products. 

All processes have some variation. Some variation is natural and nothing to be concerned about. But in other cases, there is unusual variation that may need attention. 

By graphing process data against an upper and a lower control limit, control charts help us distinguish natural variation from special cause variation that we need to be concerned about. If a data point falls outside the limits on the control chart, the process is out of control, and you need to investigate.

Simple, right? 

But most people who’ve worked extensively with control charts, particularly attributes charts, can tell you about “false alarms” where the control limits fell close to the mean and data points exceeded the limits even though the process was in statistical control.

The problem of false alarms on traditional P and U control charts due to overdispersion has been known for decades, but a good solution didn’t exist until quality engineer David Laney devised P' and U' charts. 

Using P' and U' control charts enables you to avoid these false alarms so only important deviations in your process are detected. Like traditional P and U charts, we use P' and U' charts to monitor defectives and defects, respectively. But where traditional charts assume your defective or defect rate remains constant over time, P' and U' charts assume that no process exhibits a truly constant rate, and factors that into its control limits.

As a result, these charts can give you a more reliable indication of whether your process is truly in or out of control.

Minitab makes it easy to create P' and U' charts, and includes a diagnostic tool that tells you when you need to use them, so you can be confident in detecting special-cause variation only when it truly exists.

False alarms can be caused by too much variation in process data, or “overdispersion,” especially if you collected your data using large subgroups. The larger your subgroups, the narrower your control limits will be on a traditional P or U chart. This situation creates artificially tight control limits, causing points on a traditional P chart to look like they’re out of control even when they are not.

Alternatively, too little variation, or “underdispersion,” can also be problematic. When underdispersion exists, control limits on a traditional P chart or U chart may be too wide, so data points (or “processes”) you should be concerned about appear to be in control.

Whether your data exhibit overdispersion or underdispersion, a P' or U' chart will help you distinguish true common cause versus special cause variation. 

Use the P Chart or U Chart Diagnostic in Minitab to test your process data for overdispersion and underdispersion, and to see whether a traditional P or U chart or a Laney P' or U' chart is more appropriate.

To run a diagnostic on your data, choose Stat > Control Charts > Attributes Charts > P Chart Diagnostic or Stat > Control Charts > Attributes Charts > U Chart Diagnostic. 

You’ll see the following dialog box:

In Variables, enter the worksheet column that contains the number of defectives. If you collected all of your samples using the same subgroup size, enter that number in Subgroup sizes. Or if your subgroup sizes varied, use a column.

Let’s run this diagnostic test on the DefectiveRecords.MTW sample data included in Minitab. The subgroups in this data set are large, with an average of about 2,500 observations in each.

The P chart diagnostic gives the following output:

The diagnostic gives you the ratio of observed to expected variation. If the ratio is greater than the 95% upper limit, your data exhibit significant overdispersion. If the ratio is less than 60%, your data exhibit significant underdispersion. In either case, you should consider using a Laney P' chart instead of a traditional P chart.

The test found overdispersion in our sample data and recommends that we use a Laney P' chart instead.

To create a P' chart, choose Stat > Control Charts >Attributes Charts > Laney P'.  The P' chart for our sample data shows that the process appears to be stable—no points are out of control: 

When we use the same data to create a traditional P chart, overdispersion results in narrower control limits, and several of the subgroups appear to be out of control.  

So why do the data points seem to be out of control on the P chart, but not the P' chart?  It’s due to the way each chart defines and calculates the variation in your process. The limits calculated in the Laney P' account for the overdispersion when calculating the variation and eliminate these false alarms.

The calculations for the Laney P' chart include not only within-subgroup variation, but also the variation between subgroups to adjust for overdispersion or underdispersion.

If there’s no problem with over- or underdispersion, the P' chart is comparable to a traditional P chart.  But when overdispersion exists, the P' chart expands the control limits so only important deviations are identified as out of control. If underdispersion exists, the P' chart calculates narrower control limits.

Quality practitioners across all industries can take advantage of Laney's powerful charts and the easy-to-use diagnostic tool and be confident in detecting special-cause variation only when it truly exists.

Read On the Charts: A Conversation with David Laney to learn more about the statistical foundation behind them.

Earlier, I wrote about the different types of data statisticians typically encounter. In this post, we're going to look at why, when given a choice in the matter, we prefer to analyze continuous data rather than categorical/attribute or discrete data. 

As a reminder, when we assign something to a group or give it a name, we have created attribute or categorical data.  If we count something, like defects, we have gathered discrete data. And if we can measure something to a (theoretically) infinite degree, we have continuous data.

Or, to put in bullet points: 

• Categorical = naming or grouping data
• Discrete = count data
• Continuous = measurement data

A statistical software package like Minitab is extremely powerful and can tell us many valuable things—as long as we're able to feed it good numbers. Without numbers, we have no analyses nor graphs. Even categorical or attribute data needs to be converted into numeric form by counting before we can analyze it. 

At this point, you may be thinking, "Wait a minute—we can't really measure anything infinitely,so isn't measurement data actually discrete, too?" That's a fair question. 

If you're a strict literalist, the answer is "yes"—when we measure a property that's continuous, like height or distance, we are de facto making a discrete assessment. When we collect a lot of those discrete measurements, it's the amount of detail they contain that will dictate whether we can treat the collection as discrete or continuous.

I like to think of it as a question of scale. Say I want to measure the weight of 16-ounce cereal boxes coming off a production line, and I want to be sure that the weight of each box is at least 16 ounces, but no more than 1/2 ounce over that.

With a scale calibrated to whole pounds, all I can do is put every box into one of three categories: less than a pound, 1 pound, or more than a pound. 

With a scale that can distinguish ounces, I will be able to measure with a bit more accuracy just how close to a pound the individual boxes are. I'm getting nearer to continuous data, but there are still only 16 degrees between each pound. 

But if I measure with a scale capable of distinguishing 1/1000th of an ounce, I will have quite a wide scale—a continuum—of potential values between pounds. The individual boxes could have any value between 0.000 and 1.999 pounds. The scale of these measurements is fine enough to be analyzed with powerful statistical tools made for continuous data.   

Not all data points are equally valuable, and you can glean a lot more insight from 100 points of continuous data than you can from 100 points of attribute or count data. How does this finer degree of detail affect what we can learn from a set of data? It's easy to see. 

Let's start with the simplest kind of data, attribute data that rates a the weight of a cereal box as good or bad. For 100 boxes of cereal, any that are under 1 pound are classified as bad, so each box can have one of only two values.

We can create a bar chart or a pie chart to visualize this data, and that's about it:

If we bump up the precision of our scale to differentiate between boxes that are over and under 1 pound, we can put each box of cereal into one of three categories. Here's what that looks like in a pie chart:

This gives us a little bit more insight—we now see that we are overfilling more boxes than we are underfilling—but there is still a very limited amount of information we can extract from the data.  

If we measure each box to the nearest ounce, we open the door to using methods for continuous data, and get a still better picture of what's going on. We can see that, on average, the boxes weigh 1 pound. But there's high variability, with a standard deviation of 0.9. There's also a wide range in our data, with observed values from 12 to 20 ounces: 

If I measure the boxes with a scale capable of differentiating thousandths of an ounce, more options for analysis open up. For example, now that the data are fine enough to distinguish half-ounces (and then some), I can perform a capability analysis to see if my process is even capable of consistently delivering boxes that fall between 16 and 16.5 ounces. I'll use the Assistant in Minitab to do it, selecting Assistant > Capability Analysis: 

The analysis has revealed that my process isn't capable of meeting specifications. Looks like I have some work to do...but the Assistant also gives me an I-MR control chart, which reveals where and when my process is going out of spec, so I can start looking for root causes.

If I were only looking at attribute data, I might think my process was just fine. Continuous data has allowed me to see that I can make the process better, and given me a rough idea where to start. By making changes and collecting additional continuous data, I'll be able to conduct hypothesis tests, analyze sources of variances, and more.  

Does this mean discrete data is no good at all?  Of course not—we are concerned with many things that can't be measured effectively except through discrete data, such as opinions and demographics. But when you can get it, continuous data is the better option. The table below lays out the reasons why. 

Continuous Data

Discrete Data

I hope this very basic overview has effectively illustrated why you should opt for continuous data over discrete data whenever you can get it.