Data, Data and More Data – How to Make it Work for You!
by Melanie Askew, CPA, Consulting Manager
Posted on February 11, 2021
In the past, decision-making was much simpler, but we often got it wrong because although we knew a lot about one or two options, we didn’t know what our other choices were or we made poor choices anyway. A historic example of this was Christopher Columbus’ landing in the East Indies. This was actually due in large part to bad data that resulted in miscalculations when charting the distance between Europe and Asia. He used a less accurate geographer, and assumed the miles being referenced were Roman when they were actually Arabic (which are longer) resulting in his assumption that he made it to Asia when he was quite a bit further away.
Now, decision-making is much more complicated since we have the internet, multiple software applications and many different sources of data. We often still get it wrong because we have too many choices that we know just a little bit about and we sometimes still use bad data. In 1999, NASA lost the Mars Orbiter. Part of the problem that led to its loss was very similar to Columbus. The engineering team developing the Orbiter used English units of measurement while NASA used the metric system resulting in the loss of the $125-million craft.
How can we make better choices? Additional data may not be the answer, but better data and analysis will always be the answer. There are a few steps to ensure you are getting the best out of your data. First, you must know what kind of data you are working with and what kind of biases might influence how you review the data. Second, you need to determine what you really want to get out of the data. What is the result you are looking for? You will need ask the right questions and pull the data out into useful information. Finally, you want to ensure the base data used is accurate.
There are three main kinds of data we often act on: salient data, which captures our attention because it is noteworthy or surprising; contextual data, which has a frame that may impact how we interpret it; and patterned data, which appears to have a regular, intelligible, and meaningful form. For example, if we are trying to determine our budget for the next year, the different data sets might include: last night’s news story about the increase in parents concerned their children are not getting enough recess time, a web article we read this morning about an increase by 52% in the number of students being homeschooled, and our financial records from the last three years.
Different kinds of data trigger different biases; consequently, identifying the data type and its related bias makes it easier to avoid bad choices based on that data.
Salient data, such as the news about parents concerned about recess can activate salience bias. This is a result of weighing new information or newsworthy information higher than it should be. In our example, we assume there will be much fewer students next year because of one news article, when the reality is, the story does not really predict the actual change in students in our community.
Contextual data can lead to a framing bias. How we hear about the data can make something seem much worse or better than it is. In our example, is it an increase of 52% of students already being homeschooled or are 52% of all students changing to being homeschooled? This could mean 5 students, or 500 students depending on the context.
Patterned data often prompts clustering illusion or bias where we assume that random events are information that help us predict a future event. We are wired to look for patterns, even when they do not exist. Or, if patterns do exist, they don’t necessarily predict the future. Our financial data from the last three years may show that we served 1,200 lunches each week on average for the last three years. This does not mean that we will serve exactly 1,200 lunches this week or next week.
It is important to realize how we might make mistakes in evaluating data based on these biases or assumptions so we can avoid common errors in judgement.
The next step is to realize that you don’t need to know everything, but you do need to identify what matters most to your decision-making. If we are creating an annual budget, do we need to know that parents in some community in another state are concerned about recess? Do we need to know exactly how many lunches may be served in one week or just on average for the year? Do we need to estimate how many students might be enrolled next year?
We do not need every piece of available data in the world to make a decision; we can make decisions based on only the information most relevant to us. It is more important to ask the right questions to get the right data. This can be very difficult to achieve. Many of us struggle with this because we don’t know what we don’t know.
One way to help move forward is to organize questions into basic categories. For example, actions, perceptions, and knowledge. This can help get better context for interpreting answers and help remove the mental clutter.
Action questions would include gaining information on what someone does or has done and will generally give results of actual experiences, activities, and actions. Looking at your community and seeing how many parents are actively homeschooling their children would be an example of this. Asking how many meals were served in October versus March can also provide this type of information.
Perception questions are more about individual emotional response. For something such as a budget, these may sound less useful. However, they may help you dive into the whys behind things that can be addressed when discussing the budget. Are parents feeling nervous about sending kids to your schools? What can be done to address these concerns? If they want more recess time, can that be addressed by increasing the number of student aids hired to oversee recess time or by increasing physical education? All of these may become factors in developing a budget. On the other hand, maybe once you dive into the details, it turns out that only two households in your community are concerned about recess, and it may be better not to adjust your budget for this.
Now that we have narrowed down what data we need to make our decisions, we also need to look at how that data was obtained and whether it is accurate. Going on with our examples, where did the news program get its data about parents being concerned about recess? Did it come from a survey or a study? Did someone review this study? If it was a survey, was it a survey of 10 parents, 1,000 parents, or was it based on one or two people complaining to their friend who was a reporter? Depending on the answers to these questions, we might completely ignore this information or at least not rely completely on it. What about the increase in homeschooled children? We can ask all the same questions about where this information came from.
The third item seems the most reliable at first. Although our financial system states we served 1,200 lunches per month, were those meals actually served or is that the number lunches we made? Are we sure the staff person counting the meals made an accurate count? When we pulled the information from the software, did we pull the correct information or did we accidently include breakfasts? These are items we have more control over. We can ensure our staff and that we ourselves are properly trained and that we have internal controls in place to help ensure our data is correct. Annual audits can help ensure information is materially correct and that our internal controls appear to be working appropriately.
The keys to good decision making begin and end with good, reliable data, clean and complete audits, strong internal controls and taking a little more time ensuring that we evaluate the right data at the right time.