Balancing Data and Intuition for Better Business Decisions
Imagine you are a captain of a ship. You have a destination in mind, but you don’t know exactly how to get there. You have a map, a compass, a sextant, and a telescope. You also have a crew, a cargo, and a budget. How do you use all these resources to navigate the seas and reach your goal? How do you deal with the uncertainties, the risks, and the opportunities that you encounter along the way? How do you balance your intuition and your data to make the best decisions for your ship? This is the challenge that many business leaders face today. Data has been referred to as the oil that drives the engine of society, the impetus that pushes us forward, the stuff that decisions are made of. But data alone is not enough. You also need to have a vision, a strategy, a plan, and a process. You also need to have a sense of innovation, creativity, and intuition. In this article, we will explore how you can use data analytics to inform your decisions, and how you can use your intuition and acumen to guide your data analysis. We will also look at some of the tools and techniques that can help you become more data-driven in your business.
While data is essential for driving our decisions, it is not sufficient by itself. People have great business acumen, instincts, yearnings, and understanding of their business cycle. But sometimes, they forget that these qualities need to be complemented by data. They need to use the data available to them to inform their decisions, and to illuminate the insights they already have. It takes a combination of both intuition AND data.
When people think of data, they often imagine rows and columns of numbers. But data is much more than that. Data is an asset or a commodity that can be used for various purposes. Data also includes anecdotal information, such as stories that can describe what’s happening in different areas of the business, such as production, sales, manufacturing, employee engagement, customer engagement, vendor engagement, and more. By recognizing this, people can understand that data encompasses all the information that decision-makers need to identify and execute the steps they must take, the issues they must address, the problems they must solve, and the direction they must follow.
All of this is driven by data, even if they don’t realize it. All these different pieces of information are data as well. Data is not limited to rows and columns on a spreadsheet or database, but rather, it is information that you can act on. Techopedia defines actionable insight as information that can be acted upon or information that gives enough insight into the future so that the actions that should be taken become clear to decision-makers. Decision-makers can act on it and will have enough reason or insight to move forward and make those decisions. To be actionable, they must achieve one of three goals: make a decision, answer a question, or solve a problem, by using everything at their disposal to make that happen.
But how do you get actionable insights from data? How do you know what data to collect, analyze, and act upon? How do you make sense of the vast and complex data that is available to you? This is where data analytics comes in. Data analytics is the process of transforming data into meaningful information that can help you make better decisions, answer questions, or solve problems. There are four main types of data analytics that you can use to get actionable insights from data: descriptive, diagnostic, predictive, and prescriptive. Let’s take a look at each one and see how they can help you achieve your goals.
- Descriptive analytics tells you the story of what has happened in your business. It uses data visualization and reporting to reveal the patterns and trends that shaped your past performance. For instance, you can use descriptive analytics to see how much revenue, sales, and customer satisfaction you achieved in a year.
- Diagnostic analytics helps you understand why something happened in your business. It uses data mining and root cause analysis to identify the factors and reasons behind your results. For example, you can use diagnostic analytics to discover why your customers are leaving, why your churn rate is high, and why your product is underperforming.
- Predictive analytics gives you a glimpse of what might happen in your business. It uses statistical modeling and machine learning to make forecasts based on your past data. For example, you can use predictive analytics to estimate how much demand, revenue, and profit you can expect from a new product launch.
- Prescriptive analytics guides you on what to do next in your business. It uses optimization and simulation to suggest the best decisions for your goals and constraints. For example, you can use prescriptive analytics to find the optimal price, inventory, and marketing strategy for your product.
To illustrate this process, imagine you are rowing a boat (a smaller version of the ship I mentioned in the opening to this article). You have a destination in mind, but you can’t see it because your back is facing the direction you are moving. Instead, you can only see where you have been. This is similar to how data is collected and stored. You are constantly looking at the data that reflects your past actions and results. It may seem paradoxical: how can you move forward by looking backward? But as you row, you adjust your direction based on the feedback you get from the water and the wind. Likewise, decision-makers can use data to find patterns and trends that inform their future choices.
However, a predictable pattern is not always a reliable indicator of what will happen next. Sometimes, you need to be able to break away from the routine and try something new. This is where the danger of over-analysis lies. If you spend too much time looking at the data and trying to find the perfect solution, you may miss the opportunity to act and innovate. This is what some people call “analysis paralysis”. To avoid this trap, you need to balance data analysis with creativity and intuition. This is how you can overcome the procrastination that often stems from over-analysis.
But innovation is not just about data. It is also about intuition and business acumen. You may have a hunch. You may have spotted a pattern. You may have a vision of where things will go if you take a certain action based on your experience, intuition, and instincts. That is something that cold-hard numbers cannot give you. That is why we humans are the ones who make the final decisions. The artificial intelligence and machine learning engines (AI/ML) are far superior to us humans in performing the same calculations over and over with flawless accuracy, blazing speed, and consistent reliability.
However, this advantage only applies to a limited extent: when you are dealing with “yes” and “no,” left or right, ones and zeros, negative and positive, all those finite, “digital” type choices. But when it comes to any non-digital choices, i.e., those gray areas, those nuanced areas, those are up to us humans to make. We do this by using the information that comes our way and relying upon our previous experience and our alignment with both existing business processes and our goals. Then it becomes much easier to navigate that course, as it were, returning to the nautical metaphor of rowing the boat.
To succeed in this decision-making realm, you need to balance the strategic with the tactical. You need to have a “big-picture” mindset, knowing where you want to steer the boat. How do you do that? By using all the data, you have from various sources, such as the information that is given to you, the information that you collect from different business areas, and the information that you sense from the pulse of the business. That gives you the overall high-level strategic view.
But you also need to pay attention to what is happening on the ground. You need to keep track of the day-to-day affairs of the business, such as the daily decisions, the challenges and solutions, and the incremental improvements in your processes. These are the tactical, short-term operational things that will make your long-range vision a reality.
As a leader, you must have a clear vision for your business; that is, your mission, vision, and goals. That is the strategic part. And within that vision, you must monitor the different lines that are moving towards you; that is, the different tactical operational processes in your business. And (all at the same time) you must ensure that those daily operations are aligned with your long-term goals. So, keep your eye on the horizon, the strategic, but also keep your hand on the wheel, the tactical.
By balancing the strategic and the tactical, you can make better decisions for your business. You can use data to inform your intuition and acumen, and you can use your intuition and acumen to guide your data analysis. You can use data to innovate and create value for your customers and stakeholders. This is what it means to be data-driven.
And while doing so, as for the people who are doing their jobs, let them do your jobs while you manage the overall framework and keep everything moving forward. Let me share an allegory to help you understand the importance of the strategic view. Remember, when you use stories and allegories, you are using tools that help shape people’s thinking. I have a passion for this, as I was an educator for ten years before I got into IT. Some people think that vision and the future have to be detailed before they can move forward. But in reality, it’s more like a painting. You don’t need to see every single detail to start painting. You just need a rough sketch of the general direction you want to go, with some broad goals and some intermediate steps to get there. You won’t know exactly what the final picture will look like. You have an idea or a concept that you need to flesh out as you go. But you don’t sit back and do nothing for fear of not having enough details. That would paralyze you and stifle your innovation, as previously mentioned. People who demand too much precision in their forward vision will stagnate. I believe that if you’re not innovating, you’re stagnating. You should at least make a move and act on what you already know.
For example, you might be looking at a line graph that shows your progress. You prefer the line to go up than down, so you’re fine as long as it’s going up. You might say, “Okay, we’re moving in the right direction. Let’s keep doing what we’re doing, but also let’s keep improving our processes. Let’s look for ways to do better tomorrow than what we did today and yesterday.” You proceed in this manner one day at a time, one step at a time, one procedure at a time, one process at a time. And as you meet these incremental goals, you’ll find that you are making improvement. As time goes on, the long-range picture will become clearer. Then you can focus your efforts on reaching the next incremental goal, while keeping the overall vision of the long-range picture in mind. You may not know all the details of what the future will look like in 5, 10, or 15 years, but with this approach, you can at least know where you’re going to be tomorrow. Then you keep progressing towards that long-range goal, even if it’s still vague and nebulous, but constantly adjusting. And sometimes you might need to change course. Unexpected things happen. You might need to rethink your strategy. You might need to reposition yourself. But even then, you still have that overall goal of improvement, achievement, and reward for a job well done. And as time goes on, the picture will become clearer and clearer. You may still have things that are far ahead that are not clear yet. But that’s okay. You may only be able to see a few days ahead. It doesn’t matter. Just keep moving in the direction and do the next right thing. And as more detail emerges, keep moving in the right direction and keep progressing towards your overall goal. Eventually, things will come into better and clearer focus.
Another dimension of this approach is the role of risk and AI/ML. How willing are you to take risks? Can AI/ML engines be risk-takers as well? Some people are more cautious than others. How do you know where you are on that spectrum? What are the factors, the consequences, the costs and benefits, the risks and rewards, the challenges and opportunities? These varied factors are weighed in your own framework of the decision-making process, based on your moral values, your sense of right and wrong, and your sense of what is acceptable and what is not for a risk. This is one of those “gray areas” that is hard to capture precisely in an algorithm that an artificial intelligence engine can follow.
However, AI/ML engines are good at dealing with situations that are clear-cut, made of ones and zeros, and can be resolved as either this option or that one. Even if you program many contingencies into the AI/ML engine of your choice, at the end of the day, it will boil down to a choice between yes or no, one or zero, left or right, white or black, right or wrong. There is no middle ground in the digital world.
Going back to the risk factor, you might decide to take the plunge. When you do, you might find that the risk is not worth it, so you ask yourself questions like: What is the exit strategy? What can I do to avoid disaster? How can I minimize the losses? You took a gamble and lost, then you start asking yourself: What are the alternatives? What steps can I take to mitigate the losses or to recover, reserve, respond, and redirect? How can I turn the lemons into lemonade? How can I make the best of this bad situation? Sometimes you are going to make wrong decisions. Sometimes you are going to crash and burn. If and when that happens, you get up and try something else, then you see what you can do to get out, redirect, refocus, and try to move forward despite what has happened. It is obvious that such a scenario as I have described CANNOT be handled effectively and successfully by any artificial intelligence available today. In their current form, AI simply does not have the ability to make that level of a value judgment. And that’s because artificial intelligence depends on its programming in a similar way that we humans depend on the mental framework of biases and experiences formed over a lifetime. This is how we judge a situation, which direction we are going to take, and how we are going to deal with the mess we have created and see how to get out of it. This is why I see “risk aversion quotient” as more of a human factor than a machine factor.
In this article, we have explored how you can use data analytics to inform your decisions, and how you can use your intuition and acumen to guide your data analysis. We have also looked at some of the tools and techniques that can help you become more data-driven in your business. We have seen how data is not just rows and columns of numbers, but also stories and insights that can help you achieve your goals. We have also seen how intuition and acumen are not just gut feelings, but also skills and knowledge that can help you innovate and create value. We have also seen how risk and AI/ML are factors that can influence your decision-making process, and how you need to balance the strategic and the tactical aspects of your business. By combining data and intuition, you can make better decisions for your business, and navigate the uncertainties and opportunities that you encounter along the way. You can be the captain of your ship, and steer it towards your destination, using data as your oil, and intuition as your compass. Wishing you the best for success in your journey!
“Bringing Data to Life and Life to Data”
About the Author:
Team Lead / Senior Systems Analyst,
NC Department of Health and Human Services
Dr. Joe Perez ( Dr.Joe ) is also the Chief Technology Officer – SolonTek
Dr. Joe Perez is a truly exceptional professional who has left an indelible mark on the IT, health and human services, and higher education sectors. His journey began in the field of education, where he laid the foundation for his career. With advanced degrees in education and a doctorate that included a double minor in computers and theology, Joe embarked on a path that ultimately led him to the dynamic world of data-driven Information Technology.
In the early 1990s, he transitioned into IT, starting as a Computer Consultant at NC State University. Over the years, his dedication and expertise led to a series of well-deserved promotions, culminating in his role as Business Intelligence Specialist that capped his 25 successful years at NC State. Not one to rest on his laurels, Dr. Perez embarked on a new challenge in the fall of 2017, when he was recruited to take on the role of Senior Business Analyst at the NC Department of Health & Human Services (DHHS). His impressive journey continued with promotions to Senior Systems Analyst and Team Leader, showcasing his versatility and leadership capabilities.
In addition to his full-time responsibilities at DHHS, Joe assumed the role of fractional Chief Technology Officer at a North Carolina corporation in October 2020. A top-ranked published author with over 16,000 followers on LinkedIn and numerous professional certifications, he is a highly sought-after international keynote speaker, a recognized expert in data analytics and visualization, and a specialist in efficiency and process improvement.
Dr. Perez’s contributions have not gone unnoticed. He is a recipient of the IOT Industry Insights 2021 Thought Leader of the Year award and has been acknowledged as a LinkedIn Top Voice in multiple topics. He holds memberships in prestigious Thought Leader communities at Gartner, Coruzant Technologies, DataManagementU, Engatica, the Global AI Hub, and Thinkers360 (where he achieved overall Top 20 Thought Leader 2023 ranking in both Analytics and Big Data). His reach extends to more than twenty countries worldwide, where he impacts thousands through his speaking engagements.
Beyond his professional achievements, Joe’s passion for teaching remains undiminished. Whether as a speaker, workshop facilitator, podcast guest, conference emcee, or team leader, he continually inspires individuals to strive for excellence. He treasures his time with his family and is a gifted musician, singer, pianist, and composer. Joe also dedicates his skills as a speaker, interpreter, and music director to his church’s Hispanic ministry. He manages the publication of a widely recognized monthly military newsletter, The Patriot News, and is deeply committed to his community.
To maintain a balanced life, Perez is a regular at the gym, and he finds relaxation in watching Star Trek reruns. He lives by the philosophy that innovation is the key to progress, and he approaches each day with boundless energy and an unwavering commitment to excellence. His journey is a testament to the remarkable achievements of a truly exceptional individual.
Dr.Joe Perez is Accorded with the following Honors & Awards :
https://www.linkedin.com/in/jw
Dr. Joe Perez is Bestowed with the following Licences,Certifications & Badge:
https://www.linkedin.com/in/jw
https://www.thinkers360.com/tl
Dr.Joe Perez is Voluentering in the following International Industry Associations & Institutions :
https://www.linkedin.com/in/jw
Dr.Joe Perez can be contacted at :
E-mail | LinkedIn | Web | Sessionize | FaceBook | Twitter | YouTube
