Data Collection for Analytics

Blogs

Sign-up to receive the latest articles related to the area of business excellence.

Data Collection for Analytics

Author: Palak Kumar

Data is one of the most valuable resources for businesses today. Data analysis helps businesses understand their customers, their interests, wants, needs, and also the changing market conditions so that businesses can take proactive steps to stay ahead of the competition. Your company may already have access to tons of data. But the question is, is that data sufficient, what data should we collect and how do we go about collecting this data?

Data Collection requires a certain level of planning and effort to collect the data. It may not be possible to get all the data that we want on a project due to time and effort required. Hence, we have to be efficient and effective in data collection. We need to create a data collection plan that answers specific questions that can add value to our business. A properly planned approach helps to avoid collecting unnecessary data and wasting resources.

What data to collect?

Before going into the data collection process, it is important to identify the most important problems in the business and look for data to address these questions or gaps. Let us look at the types of data we may want to collect on three types of projects:

DMAIC Six Sigma: On a DMAIC Six Sigma project, we usually have a process that we want to improve, and each project may have one or more primary and secondary metrics. This could be a good place to start to plan for your data collection. You can collect this data to establish a baseline and also demonstrate improvement on your project. However, just this data is not sufficient to work on your project. You would also need data to diagnose the problems and understand the root causes. If you have solutions that you want to deploy on your project, you will need data to validate and prove that your improvements actually are working. In addition, you may want to collect data to validate your financial benefits on a project.

Design for Six Sigma: On a Design for Six Sigma project, you may want to collect data to establish the voice of the customer. You will need data to understand the customer needs, and their CTQs. As you design your new process, you will need to model your process and as such will need data to understand the sources of variation and their impact on your critical metrics so that you can optimize the design to create a robust performance.

Lean: On a Lean project, you typically need data on your key performance indicators to understand if they are able to meet the standards, getting better or getting worse. If a particular metric is getting worse, you will probably need to use a tree diagram to understand where the problem is coming from and you will need data to create and analyse the Pareto charts. If you are working on a root cause analysis, say using 5 Why technique, you will need data to validate each why so that your analysis is fact based and not biased by team’s opinions.

Hence, the data that you collect must answer the specific requirements and must be feasible to collect.

How much data to collect?

We also need to know how much data we need to see the patterns and trends. Too little data might not show us the correct patterns and the data may not be representative of what is happening in the real-world. Collecting too much data is also a waste of time and resources especially if you are not going to be using the data for any improvements for business purposes. When you collect data to make decisions, you can make two types of mistakes. The first is to observe patterns in the sample data that do not exist in the real-world and the second is to miss patterns in the real data based on the sample. Ideally, we do not want to make either of these errors, hence selecting a right representative sample for the population data is very important. There are a number of tools available for determining how much data to collect and we will cover this aspect in a different article.

Collecting the Data

After choosing the type and amount of data you want to collect you have to determine a data collection method. Let us take an example of customer data. We are currently selling a product to our customers, but our sales have not been increasing in the recent years. We want to collect data to understand customer perception and thoughts on our existing product line and actions we can take to increase sales. There are different methods to collect this data and the right method to use depends on your goals, type of data you are collecting, and the time and effort required to collect the data. Following are some data collection methods you can use for this case:

Surveys: In a survey, you directly ask the customers for the information you are looking for. You ask them a set of questions which they can respond to in the form on checkboxes, radio buttons, free-form text fields etc. Surveys can be conducted online, offline in person, via emails, or over the phone.
Online Tracking: A business website is an excellent tool to collect customer data. When a person visits your website, they leave a trail of data that you can mine. For example, you can determine how much time people spend on your website, what information they are looking for in their searches, where they click and what they read. You can use tools such as Google analytics for this purpose.
Transactional Data: When you sell goods online or offline the transactional data can give valuable insights about the customers. For example, the demographics information about your customers, the types of products sold, customer returns information. This information can be used to retarget your customers and make strategic decisions.
Marketing Campaigns: Marketing campaigns can give valuable insights into customers. These insights tell us who clicked on our ads, at what times, what were the devices they used, and more. This information can be used to assess the efficiency of current campaigns and plan future ones.
Social Media: You can look for customer data on social media platforms. You can go through their follower lists and observe the common characteristics and understand your target audience. You can also keep track of your competitors and see what they are doing to attract their audience. You can use analytics provided by the social media platforms and using certain third-party tools you can get some deeper insights.
Subscription Data: Subscriptions and registrations are also an excellent way to gather data about your customers. You can allow them to sign up for a subscription of your newsletters or register for your events and in return, you can get valuable data. Search customers are more likely to convert as they have shown considerable interest in your brand by interacting. You can use the contact information to poll them and send them information related to what they are looking for or may be interested in.

In summary, data is very important to make fact-based decisions. Data can be used to establish a baseline for your key performance indicators, quantify the problem, understand root causes, validate improvements, and quantify benefits. Before you collect data, make sure you are clear on what are the business questions you want to answer. Make sure you collect enough sample data that is representative of the population to avoid making type 1 and type 2 errors in decision making. Finally, you need to use creative methods to collect the data that you are interested in as there can be a variety of sources of data that you can tap into. Data from different sources can provide different perspectives of the same problem and help generate a holistic view of the business problem at hand.

Blogs