Category Archives for "Data Analysis"

Dec 09

How to Make Better Decisions with Data

By ganpati | Data Analysis


Now a days, most analyst/managerial/sales jobs require data analysis skills. These basic skills are – ability to collect and format data, familiarity with charts and graphs, calculating the performance metrics (like revenue, sales, spending, incidents, risks etc.), highlight data that meets certain conditions, creating reports and forecasting certain figures based on past trend. Though Excel has so far been the favorite, new age tools are gaining momentum fast and people who are able to switch to a more sophisticated tool will be having an extra edge in their career. Based on my years of experience in data analysis with multiple tools following 7 areas form the core of Data Analysis:

Getting Started

Collecting and Formatting Data

Data collection is one of the most crucial steps and if done properly it may save a lot of hassles at the later part of data analysis. Many a times your choices are limited as you are provided with the data that has already been stored in company data bases. In that case you can take advantage of thousands of secondary data sources and API’s provided by a number of organizations to augment your judgement. If you restrict your analysis only to the data provided to you, it may restrict your analysis as well as the outcomes.

Visualizing Data with Graphs and Charts

A picture is worth a thousand words. But in case of data, a picture or visualization can be worth more than a million data points. A well plotted graph or chart can captures the essence of a whole data data base or hundreds of thousands of rows. So, whenever you are starting to analyze a table loaded with data points, you should try some of the plots to understand what’s lying inside it. It’ll sure give you a proper direction in which to drive your analysis. I’m currently working on a complete tutorial that will depict the jouney with data where I’ll create a flowchart on types of problems and what charts to be drawn to for analyzing them. Once you know the type of problem and the data that’s available, you’ll know which graphs to plot and what to look for in those graphs.

Presenting Scenarios with Data

Data helps us to present alternative blue prints of future. There can be multiple possibilities in predictive analysis, with a certain probability associated with them. If you can present all those possibilities and summarize the actions that ca be taken under each of those situations and its budgetary impact, it will add a lot more value to the analysis.

Making Effective Forecasts

Creating Quick Reports

Advanced Analytics

Dec 09

First Step in Data Analytics: Defining the Business Objective

By Ani Rud | Data Analysis

Each industry has its own set of problems and CXO’s look for specific solutions in analytics. That’s the reason analytics is becoming popular every day, and every company is trying to hire the best talent from the market. The reason there is a scarcity of analytics talent is because, until recently, there were not enough trainings available in this field that could provide all the necessary skills. But before telling you about the necessary skills, I’ll fast forward a little and tell you about the problems that you need to solve as an analytics professional. Mind you, sometimes the problem statements aren’t that simple and it takes few days to weeks to clearly define the problem statements. But to start with I’ll demonstrate a few simpler ones.

The steps in problem solving process using analytics are listed here. Some analytics processes may be a subset of these steps but I’ve explained the whole journey so that you know where you’re going with this. And you’ll know when to jump a step, with experience!
The checklist for the steps in analytics are:

1. Defining the goals
2. Explore the data
3. Analyze the data
4. Test alternatives
5. Find the best solution- Optimize
6. Implement the decision
7. Monitor results from the decision and define new goals if necessary

The first step in the business analytics process

If you are preparing for an interview this is a very important step to understand in order to make a first impression. Because the first question you’re asked about a business case is to define the goal. Though in some cases you’ll be given the goal and even then you have to analyze that goal to arrive at a problem statement for using analytics. Remember, you should take a pause before stating the goal. Because this will make or break the complete solution you will create for the case. Without goal you can’t give a proper solution.

In this post we will primarily focus on the first step that involves understanding the problem. As Einstein famously said, “if I had an hour to solve a problem I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions”.

Just think for a while – what the business would like to improve on or the problem it wants solved?

If the goal is too big to analyze at one go, break it into smaller parts. Here’s an example how.

Not long back a friend of mine wanted to try his luck in the restaurant business. Coming from an engineering background he knew very little about the industry. All he knew about the industry was from his own visits to more than a hundred restaurants in the city and the opinions of his friends. But what he did next was quite amazing. And that’s why I’ve decided to use his business as a case study to explain analytics in few of the early posts.

His goal was simple- to open a successful restaurant in Bangalore.

But wait. What’s ‘successful’ here? The first step is to define the business objectives clearly. According to him the success he was looking for in terms of business are below: (aside from other parameters like financial freedom, happiness, job satisfaction etc.)
-Select the best location for the restaurant
– Create a menu with most popular items
-A lot of word of mouth publicity for his restaurant
-A lot of fan following in social media
-A handsome revenue from restaurant operations
-Repeat customers
-Substantial revenue from online ordering

Now, as he knows what he really wants to achieve in his business, he wanted to apply some analytics techniques he had used in his previous job to this. And for that he has to define a set of analytics goals. But analytics goals are different from the qualitative goals that he has stated above. Analytics goals are measurable. And in order to apply analytics we have to break down the above goals into measurable goals. Here’s how we do it:

• Go through each business objective and, if necessary, rephrase it using terms that are quantifiable.
• Once you have a list of measurable items, think which ones can be measured directly
• The ones that are complex to measure should be broken down into simple measurable goals.
• Prepare a final list of measurable analytics goals

If you look at the goals carefully you’ll find there is always something to be measured and some action specific to the measurement. So let’s first find out what’s measurable in each of the above goals-

-Best location- a best location fro a restaurant can be any area where lot of people visit during the day or night. We can further break it down in terms of what age group we are looking for?, what income group we are targeting, what type of restaurants are already present in the vicinity?, can we access the various residential area easily for online ordering? Etc. All these questions are ultimately answers one questions- which location will maximize visitors to our restaurant?
Based on above let’s say we will identify the best location as the one that will attract a footfall of 2000 customers per month in our restaurant.

-Create a suitable menu- A suitable menu consists of items that will be liked by most of the visitors. It shouldn’t be too long so that it burdens the restaurant’s inventory and it shouldn’t be too short so that visitors don’t find a dish he wants to have. The restaurant should also create a positioning about the kind of dishes it offers best. It may be fast food, Chinese, ethnic Indian dishes, European cuisine etc. But at the same time it should offer some of the dishes that are commonly ordered by customers across all restaurants like rice, paratha etc. Once all these are take into account the menu has to be revisited over time so as to find out customer likes and dislikes and additions and alterations can be made on basis of feedback. But for the time being it’s best to find the suitable food based on demography, target customers, availability, expertise etc. that will attract maximum customers. A measurable goal in this this regard can be answer to the customer feedback question “Do you like the dishes offered in our menu?” and “Would you like more items to be added to the list?” The second question can be asked only for initial two months after which the restaurant can establish its positioning and run analytics on items ordered and answer to fist question to modify the menu. Else it will find it too difficult to include all the items to cater to everyone’s taste. An effective analytics goal in this case will be a 90% positive feedback on the menu after initial experimentation phase of 2 months is over.

-word of mouth publicity- first we have to analyze the number of customers who has ‘liked’ the pages of similar restaurants in the city. Based on that we can set a specific number say 10,000 likes that we want to achieve within a year. Also, the number of customers who come through a referral are good way of knowing the word of mouth. Let’s say we keep a target referral of 2000 customers for the first year.

-A lot of fan following in social media- This again can be measured from customer review forms and social media pages. If the customer replies to the question ‘How you came to know about us?” as social media or internet it means social media presence is significant. Here we will keep the same target as 10,000 likes but we will revise the target based on customer feedback that we receive.

-A handsome revenue from restaurant operations- We can keep the revenue target as 1 crore for the first year based on 30% discount over similar restaurants in the area.

-Repeat customers- we want 20% of the customers to visit again within a period of 6 months. This can be tracked using customer feedback form. “Have you visited within past 6 months? If no, will you visit us again within next 6 months?”

-Substantial revenue from online ordering- We want at least a revenue of 30 lacs from online ordering- again based on similar restaurants in the area.

So our final analytical goals will look something like below:

 A location that will attract 3000 customers every month on average
 90% positive feedback on the menu items
 1 crore of total sales
 30 lacs of online ordering over internet and phone
 10,000 likes in social media pages
 20% repeat customers in any six months period
 2000 customers visiting through referrals

This concludes the identification of goals for the purpose of solving analytics problem. In many real world scenarios the goals are much more complicated and assumptions are much more difficult to incorporate. We will how how to deal such situations in our advanced discussions. In the next post we will see how we can gather data from various sources to explore and analyze, so that proper decisions can be taken to achieve the above goals.

Sep 12

How Websites Source Their Data?

By ganpati | Data Analysis

Not just big corporations, even individuals these days are using a lot of analytics and the demand will keep increasing in the years to come. And to satiate this demand for quality predictions many websites have come up with innovative services that can help users with a lot of decisions- right from betting to weather forecasts to investments. Some help predicting real life events (success of movie or victory of a political candidate) by measuring popular perception using bets placed by the masses on politics, movies and sports. But ever wondered where from these sites collect so much data in order to aid your bets or provide you with information on almost anything under the sun? Here are few ways in which the popular websites build their databases:

1. In-house Content Team: Many websites have in-house content teams who do the digging. They read contents online, watch videos, scan through images and even go through offline journals, documents and other materials to get the gist of those materials or publish reviews based on the business model of the website.

2. Community Members:– A lot of websites get their data sourced from their community members. Many websites that use reviews, polls and bidding from users collects a lot of perception related information which they also use for predictions etc.

3. Parse Others Websites:– Some websites parse otehr websites and collect data. Some public websites are easily accessible and it’s just a matter of employing the right code to collect all the data that users have posted on those site.

4. API’s: On the Web, APIs make it possible for big services like Google Maps or Facebook to let other apps “piggyback” on their offerings. Think about the way airbnb, for instance, displays nearby accommodations on a Google Map in its app, or the way some video games now let players chat, post high scores and invite friends to play via Facebook, right there in the middle of a game.

5. Third Party Providers: These are companies such as KBM, Acxiom and Equifax that collect consumer data from various sources. These companies collaborate with other companies to collect consumer data and build consumer databases to sell.