5 Questions to Ask About Your Big Data

In statistics, a hypothesis is proposed and then data samples are collected to prove or disprove the hypothesis with acceptable confidence levels. For example, let’s say that all our customers are aware of all our product lines. Basically, there are two ways of assessing our hypothesis that includes: (1) Proving our hypothesis and (2) Disproving our hypothesis.

The first way to proving our hypothesis is that we communicate with all of our customers and inquire if they know all our product lines. The second way is to communicate with as many customers as possible until we come across any customer that does not know all our product lines. From this example, we can see that if we find even one customer then that disproves our hypothesis. Thus, this is the reason why in statistics, sometimes it is easier to find an exception to disproving a hypothesis rather than proving it.

Big Data, on the other hand, inverts the generally acceptable process from hypothesis then data sample collection to data collection then a hypothesis. What this means is that Big Data emphasizes collecting data first and then coming up with a hypothesis based on patterns found in the data. Generally speaking, when we talk about Big Data, we are concerned with the 3 Vs that include:

  • Volume – Amount of data
  • Velocity – Rate of data analysis
  • Variety – Different data sources

Some have indicated that we need to go beyond just the above three Vs and should also include:

  • Viscosity – Resistance to the flow of data
  • Variability – Changes in the flow changes of data
  • Veracity – Outlier data
  • Volatility – Validity of the data
  • Virality – Speed at which data is shared

I would take the Big Data concept a bit further and introduce:

  • Vitality – General and specific importance of the data itself
  • Versatility – Applicability of data to various situations
  • Vocality – Supporters of data-driven approaches
  • Veto – The ultimate authority to accept or reject Big Data conclusions

For a metrics-driven organization, a possible way to determine the effectiveness of your Big Data initiatives is to do a weighted rating of the Vs based on your organizational priorities. These organizational priorities can range from but not limited to increasing employee retention rates, improving customer experiences, improving mergers and acquisitions activities, making better investment decisions, effectively managing the organization, increasing market share, improving citizens services, faster software development, improving the design, becoming more innovative and improving lives. What all of this means is that data is not just data but it is, in fact, an organization’s most important asset after its people. Since data is now a competitive asset, let’s explore some of the ways we can use it:

  • Monte Carlo Simulations – Determine a range of scenarios of outcomes and their probabilities.
  • Analysis of Variance (ANOVA) – Determine if our results change when we change the data
  • Regression – Determine if data is related and can be used for forecasting
  • Seasonality – Determine if data shows the same thing occurring at the same intervals
  • Optimization – Getting the best possible answer from the data
  • Satisficing – Getting a good enough answer from the data

Now that we understand what is Big Data and how it can be used, let’s ask the following questions:



Who is capturing data?Who should be capturing data?
What is the lifecycle of your data?What should be the lifecycle of your data?
Where is data being captured?Where should data be captured?
When is data available for analysis?When should data be available for analysis?
Why data is being analyzed?Why data should be analyzed?

Having discussed the positives of Big Data, we have to realize that it is not a panacea and has its negatives as well. Some of the negative ways data can lead to bad decisions include: (1) Data is correlated but that does not imply cause and effect, (2) Data shows you pretty pictures but that does not imply it is telling you the truth and (3) Biases can affect data anywhere from capturing to analysis to decision-making.

In conclusion, what this means is that the non-distorted quality, understanding, and usage of data is the difference between just getting on the Big Data bandwagon or truly understanding how data can fundamentally change your organization.

Big Data Vs


  1. Realizing the Promise of Big Data
  2. Beyond the three Vs of Big Data
  3. 5 Factors for Business Transformation
  4. 5 Questions to Ask About Your Business Processes
  5. 5 Questions to Ask About Your Information
  6. 5 Questions to Ask About Customer Experiences
  7. 5 Observations on Being Innovative (at an organizational level)
  8. Where is my Big Data coming from and who can handle it

Success! You're on the list.

5 Questions to Ask About Gamification

The term gamification refers to “the use of game design elements in non-game contexts.” (Deterding et al.) The non-game contexts imply that gamification is different than games and can be applied to society, business, technology, and individuals at various levels. Gartner goes a step further and defines gamification to be “the use of game mechanics and experience design to digitally engage and motivate people to achieve their goals.” Essentially what this means is that gamification is used to change the norms, attitudes, and habits of individuals and organizations from a current state to a desired future state typically through the utilization of technology. Generally speaking, the use of gamification in the organization can be categorized into external uses (e.g., customer engagement) and internal uses (e.g., employee engagement).

In order for organizations to effectively leverage gamification as a game-changer, they have to ask the following questions:



Who is using gamification externally and internally?Who should be using gamification externally and internally?
What is gamified?What should be gamified?
Where it is being used?Where it should be used?
When are gamified types of activities are happening?When should gamified types of activities be happening?
Why it is becoming a competitive advantage?Why you should be using it as a competitive advantage?

When you are asking the above questions across all levels of the organization, here are few things to keep in mind (1) have clearly defined goals for the players/users and the organization, (2) blindly applying gamification without thinking through organizational repercussions can be costly, (3) measure progress, get feedback and iterate, (4) create value since it is a not a one-way street but a multi-way street and (5) balance between intrinsic considerations and extrinsic rewards.

Here organizations have a choice about gamification as a (1) passing fad or (2) as a strategic lever that can help them transform. So, the real question about using gamification becomes, “Can you afford not to do it?”

Gamify SPICE
Gamify SPICE


  1. Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness: defining “gamification”. In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek ’11). ACM, New York, NY, USA, 9-15. DOI=10.1145/2181037.2181040 http://doi.acm.org/10.1145/2181037.2181040
  2. “Gamification – Gartner IT Glossary.” Gartner IT Glossary. Gartner, n.d. Web. http://www.gartner.com/it-glossary/gamification-2/
  3. Werbach, Kevin. “Coursera – Gamification.” Coursera. Coursera, n.d. Web. https://www.coursera.org/course/gamification
  4. Krogue, Kevin. “5 Gamification Rules From The Grandfather Of Gamification.” Forbes. Forbes Magazine, n.d. Web. http://www.forbes.com/sites/kenkrogue/2012/09/18/5-gamification-rules-from-the-grandfather-of-gamification/
  5. Stanley, Robert. “Top 25 Best Examples of Gamification in Business.” Clickipedia. Clickipedia, 24 Mar. 2014. Web. http://blogs.clicksoftware.com/clickipedia/top-25-best-examples-of-gamification-in-business/
  6. Kleinberg, Adam. “Brands That Failed with Gamification.” – IMediaConnection.com. – IMediaConnection.com, 23 July 2012. Web. http://www.imediaconnection.com/content/32280.asp

Success! You're on the list.

5 Questions to Ask About Internet of Things (IoT)

According to Gartner, “Internet of Things (IoT) is the network of physical objects that contain embedded technologies to communicate and sense or interact with their internal states or the external environment.” The communication of these physical devices with itself and/or with its environment (e.g., other physical devices, information systems, etc.) generates tremendous amounts of data. Depending upon the end goal, this data can be used anywhere from determining the foot traffic in retail stores to monitoring environmental effects on trees. Since there are so many different uses of the data generated from these IoT devices thus it is difficult to determine how many of these devices would be used in the future. However, Gartner has taken a stab at this and estimates that within 6 years (by 2020) there would be about 26 billion active IoT devices in use. With so many devices in use within a short period, it would be naïve of organizations to think that these IoT devices would have no effect on existing business models and operations.

In order to understand the challenges and opportunities, the following questions need to be asked about your current and future uses of IoT devices:




Who uses them? Who should use them?
What business processes use them? What business processes would use them?
Where is the data being captured? Where should the data be captured?
When are they used? When would they be used?
Why do they affect the bottom line? Why would they affect the bottom line?

When you are asking the above questions, keep in mind that organizations who know how to increase the bottom line through the effective use of technology would get into your space even before you think they are your competition. This means that as an organization, you have a choice of either ignoring the IoT revolution or getting ahead by fully immersing yourself into how IoT can provide competitive advantages across all business units.

In conclusion, while it may seem that IoT is only an “IT thing” but in reality, IoT affects the business-side more than it affects the IT-side and not leveraging it can mean the difference between staying alive or quickly becoming irrelevant.

IoT devices with respect to your Enterprise
IoT devices with respect to your Enterprise


  1. Gartner’s definition of Internet of Things
  2. Gartner’s Internet of Things Installed Base Projections

Service Orientated Architecture (SOA) Migration Case Studies and Lessons Learned – A Critical Review

Service Orientated Architecture (SOA) is a framework that allows business processes to be highlighted to deliver interoperability and rapid delivery of functionality. The benefits of SOA include reuse of generalized services, reduce costs and better business and IT alignment. If done correctly, it helps an organization respondent to ever-changing business needs efficiently. If done incorrectly, it can create bureaucracy and silos. This article evaluates the decisions, assumptions, and conclusions made by the research paper, SOA Migration Case Studies and Lessons Learned.

In the research paper, two research and evaluation methods are used to assess different cases for SOA. The first method is the Case Study Method where the researchers develop a theory and based on that theory they develop criterions to select the case studies that will be assessed. This Case Study Method is shown below:

Case Study Method
Case Study Method

The second method uses a customized version of the Evolution Process Framework (EPF) to evaluate SOA. This customized version is called EPF4SOA and the phases involved in the evaluation are shown below:

EPF4SOA Phases
EPF4SOA Phases

With the research and evaluation methods in place, the research paper goes on to assess three multibillion-dollar organizations that have been around for at least 50 years. These organizations have legacy systems that have become archaic and thus they are unable to respond to rapidly changing business needs. Keeping these limitations in mind, these organizations go on the path to extract as much functionality from these legacy systems as possible by creating SOA services that could be used in the organization. Based on the EPF4SOA, the research paper goes on to claim that for effective SOA migration, organizations need to have a strong business case, services design, technology selection, SOA governance, and education and training.

As we read this report, it seems obvious that the researchers have done a good job of evaluating these large organizations from the finance and telecommunications sectors and in highlighting the lessons learned on SOA migrations. However, this research has made some decisions and assumptions that need to be understood. Firstly, in the Case Study Method, there seems to be an element of confirmation bias when the cases being selected are based on an initial theory. This confirmation bias can lead to selecting cases that fit what the researchers are looking for rather than selecting cases and then determining what theories can be derived from those cases. Secondly, the research looks at organizations that have been around for a long time and by their very nature are most likely to have legacy system issues and make the assumption that other organizations would have the same issues. Lastly, the research report alludes to that SOA can help in business and technology alignment but does not take into account strong leadership and organizational change management capabilities that are needed for SOA migrations.

Keeping the above in mind and carefully reading the case descriptions, we can extrapolate that there might be some potential challenges in the cases being presented in this report. These potential challenges are explained below:

  1. SOA is not only an IT concern: One of the lessons learned in this report indicates the need for a strong business case for SOA developed by IT in order to get management support. The fundamental problem with these lessons learned is that it automatically puts the burden of implementing SOA across the entire organization on IT and takes it away from the business side’s responsibility and involvement with SOA. While IT is responsible for creating SOA services but the business has to work collaboratively with IT. Business has to understand how the organization came to a point where it became difficult for quick system changes and how to avoid situations like this in the future. Thus, SOA is not only an IT issue but an organizational endeavor that involves all parts of the organization as well.
  2. Organizational processes need to be reevaluated: One of the cases mentions the presence of too many point-to-point integrations that are reducing the ability of the organization to be more agile. While this might be the case but there is a bigger perspective here that is missing. This perspective revolves around organizational processes in place that led to this in the first place. These organizational processes not only entail IT but also the business side. It seems like in this case IT would do what businesses ask them to but there has to be some mutual understanding that the requests have to be understood holistically. Even after a SOA migration, if these organizational processes are not optimized they might still result in ad-hoc requests from the business leading back to point-to-point integrations.
  3. A long-term view on legacy systems is needed: The cases in the report indicate that replacing the legacy systems was not an option since it would be costly to do this. While in the short-term this seems like a good idea but in the long-term, there are issues with this approach. These issues entail the constant “patching” to upgrade underlying hardware and software in addition to overburdening legacy systems where new services are being added on top of systems that should be replaced rather than being continued to extend their end of life. While for some organizations it might not be possible to replace legacy systems altogether but there should be a plan to retire these systems with new systems eventually.
  4. No measurements mean no ROI exists: While some organizations in the research report did measure SOA migration ROI but that was done after the fact. So, if the organizations were not measuring pre-SOA how would they know if what SOA migrations promised is what the organization was able to achieve. Herein lies the problem where quantification and justification are made to show SOA being a success without doing the due diligence before embarking on the SOA journey.

In addition to the above-identified problems, the research report does not put enough emphasis on the importance of governance that is needed for SOA. Let’s explore what is governance and why it could be one of the differentiating factors in SOA migrations.

Governance: Governance is the policy of how things should be done and provides a framework in which business processes can operate under regulatory, time and other constraints. Thus, governance is an organizational responsibility even for SOA and not only an IT one. In order to accomplish this, a governance board should be set up that consists of a cross-functional team from both IT and business. Additionally, governance should not only include the overall organization and management of SOA activities but also the creation of success and failure measurements. These measurements should be used to actually determine the state of SOA within the organization instead of people doing vaporware measurements that have no grounds in reality.

In conclusion, while the research report is interesting in its own right but it should not be taken as the only lessons learned for successful SOA migrations. Based on a few cases these lessons learned cannot be applied across various organizations such as smaller organizations, governments, and nonprofits but should be taken with a grain of salt. The lessons learned should be a start but not the bible for successful SOA implementations. A successful SOA implementation will depend upon context, processes, technologies, and people since broadly speaking SOA is an organizational change management journey.


1. SOA migration case studies and lessons learned

5 Questions to Ask About Predictive Analytics

Predictive Analytics is a branch of data mining that uses a variety of statistical and analytical techniques to develop models that help predict future events and/or behaviors. It helps find patterns in recruitment, hiring, sales, customer attrition, optimization, business models, crime prevention and supply chain management to name a few. As we move to self-learning organizations, it is imperative that we understand the value of Business Analytics in general and Predictive Analytics in particular.

It turns out that Predictive Analytics is about Business Transformation.  But in order for this Business Transformation to take place, you have to take into account the organizational contexts in the following ways:

  1. Strategic Perspectives: Not all organizations are the same and thus what works in one organization might not work in yours. Based on the knowledge of your organization’s maturity, you have to decide if Predictive Analytics is going to be a top-down, bottom-up, cross-functional or a hybrid approach. Additionally, take into account what should be measured and for how long but be flexible in understanding those insights might be gained from data that might initially seem unrelated.
  2. Tactical Perspectives: One of the key factors in Business Transformation is change management. You need to understand how a change would affect your organization in terms of people, processes, and technologies. You have to take into account the practical implications of this change and what kind of training is needed within your organization.
  3. Operational Perspectives: It is all about how the execution of Predictive Analytics is done within your organization. To fully integrate Predictive Analytics into your organization, you have to learn from best practices, learn the pros and cons of your technology infrastructure and determine if the necessary tools are intuitive enough for people to make use of them.

Now that you understand the different organizational perspectives, it is time to ask the following:




Who uses Predictive Analytics to make decisions? Who should use Predictive Analytics to make decisions?
What happens to decisions when Predictive Analytics is used? What would happen to decisions if Predictive Analytics will be used?
Where does the data for Predictive Analytics come from? Where should the data for Predictive Analytics come from?
When is Predictive Analytics relevant? When should Predictive Analytics be relevant?
Why Predictive Analytics is being used? Why Predictive Analytics should be used?

When you ask the above questions, keep in mind that the reliability of the information and how it is used within the organization is paramount. A pretty picture does not guarantee that the insights you get are correct but you can reduce decision-making errors by having people who understand what the data actually means and what it does not.



%d bloggers like this: