Where is My Big Data Coming From and Who Can Handle It

Recently, a reader asked my insights on the article (Data Scientists are the New Rock Stars as Big Data Demands Big Talent).  Here is my response.

It seems like in today’s world people and organizations are somewhat struggling with this big data concept and do not know where to begin. Due to this reason, they are collecting everything they can think of in the hopes that one day they will be able to use this data in a meaningful way such as better customer experience, new products/services, better collaboration, increasing revenue, etc. This hope approach of “let’s collect data and later decide what we can use it for” on the surface might seem sound but last I checked hope is not a strategy. Perhaps this is one of the reasons that even now only <1% of the data collected is actually being analyzed. What good is more data when one cannot even make sense of the other 99%+ of data it already has? Are we chasing a ghost?

While it is true that vast amounts of data are and will be generated from financial transactions, medical records, mobile phones, and social media to the Internet of Things but there are questions that need to be asked to understand data’s meaningful use:

  1. How will data be managed?
  2. How will data be shared?

I believe that in order to come to a point where data becomes meaningful and useful it would require (broadly speaking) three phases:

  1. Establishment of standards, governance, guidelines. (E.g., open architectures)
  2. Creation of industry specific data exchanges. (E.g., healthcare data exchanges, environment data exchanges, etc.)
  3. Creation of cross-industry data exchanges. (E.g., healthcare data exchanges seamlessly interacting with environmental data exchanges, etc.)

Additionally, let’s keep this in mind that the data we are talking about is data that can be captured by current tools and systems but the data which is perhaps the most difficult to capture is unstructured human data which within organizations is called Institutional Knowledge. This does not reside in a document or a system but in the minds of the people of an organization who understand what needs to be done in order to move things forward.

So, the question becomes, do we really need Data Scientists who have a mix of coding skills with PhDs in scientific disciplines and business sense or do we need someone who is able to connect the dots and have the ability to create the future. The answer is not a simple one. Perhaps you need both. The ability to code should not be the deciding factor but rather the ability to leverage technology and data should be. I agree that there is a shortage of people with diverse talent but there is also a shortage of people who actually know how to leverage this kind of talent.

Before organizations go on a hiring spree they should consider:

  1. Why do they need a Data Scientist? (E.g., have strategic intent, jumping on the bandwagon, etc.)
  2. Who will the Data Scientist report to? (E.g., Board, CEO, CFO, COO, CIO, etc.)
  3. Does the organization have the ability to enhance/change its business model? (E.g., making customers happy, leading employees, etc.)
  4. Is the Data Scientist really an IT person with advanced skills or does s/he have advanced skills and happens to know how to leverage technology and data?
  5. How often will you measure the relevancy of the data? (E.g., key data indicators)
3 Phases of Big Data Harmonization
3 Phases of Big Data Harmonization

5 Factors for Business Transformation

Business transformation entails assessing people, processes, and technologies of the organization in terms of the current state (where the organization is right now) and future state (where the organization wants to be). In these assessments people, processes and technologies are not standalone areas but are part of an integrated and holistic organization. If any of these areas are ignored or not given enough attention then true business transformation is just a pipe dream.

In order to have a holistic understanding of an organization and its broader role in society, there are 5 factors that need to be considered. These factors should have an inward focus and an outward focus. If the organization only has an inward focus then sooner or later it will be taken over by competitors and if the organization only has an outward focus then it will crumble under the weight of its own (mis)management. So, both are necessary. The 5 factors that will determine an organization’s success and longevity are Strategies, Politics, Innovation, Culture, and Execution or simply called The SPICE Factors. It is critical to remember that:

  1. Strategies are to be used as blueprints. They are not shelf-ware.
  2. Politics is a reality and needs to be understood.
  3. Innovation is the lifeline and not only the responsibility of the R&D department.
  4. Culture is the soul. Lip service is not culture but your actions are.
  5. Execution is evolution. Without it you become stagnant.

All of the above need to be measured constantly, managed consistently and reviewed periodically.

SPICE Factors
SPICE Factors

Below is a poll on what people think are the most important areas and factors for Business Transformation:

How to automatically save your job search information in a log file?

How to use AppleScript to search for jobs on simplyhired.com and monster.com using a list?

How to use AppleScript to search for jobs on indeed.com and careerbuilder.com?

This video shows how you can use AppleScript to search for jobs online.

In addition to automation of job searches based on your keywords, this script gives you the ability to reduce the email job alerts that you get in your inbox since you have a choice not to sign-up for them.

Although this script has to be initiated by you and you can set it so that it runs at a certain time.

%d bloggers like this: