Is Big Data analytics too big for enterprises to handle?


By Abhijit Joshi, Sr. Manager – Revenue Optimization Analytics, IDeaS Revenue Solutions

Big Data. A term that organizations are au fait with, and are finding it taxing to manage.Rightly defined, it is the volume, variety and velocity of data exceeding the organization’s storage and processing capabilities for accurate and timely decision making.A recent report by Forrester predicts that the big data technology market will grow at a 12.8% CAGR (Compound Annual Growth Rate) over the next five years.

Businesses always had an enormous amount of data related to transactions, customers, etc., which is now being retrieved with ease through automated systems. The extensive usage of economical internet plans and Wi-Fi hotspots; and the incredibly fast evolution of internet connected devices and the Internet of Things (IoT) has led to an upsurge of the data market. Now, it is for the data scientists and the analytics teams to respond to the challenge of getting real-time, meaningful and actionable insights from this humongous data. But, do businesses have enough of these resources?

The skillset pool for the data wave
A multitude of skills are required to decode the realm of Big Data.The first step in the process is to Extract, Transform and Load (ETL) the data. Across organizations, there are software developers and Quality Assurance(QA) engineers acquiring this skill on-the-job. A subset of the ETL skill are the statistical programming languages like SAS or R, where there are established curriculums to facilitate learning. Analytical concepts is the key and a scarcely available skill that needs to be addressed through hiring and training. Deep analytical skills when combined with the business knowledge aid the process of deriving meaningful insights, thereby giving the organization an edge over its competitors.

Organizations have been putting their efforts to address this talent gap by recruiting domain and analytical experts into one team so that the relevance of the insights derived from the data can be validated. The report IDC FutureScape: Worldwide Big Data and Analytics 2016 Predictions states that the technology skills to deploy and optimize the big data infrastructure cannot be ignored.

The growth in demand for analytical skills can be classified into two broader sets. The first set is the companies providing analytical tools/solutions and includes analytical software developers, QA engineers and data scientists to name a few. The second set of analytical talent is the one that can make use of these solutions to generate meaningful and actionable insights. It includes primary users, analysts, analytics managers, analytical practitioners,chief data officers etc.

Cultivating the ‘Big’ talent
Big Data and predictive analytics are fast gaining traction. But, here is the glitch – finding the right talent for each job role along with the right domain experience. The report by IDC reveals that the shortage of skilled staff will extend from data scientists to architects and experts in data management. There is also a mention of how Big Data related professional services will have a CAGR of 23% through 2020.

Is hiring the only solution? Not really. But, it is imperative to decide upon the competencies like analytical skills and domain expertise that need to be hired in comparison to those, where,acquisition of skills is relatively easy. Finding talent under the same roof is one cost-efficient approach that the small and mid-sized companies are preferring over hiring. Ever thought of a lateral transfer from the technical support team to the analytical QA team?

A structured 10-12 week training program can accelerate the grooming of such talent with minimum knowledge on the domain or even fresh graduates with a Mathematics or Statistics background. There are companies that have internal talent development programs, where,analytical software developers and QA Engineers are absorbed directly into the R&D teams as Data Scientists and are ensured of job rotation, from either one product or project to the other.

In a study conducted by MIT Sloan Management Review and SAS, it was found that the organizations achieving the greatest benefits from analytics are much more likely to have a plan for building their talent bench.Another key factor that can affect hiring in this space is that of the average salary for any role at any level is relatively higher than a role in other areas, a mismatch between demand and supply. Hence, it is vital, to not only give preference to candidates with analytical capabilities while hiring but also nurture and retain this talent.

As more and more businesses explore the Big Data sphere to gain a sustainable competitive advantage, the demand for analytics, and hiring, in turn, are bound to see an exponential increase. This is a call for the data analytics enthusiasts to invest their intellect not only in the domain of their choice but other parallel domains to reap the benefits from the market. Now is, also, the time for the talent management teams to integrate new talent with traditional data teams that can help push the boundaries of an organization’s analytical proficiency.