Data is everywhere. According to a report by IDC and EMC, the digital universe will grow by a factor of 300 between 2005 and 2020, increasing from 130 exabytes to 40 trillion gigabytes. This means that there will be more than 5,200 gigabytes for every single person alive in 2020.

This tremendous amount of data is heralded by organizations which understand that this information is essential in helping them improve their businesses. Gautan Madiman, digital marketing executive at Autodesk, notes that data is creating a lot of opportunities for businesses, including the need to hire analysts. Yet, data on its own isn't enough and savvy business leaders know that they need to invest in the right human resources to crunch the numbers and extract analytic insights from them. "We need people to comb through data and provide insights and recommendations that can add value to customers and businesses," Madiman notes.

However, there's a major problem. Data analysts are in high demand and there isn't enough of them. In fact, in a research note released last year, Gartner predicted that in a mere two years Big Data will create 4.4 million jobs globally. Yet, only one-third of these positions will be filled, resulting in "real financial and competitive benefits for organizations." No wonder the Harvard Business Review calls data scientist positions "the sexiest job of the 21st century."


Considering the known shortage of data scientists, there's a lot of pressure on the education system to prepare the next generations of employees to take on these jobs. A number of higher education entities are doing a good job of preparing data scientists who can make sense of data and a lot of money is being invested in such courses. As The New York Times puts it, "in the last few years dozens of programs under a variety of names have sprung up in response to the excitement about Big Data, not to mention the six-figure salaries for some recent graduates." However, experts believe it's important to expose youngsters to analytics from an early age. Elli Sharef, co-founder of recruitment firm HireArt, says a crucial problem is lack of focus on mathematics in middle and high school in the United States, especially when compared to very analytics-driven countries like India and China.


Shortage is a concern for organizations

The shortage of data analysts is a legitimate concern for organizations which understand that data on its own isn't enough. As Lara Albert, senior vice president of global marketing at Globys, notes, organizations have enough proof points to demonstrate that data-driven decisions drive better results. Cognizant of this, many organizations are trying their utmost to hire analytic talent that can make the most of their data. "Organizations need to constantly look for individuals with competencies to wade through data clutter and generate meaningful insights," notes Deepinder Dhingra, head of products and strategy at Mu Sigma. The following are some of the traits that organizations should look for when hiring analytic talent:

  • The right raw skills: Dhingra believes that there are very few decision sciences professionals in the market. "One has to focus on getting the right raw talent and then training them to become decision science professionals," he says. Dhingra explains that at the very core, each individual needs to understand data analytics tools and techniques and have consultative first principles-based thinking to be able to define problems. Organizations should also look for individuals who have a natural inclination towards problem solving. "Creative problem-solving skills with a quantitative bent of mind and curiosity to ask more relevant questions is a must," he says. Additionally, business leaders should test these professionals' ability to communicate, synthesize, and articulate their learnings as well as their ability to learn new skills and use new technologies. Sharef says rather than focusing on the computer language skills, companies should make sure that new recruits have the right quantitative knowledge to be able to make the most out of raw data.
  • Ability to ask the right questions: It's not enough to be able to come up with the answer to a question. Experts agree that good analysts need to know which questions to ask. "Asking the right questions is a very important part," Dhingra says. In fact, Mu Sigma's recruitment process includes a "curiosity" test, which analyzes how each candidate demonstrates curiosity in the course of his academic and professional experience. Organizations need to make sure that analysts don't have a preconceived notion of what's driving a problem and use the data to validate their perception. "Data scientists don't make assumptions," Albert explains. She uses the example of a mobile phone operator which wanted to increase revenue and reduce churn by migrating customers from pre-paid plans to a contract. However, the company was working off the wrong assumption that customers with the highest tenure would be more willing to migrate whereas data showed that those who had been on a pre-paid plan for a specific amount of time wouldn't want to switch. "Knowing how not to make assumptions and continually seeking to ask the right questions is a very important attribute," Albert says.
  • Attention to detail: Data can hide a lot of information, making it essential for data scientists to be extremely detail-oriented, allowing them to identify and understand the smallest nuances in the information. Further, the smallest change in a computer code can give a different answer, notes HireArt's Sharef.
  • Business-driven: Simply being able to work with mathematical models to analyze data isn't enough. The best data scientists have good business acumen and understand the business problem that an organization is trying to resolve. "You need the right combination of analytic skills and business and industry acumen," says David Nelson, eClerx' practice lead on web analytics and competitive intelligence. Albert agrees, adding that the best data scientists have expertise in diverse backgrounds.
  • Team players: Traditionally analysts were part of IT departments and spent most of their time crunching the numbers on their own. However, experts today believe that the best data scientists are ones who can work hand-in-hand with colleagues from different departments to gain a complete understanding of the business and allow them to search for answers in the data. Albert notes that this requires a commitment from the organization's business leaders to get rid of departmental silos. She says telecommunications companies are recognizing the importance of putting data scientists in a central role, and are integrating them within customer-facing teams, allowing the analysts to know exactly what's happening.

Finally, once organizations have good data scientists in place, they need to make sure that they're focused on extracting insights rather than spending time managing data, which is very time-consuming, Nelson notes. Autodesk, for example, outsources a lot of its repeatable and manual tasks, like data cleaning and combing, creating reports, and standard processes, Madiman notes. This gives the more experienced data scientist the ability to focus on making sense of the data and extracting insights that will lead to business benefits.