It is pretty staggering to realise there is so much data in the world that last year we were on the verge of running out of words to describe it. What should come after the yottabyte? Would the International Committee for Weights and Measures be able to approve the naming of a number above yotta’s 24 zeroes in time for the anticipated existence of 1,000,000,000,000,000,000,000,000,000 bytes of data? (I’ll leave you to discover the answer for yourselves.)
It is yet more mind-blowing to think that 90% of that data was created in the previous two years, according to IBM.
“Big data” is the phrase you will have heard bandied around increasingly over the last couple of years to describe large volumes of data (though not quite as big as one yottabyte). Big data has become big business – and more and more people are trying to understand it. There were 110,000 searches on Google for “big data” in December 2012. By September this year, that number had almost doubled to 201,000 searches.
Driving that interest is the potential data holds for business. “Companies have always collected data for accountancy purposes, but traditionally that was after the fact – you would report on what happened to the business last year,” explains Duncan Ross, director of data science at Teradata, a global provider of data analysis technologies and services. “We’ve got beyond the stage of elderly white men in suits making decisions on gut instinct now. If companies want to outperform the market, they have to take notice of the data and understand what’s happening.”
Forward-thinking companies are going extensively beyond recording and analysing traditional business metrics such as sales and profit margins. The much more granular data they collect is allowing them to unearth new efficiencies, analyse trends and understand customers to unprecedented new levels.
Ross gives the example of a retailer whose delivery teams collect data on, say, whether a customer is home, the route taken, how long it’s taken, which products got damaged on the way, and so on, for every delivery made. The company can then analyse all that data to determine better routes, suggest to the customer more convenient times to visit, find new efficiencies in its logistics.
The data can also be sold on to other companies – to the packaging company responsible for boxing up certain products, for example, to show them which types of packaging are most prone to damage in transit.
“Data analysis has become so important because everything is now measured and digitised: who we are, where we are, who we know, what we’re doing and how often, what we buy and when,” says Harvey Lewis, analytics research director at Deloitte.
Most companies now collect data in some shape of form, Lewis says. But simply having all this data is of no use whatsoever if companies don’t know what do with it. Organisations are now able to collect – and in many cases prone to collecting – such a morass of information that they become “so overwhelmed with data that they just don’t know where to start”, Lewis says.
This is where data analysis comes in – the function of drawing out meaningful insights from data.
How data analysis is changing the game
“Data analysis has become the digital equivalent of H.G. Wells’s time machine,” Lewis says. “It allows us to peer into the past to understand what has happened and why; it allows us to examine the present, anywhere in the world, to find out what’s happening right now; and it allows us to travel into the future to predict what might happen and how we can take actions that help make that predicted future actually come to pass (or not).”
Ben Dalgliesh is the manager of OPEN, marketplace development, at MediaMath, a creator of data analysis technologies for the media sector. He believes that without a data analyst, “organisations fly blind”. “Essentially, the data analyst can quickly inform decision-makers about what is working well versus what isn’t.”
Informing strategy is at the heart of data analysis. Lewis says: “The importance of data analysis isn’t in how smart your analysts or how powerful your data mining tools are – although these things are important – it’s how well do you understand the role that data and analysis play in every part of your business, from strategy and organisation to processes and finance?”
That means businesses must have a clear structure in place to be able to collect and harness useful data. Lewis adds: “When businesses are successful because of data analysis, it’s because they didn’t start with either the data or the analysis – they started with a set of objectives or business problems, and they reconfigured their operating model to solve them. In other words, they put the horse before the cart.”
Lee Chant is managing director at Hays IT and Telecoms, the leading recruiting experts. He has seen a huge surge in demand for data analysts and related roles in recent years, such as data architects, security analysts, development specialists and database administrators. “This is very, very buoyant area and the whole business intelligence piece is a growing area,” he says.
Chant agrees that it is critical to get data analysts to structure objectives and projects before data is collected, to avoid being stranded with endless “big data that is like a bucket where everything sits” but that ultimately companies don’t know how to meaningfully use.
He explains: “You can collect as much data as you want and build huge databases, but if you haven’t got good analysts/architects right at the start of the whole project, then you won’t have any idea how to use it or mine it properly.”
The code at the core of a great data analysts
So what makes a great data analyst? The best “have a good appreciation of business and strategy”, says Lewis. “This means that they ask the right questions at the outset, assemble the right data and the right analytical approaches to derive the right insight at the right time to allow the business to make the right decision that leads to the right outcome.”
Once they have collected the data that they know will be useful, it is about drawing out the conclusions that will inform strategy – and communicating those conclusions effectively.
Dalgliesh says: “Crunching data is one thing, but the ability to turn data into a story with insight and conclusions is critical to the role. […]The role of the data analyst becomes central to effective business operation at all levels of the organisation, providing clarity to stakeholders and direction to business owners.”
This has meant that data analysts are more visible in the business than some might expect. “I think the general impression of what a data analyst does is archaic,” Dalgliesh says. “Many are client-facing and more ‘front room’ than people might think. I think this development will continue: why employ a good data cruncher and a good client-facing employee when you can employ both at once?”
Attracting top data analyst talent
So how do you go about attracting these all-rounder business must-haves? It’s tough – not least because there is currently huge demand and a significant under-supply of data analysts in the UK.
Chant says there are typically two key drivers for top data analyst talent. The first is “decent remuneration and benefits”. That can be anything from £25,000 to £150,000 base salary, depending on levels, he says. Contractors can range from £400 a day up to around £1,500 a day.
The second is to present an opportunity that is distinctly challenging and exciting, making it clear how integral the data analyst will be to your business’ development. “If you’re a data analyst, and in an interview the employer says, ‘We don’t understand what we’re doing with all this data, can you take a look,’ that doesn’t sound particularly attractive compared with an employer who says, ‘Data is at the heart of what we’re doing for the next 10 years, and we need someone who’s instrumental to that’.”
Ross agrees that data analysts will always want to work on genuinely interesting projects. This is where large companies can often lose out on talent to more entrepreneurial smaller companies and start-ups, he says. “It is very difficult for large companies to innovate, because they become very risk-averse. So they might have a data analyst 100% occupied on genuinely valuable projects, but if they’re not innovating then the best talent will always walk to something more exciting and fun, because it’ll probably be on the same money.” He says that among his peers there is particularly a lot of “buzz” around the London start-up market.
British companies also face an issue in that top analyst talent is often poached by the US, where higher salaries, “more exciting challenges” and a more entrepreneurial attitude are big lures.
There is also a risk to the data analyst talent pipeline in the UK, as schools and universities are being slow to create courses and qualifications that would equip students with the necessary education for data analysis. “It’s a field that doesn’t fit naturally into the UK academic spectrum,” Ross says.
Currently, data analysts come from a variety of backgrounds, Chant says, although they will typically have a scientific background. Ross agrees that they “almost always have a numerate background”.
Ross says: “Simply piling more graduates into bottom of pipeline doesn’t necessarily help now though, as you need experience and business experience. [The role] is a lot about mind set as much as coding kudos.”
But the amount of data created by the world certainly doesn’t look like it’s going to diminish anytime soon. We will need new number-names to quantify its vastness, and vast numbers of new data analysts to interpret it. Until there is enough top talent to go around, companies looking to hire the best in the field would be wise to offer opportunities that are exciting, challenging and strategic if they want to avoid having to pay salaries in the realm of 24 zeroes.