The Migration of Big Data Processing To The Cloud

Better, Faster, Cheaper

First and foremost among the benefits are the flexibility that Cloud affords. The companies of the first dot-com boom had to run their own physical servers and maintain capacity to serve peak demand which was rarely actually required. Cloud computing, on the other hand, allows businesses to minimise their cost structure and to scale on demand – rapidly increasing server capacity and processing resources, as required. At the same time as being vastly more flexible, Cloud also presents the opportunity to significantly simplify infrastructure requirements.

“The great thing about Cloud applications is always that you only pay for what resources you use and when you use them,” says Empiric’s Big Data expert, Sebastian Rodriguez. “Running resource-heavy applications (with most Big Data applications being resource intensive) requires a big setup. Deploying real time Big Data applications on the cloud or running analytics on cloud resources means that companies that are not using these resources constantly can spin them up when they need to, run the processes they need and then shut them down.”

Alex Moore, Delivery Director UK & Tech Evangelist for CAPSiDE, also highlights Cloud’s capacity to incorporate automation tooling into infrastructure, working “in a systematic, reproducible way” and thereby saving time and effort.

There is also the advantage of connecting systems together. “Once you’ve got that large data platform, you want to be able to exploit the data at scale. While that might be running some kind of analytics or reporting, or whatever action it needs to be, it might also flow through any number of other systems,” says Alex. “By having them available in the Cloud environment you can naturally connect it into some of those wider ecosystems... Compared to legacy infrastructure, the flexibility, the scale, the immediacy makes it a no-brainer, I think.”

“When companies are going to Cloud they need to consider costs for certain – although applications should be cheaper – they need to be well written so they don’t use too many resources to return the desired results,” notes Sebastian. “Costs can quickly rise if applications are not well written.”

There are also an increasing range of Infrastructure as a Service and Platform as a Service options available – with Microsoft and Google both tackling Amazon’s status as industry incumbents.

“Microsoft and Google have tried to combat AWS’s dominance by offering their cloud services fairly cheaply or, in the case of Microsoft, for almost free as a bolt on to their enterprise software,” says Sebastian. “AWS still offers the largest range of services, with nearly 100 across compute, storage, database, analytics, networking, mobile, developer tools, management tools, IoT, security and enterprise applications. And it has been around the longest.”

There’s a good case to be made, though, that while AWS is highly competitive on pricing, Microsoft Azure may be a good bet for organisations that are highly invested in the Microsoft ecosystem.

Of course, there are also many other providers. “[Things open up] when you start looking beyond the top three,” notes Alex. “You’ve got people like IBM Cloud and Oracle Cloud and potentially Alibaba – and if you’re doing anything in China on Cloud, you’ll probably end up moving towards Alibaba.”