Enterprises today have to cope with powerful and demanding online communities, round-the-clock customer contact, ubiquitous mobile and tablet usage, and multiple customer touchpoints.

All of these yield valuable customer behavioural data, which can be used by businesses to capitalise on customer relations, and all pose challenges to organisations’ IT infrastructure.

The answer, for many organisations, is to boost their data processing capabilities in order to better meet their customers’ needs, whether that means through cloud services, or on premise, in the data centre.

However, these rising customer interactions, and the related growth in structured and unstructured ‘Big Data’, calls for a new IT infrastructure and computing approach, experts argue.

Sanchit Gogia, analyst at Forrester Research, says that organisations have a two-fold problem of having to deliver best-in-class products and services, while also optimising the quality of the customer interaction at every opportunity.

“To succeed, firms must move away from transactional customer interaction methods and toward customer engagement. The CIO's office can help a company use cloud computing, social technologies, mobility solutions, and analytics to make this transformational shift,” says Gogia, adding that there is a need “for businesses to understand the customer engagement process, and adopt the right technology framework to enable this shift”.

Big Data is not only related to customer interactions, however. For example, the BBC and Press Association handled massive volumes of data during the London 2012 Olympic Games by implementing a dedicated Big Data operational database from MarkLogic to process what it calls its ‘Content Store’.

A spokesperson for MarkLogic said: “Every point scored, every goal counted, every millisecond of a race will need to be recorded. Visuals and text will need to published on multiple platforms. Tens of thousands of pages will need to be updated in real-time and made available to millions of readers, viewers and listeners. The BBC expects that one third of the UK internet traffic will be coming to its channels for information.”

He adds that the MarkLogic database platform will be used to manage and manipulate, in real-time, the enormous volumes of data that will be generated during the Olympics. “It will be the first time that PA and BBC will be processing and publishing content across all their properties in real-time, managing unprecedented volumes of information. In readiness for the deluge of data that the Press Association IT infrastructure is expecting to deal with, the news service has rebuilt the content systems and real-time data delivery architecture.”

Jem Rayfield, lead technical architect at the BBC’s News and Knowledge Core Engineering department, adds, “The Content Store which currently powers all of the statistics and navigation on the sports site has been scaled to handle ingesting many thousands of content objects per second whilst concurrently supporting many millions of dynamic page renditions and impressions a day.”

“A technical architecture that combines a document/content store with a triple-store [used to locate, query and search documents by ‘concept’] proves an excellent data and metadata persistence layer for the BBC Sport site, and indeed future builds including BBC News mobile.”

Big Data infrastructure

Having the right IT infrastructure, including the server hardware, software, networking, storage, and data management technologies, are all essential parts of the puzzle when it comes to Big Data.

“One of the greatest pitfalls associated with data management, especially in light of Big Data, lies in developing the appropriate strategy to manage it, and using the right technology infrastructure and resources to help attain, interpret and act on that data in a way that truly contributes towards the growth of an organisation’s bottom line,” comments Kieran Kilmartin, marketing director EMEA and India at Pitney Bowes Software. (The firm produces software and services designed to help create business value from customer information.)

Indeed, as organisations begin to recognise the need for more powerful data centre and public/private cloud infrastructures, there is a growing recognition that software vendors and consultants and systems integrators may have got it wrong in the past, by under-speccing and under-powering IT systems for enterprise applications such as Enterprise Resource Planning (ERP).

In a YouGov survey of mid-sized UK firms in October 2011, 23% of IT managers said their ERP system was underperforming, with 61% using at least one of the three top ERP brands: SAGE, SAP and Microsoft Dynamics. In fact, three quarters of survey respondents said they are experiencing problems, and many said they want to replace their systems.

Keith Langmead, a systems administrator, comments that in many cases, it’s the user organisation itself that makes the mistake of under-speccing the hardware in their rush to implement an enterprise solution. “I've seen too many companies rush to buy hardware to fit a perceived requirement, only to find they under-spec some part of it, and can't easily upgrade to resolve the problem.”

All this indicates a need for more powerful data processing environments, in both the corporate data centre and the private/public cloud, which can offer massively scalable computing as and when required. As a result, organisations will be able to handle Big Data, and run their enterprise applications more efficiently, while improving the customer experience by capitalising on Big Data.

Processing boost is an example of a cloud services firm that boosted its server infrastructure to support the growth in its customer numbers, who are also grappling with Big Data. “To accommodate its rapid business growth, Salesforce needed hardware that would give customers fast and continuous access to their data and help the company avoid growing pains,” says Ana Cantu, Dell enterprise technology blogger, who adds that Salesforce “moved from proprietary systems to Dell servers with industry-standard hardware and software components”. This was “a cost-effective way to scale its global database infrastructure” says Cantu.

According to David Fearnley, senior director of technical operations at, the firm’s double-digit growth each year since its founding in 1999 created numerous challenges for the IT department. "To gain customers, we need to prove that our cloud offerings can deliver fast and continuous access to their applications and their data - and we publish real-time information on our infrastructure so current and future customers can see how we're doing."

The new server infrastructure, which “packs more processing power into a smaller space” according to Cantu, doubled database performance for Salesforce.

Forrester Research’s Sanchit Gogia, says cloud-based services, like Salesforce, can help businesses to respond to customer interactions effectively because of their ability to scale up and provide more ‘power’ when needed.

“Relative to more traditional on-premises applications, cloud-based solutions allow companies to be more agile and better respond to business needs. Coupled with the pay-as-you-go economic model, this makes cloud a compelling option for organisations, particularly in areas where no existing on-premise solutions are in place. A good example of this is cloud-based CRM from vendors like Salesforce. The cloud delivery model ensures that the application is available at competitive prices, yet offers similar (if not the same) functionalities and much higher scalability,” says Gogia.

Another example of a company that has made a significant effort to grapple with customer interactions and Big Data is Halfords, which enhanced its server infrastructure, by adding additional data processing capabilities.

In November 2010, the retailer implemented a SAP Business Warehouse Accelerator (BWA), a server appliance with additional processing capabilities, to help manage Halford’s growing volume of Big Data.

Halfords was already using SAP business warehousing software to provide it with a certain level of buying information, but the capabilities of the original system didn’t match the depth or performance Halfords required as it extended its footprint to 468 stores nationwide, and grew its online presence.

In particular, Halfords wanted to scale up its ability to “examine and anticipate customer needs” through an analytics system that offered ‘basket analysis’, allowing it to understand the performance of product offers and the shopping habits of its customers.

Halfords’ BWA appliance was installed and managed by its service provider Centiq, whose managing director, Glyn Heath, explains, “Centiq’s BWA accelerator added processing capacity to Halfords servers and IT infrastructure. BWA is a separate technological appliance which works with, and accelerates, the data reporting systems already in place. This improved processing potential allowed Halfords to do in-basket data analysis and speed up its business report process.”

Heath adds that, since implementing basket analysis and fitting the BWA accelerator technology, Halfords’ data cubes have run 10 to 20 times faster than previously. “Reports that would previously take days can now be turned around in minutes allowing for faster reaction to consumer trends and demands.”

In business terms, the retailer can now work out customer behaviour and consumption patterns quickly and act accordingly, responding in real-time to consumer demands and trends.

This is the promise of both cloud, and a data centre that has increased compute horsepower: the ability to cope with the additional large volumes of Big Data generated by social computing, and other data sources. Only by having the right underlying infrastructure in place, will organisations be able to stay ahead of the game.