Follow Us

We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message

Blogs

Incubator

Sophie Curtis

Attention start-ups: Big Data does not end with Hadoop

Article comments

Some companies are now looking at alternative methods of handling and processing their data

For many people, the phrase Big Data has become almost synonymous with Apache Hadoop - the open source framework originally developed by Yahoo.


But as organisations become increasingly familiar with the nuances of Big Data, it is starting to look like there will be a much greater appreciation for different approaches to managing Big Data in the years ahead, presenting an exciting opportunity for start-up companies working in this space.


Last month, analyst firm Ovum produced a report which claimed that although Hadoop garners much of the spotlight as a Big Data platform, alternatives from the likes of 10gen, IBM and Teradata, as well as newer companies such as Splunk, are attracting plenty of attention.


A separate survey of business and IT managers, commissioned by EMC revealed that big data analytics is one of the most likely targets for investment over the next 12 months, with 47 percent of business leaders citing big data as a disruptive technology in their industry.


Hadoop has revolutionised data warehousing by moving away from the idea of having a centralised database system. Instead, an application can run on a number of different servers simultaneously, and the results can then be reduced down to a single set of data.


This means that data processing is no longer restricted by the server CPUs, or the fact that the data in question does not fit nicely into tables. The infrastructure can scale to accommodate the amount of data that needs to be processed - whether that data is structured or unstructured.


However, Hadoop was built on the paradigm of legacy data warehouses, which assumed that organisations could wait hours or days for the answer to a business question and its distributors are still working to overcome this problem. It also requires the integration of several components to create a solution, which can make it quite costly.


Some companies are now looking at alternative methods of handling and processing their data, so that the same insights can be gained in real-time, and with very little investment on the hardware side.


One company attempting to do this is British start-up Logscape. Rather than having to pull all of the data into dedicated servers to do the data processing, Logscape agents are installed on the various end points where data already resides.


These agents can pick up the file system, detect all the changes in line, and build a little warehouse in the machine - plus there is no need to transport the data, so the whole process is a lot faster.


If a company has 200 machines doing this they create what is essentially a large distributed database, which can then run the background and enable the machines to carry on with their primary tasks. There is no need to spend money on dedicated infrastucture as it's all already there.

Logscape is specifically designed for systems monitoring - as in the case of its customer Sportingbet, which uses the technology to improve application performance by monitoring the end points in real time. The emphasis is on providing quick feedback at low cost, which is something that Hadoop simply cannot deliver.


However, systems monitoring is just one of a myriad of scenarios where the primary aim is to gain real-time insight into a company's existing data with minimal investment, meaning that there are many more opportunities for start-ups looking to carve out a niche in the Big Data landscape.


Hadoop will of course continue to play an important role too, allowing customers to get affordable storage for heavy-duty data sets and offering solutions to some of the really big and complex data processing problems.


However, many companies have not yet found a use for those kinds of Big Data applications. A lot of them are simply looking for ways to gain data insights into the day-to-day running of their businesses, and simple solutions that can carry out these Big Data functions at minimal cost may well win out.


With strong indications that venture capitalists are increasingly turning their attention to investment in enterprise technology start-ups, entrepreneurs could do worse than finding new and intelligent ways to solve companies' Big Data problems.

Enhanced by Zemanta

Share:

More from Techworld

More relevant IT news

Comments

Send to a friend

Email this article to a friend or colleague:

PLEASE NOTE: Your name is used only to let the recipient know who sent the story, and in case of transmission error. Both your name and the recipient's name and address will not be used for any other purpose.

Techworld White Papers

Choose – and Choose Wisely – the Right MSP for Your SMB

End users need a technology partner that provides transparency, enables productivity, delivers...

Download Whitepaper

10 Effective Habits of Indispensable IT Departments

It’s no secret that responsibilities are growing while budgets continue to shrink. Download this...

Download Whitepaper

Gartner Magic Quadrant for Enterprise Information Archiving

Enterprise information archiving is contributing to organisational needs for e-discovery and...

Download Whitepaper

Advancing the state of virtualised backups

Dell Software’s vRanger is a veteran of the virtualisation specific backup market. It was the...

Download Whitepaper

Techworld UK - Technology - Business

Innovation, productivity, agility and profit

Watch this on demand webinar which explores IT innovation, managed print services and business agility.

Techworld Mobile Site

Access Techworld's content on the move

Get the latest news, product reviews and downloads on your mobile device with Techworld's mobile site.

Find out more...

From Wow to How : Making mobile and cloud work for you

On demand Biztech Briefing - Learn how to effectively deliver mobile work styles and cloud services together.

Watch now...

Site Map

* *