There is no such thing as big data, according to Matt Aslett, leading analyst for The 451 Group. Data is not getting bigger, there is simply more of it.
Aslett has coined the term “total data,” to describe a broader approach to data management, managing the storage and processing of big data to deliver the necessary business intelligence.
Total data involves processing any data that might be applicable to the query at hand, whether that data is structured or unstructured, and whether it resides in the data warehouse, a distributed Hadoop file system, archived systems, or any operational data source – SQL or NoSQL – and whether it is on-premises or in the cloud.
The term “total data” is inspired by total football – a football tactic that emerged in the early 1970s and enabled Ajax of Amsterdam to dominate European football in the early part of the decade.
Unlike previous approaches that focused on each player having a fixed role to play, total football encouraged individual players to switch positions depending on what was happening around them while ensuring that the team as a whole fulfilled all the required tactical positions.
Speaking at the Talend Connect conference in London yesterday, Aslett said that there are four key lessons that data management can learn from total football:
1. Promote individuality within the overall context of the system
With total football, players were encouraged to move into space and get beyond the idea that they had a fixed responsibility based on their positional role.
From a data management perspective, organisations are accepting that specialist data management technologies can be used where appropriate alongside existing relational database management technologies, rather than forcing the relational database to do something it was never designed to do.
Examples include the adoption of NoSQL databases alongside relational databases. Also the adoption of MapReduce, alongside SQL analysis approaches to complement those skills and tools, and to provide the potential for competitive intelligence.
2. Abandon restrictive self-imposed rules about individual roles and responsibilities
In total football, the maintenance of formation was important in balancing the skills and talents of the individual team members within the overall team system.
From a data management perspective, there is now a greater willingness to adopt specialist technologies where appropriate to the individual application, but that does not require the abandonment of existing SQL analysis tools and techniques.
“We've seen a lot of talk in the last years or so about, does NoSQL mean the end of the SQL database? Does Hadoop mean the end of the enterprise data warehouse. It absolutely does not,” said Aslett.
“It's about where those technologies can give you differentiation alongside your existing investments, and can be used either in a complimentary manner or together in some cases to achieve the desired result.”
3. Enable fluidity and flexibility to respond to changing requirements
Total football wasn't an absolute abandonment of responsibility. The main advantage lay in being fluid and being able to respond to how the opposition was playing.
From a business perspective, the data warehouse is very good at answering queries, but not so good at proving flexibility. It is not good at enabling organisations to respond to new business requirements, for example.
So alternative platforms like Hadoop are adopted specifically for ad hoc iterative analysis, enabling organisations to better respond to new requirements and to experiment with new analytic approaches without affecting the performance of the data warehouse.
“The data warehouse is great at what it does so leave it alone. But adopt other technologies to develop potential new applications,” said Aslett.
4. Exploit improved performance levels
If you're asking football players to cover more of the pitch and not be restricted by their position, they have to have the fitness levels to do that. It is no coincidence that total football arose at the same time as a new focus on sports science and diet and medicine, and the role those things had to play in improving athletic performance.
It is also no coincidence that the new big data management technologies have emerged at the time of more efficient hardware, greater processing power and more in-memory storage technologies. It is that improved efficiency which means that organisations are now in a position to store and process more data, more efficiently than they ever have before.
Data management today is far more complicated than it used to be – there are a lot more moving pieces, according to Aslett. However, companies that learn the lessons of total football and enable more fluidity and flexibility within their data management processes will be able to respond more easily to changing business requirements.