Using Hadoop in the cloud, JustGiving has moved from simply supporting transactions, to actively encouraging donations to charity.

While many traditional enterprises are hoping to polish the hordes of unstructured, historical data and package it as a new revenue stream, fundraising website JustGiving’s data lead says that data should be used primarily to improve customer services.

Millions will be donated through JustGiving's site in the run up to the London Marathon this month ©Flickr/Josiah Mackenzie

“If you spend more time trying to delight the user, the lifetime value increases as a side-effect. That means beginning to understand your users, and not making bold assumptions,” Mike Bugembe told Techworld at the Hadoop Summit in Brussels this week.

JustGiving is the platform that lets fundraisers collect and transfer money to their chosen charities online. Marathon runners, for example, will send a link to their profile on social media or through an email to encourage people to donate with just a few clicks.

But JustGiving wants to be more than just a donation site.

“We have a mission that every great cause gets the funding it requires. This means removing barriers to giving and understanding what makes a person want to donate.”

The 14-strong data team are writing proprietary algorithms to better understand why certain people choose a specific charity, or what means of communications (for example, over Facebook or over an email) makes them more likely to part with their cash.

When Bugembe first joined, he found the firm sitting on 10 years’ worth of historical data.

“It was very transactional. You gave the money, you disappeared, we didn’t give you a chance to discover what you were passionate about. After working with researchers, we figured that we could fundamentally change the action of giving, moving it from transactional to engaging, personal and relevant,” he said.

JustGiving previously used traditional SQL transactional databases, but found that analysis became tricky when looking at web logs, which are terabytes of data in quantify. To combine these web logs with transactional data, the team were “doing calculations over terabytes of data several times over so we can remain relevant to the donors at the time”.

The firm found it could only get the compute necessary through the cloud, and chose Microsoft Azure’s HDInsight - a cloud-based Hadoop platform - to power its data crunching.  

What data does it use to encourage people to give?

Bugembe explained that the firm analyses "whatever we can get our hands on" which could include social media interactions, web browsing following a donation and a user's profile information, such as gender or age.

“It’s whatever we can get our hands on and whatever the user allows us to use,” Bugembe said. “We must give people the choice. We cannot do things undercover and people must be aware of what is taking place as it is ethically right. We want to make it clear why we are collecting this information.”

Selling as a revenue stream?

The data team “haven't thought that far ahead”, Bugembe said about the possibility of data as a revenue stream. User privacy and suspicions are a serious consideration for the business.

Further, Bugembe believes simply looking at data as an extra revenue stream is not the way forward.

“Others may jump to conclusions saying ‘we have data let’s sell it’ but there is so much more you can do with it - like making your users’ lives better,” he said.

Meanwhile, Bugembe believes the age of the data warehouse may be over thanks to new technologies.

“Thanks to Hadoop you can collect everything and worry about structure when you need it.”

JustGiving uses Azure Table Storage, F# components on cloud algorithms and is considering Apache Spark for real-time analytics in the future.  

Find your next job with techworld jobs