It’s Big Data Week, a collection of events devoted to exploring the juicy opportunities that huge datasets present. These events are primarily for techie and business folk, but I thought I’d go along to one of the gatherings to see bounty there is for journalists too. I schlepped along to a community meetup to hear speakers from big data players like Bit.ly and DataSift.
I have to admit I felt a bit reticent about going; would I be able to follow the discussion? Or would it be all JSON, SQLs and APIs?
Thankfully, it was thoroughly enlightening.
First, there were some juicy data nuggets from Hilary Mason, Bit.ly’s chief scientist and co-founder of HackNY. For instance, 3% of clicks in 2011 went to web pages containing the name of one of the top 100 celebrities internationally. Oh and there are more photos of dogs on the internet than cats.
And as well as counting, Bit.ly devote time to monitoring data. “It’s actually more interesting to look at things in context,” said Hilary. “So we’re starting to watch how stories and ideas start to emerge and come out. For example we were watching the Trayvon Martin case, which started off as a very small local story in Florida state with very geographically dense interest and we saw spread to the whole country, and then to the whole world. We were able to watch that happen.”
Fascinating stuff! How my journalistic ears pricked up.
The panel seemed to agree that it’s not just the growth in the amount of data available that is stark – but the proliferation of methods by which we can cross-examine it. “I think the real innovation is that is that we can ask questions and get the answers back before we’ve forgotten why we asked the questions in the first place,” said Hilary.
They also talked about the emergence of data scientists; that alongside the technical know-how, the key skills scientists need are creativity and empathy. “We see people with maths and statistics skills, but they may not have that spark of understanding,” commented Nick Halstead, founder/CTO of DataSift. “It’s one thing to understand the process, but it’s quite another to come up with a way of how you can dive into that mass data. What I care about is open minded-ness to look into things.”
The stand-out moment from the evening was listening to people say that they actively need the skills of a journalist. Words, in this age of gloomy circulations and swingeing job cuts, that you don’t hear all that regularly. Or at all, in fact.
Andy Kirk of Visualising Data talked of the absolute need for journalistic sense in a data scientist’s skill set.
I’ve uploaded this section on Audioboo so you can listen to some of the discussion.
Anyway, it was an insightful evening – and got me thinking about the ways to bridge and marry technical and journalistic skills.
There are still lots of other BDW events going on in the next few days. To find out more, click here