Using the past to explain the present

Interlinking current affairs with archives via the Semantic Web

Yves Raimond (@moustaki), Michael Smethurst, Andrew McParland and Chris Lowis

BBC R&D, Internet Research and Future Services

The BBC Archive

The BBC World Service archive

Interlinking archive content with current news events

Automated interlinking of archive content with DBpedia
User validation of automated links
Automated interlinking of our news output with DBpedia
Visualisation

We tackle this issue in the following way. We start by automatically deriving links between archive content and DBpedia, capturing the various topics mentioned in the programme and the various people contributing to it. We then use the resulting interlinks to publish a browsable archive on the Web. As automated interlinks will never be entirely accurate we put mechanisms in place for users to feed back on the qualitfy of the data, which means that our interlinks are continuously improving. We then apply a similar process to our news output, and use the resulting indirect interlinks (going through DBpedia) between our news output and archive content to drive a visualisation, which shows archive programmes related to current news events. I will now describe each of these steps in more details.

Automated interlinking of archive content

From textual metadata

dbpedia lite

From audio

Speech detection and speaker diarization (diarize-jruby, wrapping LIUM's algorithm)
Speaker identification (using speaker supervectors and ruby-lsh)
Speech to text (using CMU Sphinx with custom models)

Interlinking automated transcripts (LDOW'12)

The resulting automated transcripts are pretty noisy - the programmes in the archive cover a wide time period, lots of different genres (drama, factual, ...), lots of different speakers and accents. However they include useful clues as to what the programme is about - proper names, locations, organisations... We need to isolate these clues and infer the topics of the programme from them. Most of the existing concept tagging tools are designed to work on text that was manually written, and rely on punctuation, capitalisation, etc., so don't perform well on noisy transcripts. We therefore developed our own tool, which we originally presented at LDOW'12 and improved quite significantly since then. This tool uses the structure of the DBpedia graph to disambiguate and rank keywords spotted throughout the transcript. For example, if a programme mentions Paris and Tour Eiffeil a lot, we will pick Paris in France, as it is closer in the DBpedia graph to the Tour Eiffeil. If a programme mentions Paris and Texas a lot, we will pick Paris in Texas, as it is closer in the DBpedia graph to the Texas resource. For each programme, we get a ranked list of DBpedia tags, describing what the programme is about.

Example results

Processing archives in the cloud

User validation

http://worldservice.prototyping.bbc.co.uk

Correcting data

Automated interlinking of our news output

Live Topics API

Visualisation

Conclusion

Archives as repositories of institutional memory
Can we use archives to provide historical context around current news events?
Our solution:
- Automated interlinking + user feedback
- Continuously improving set of interlinks between archive and news content
- Visualisation