In the weeks to come you can look forward to a brand new R&D website, and within that a dedicated project updates feed, where we'll take all the content that we have hosted on this blog over the last three and a half years and integrate it with the project pages, departmental info, and the publications.  All of this will have its own dedicated search function, independent of the wider BBC web search tool.  From looking at the site usage data, and talking to a few key users, we have decided upon this pretty radical change as being the best way to get the information about BBC R&D in front of our colleagues in the wider industry and to the public at large.

Shutting down one tool before the next one is quite ready is far from ideal, but it's a minor miracle our colleagues have kept MT going as long as they have.  We'd like to extend a huge thanks to all our colleagues in the Blogs Team, the iBroadcast team and across FM who have helped give us a platform for the blog.  And while we are at it a massive thanks to the BBC News (ne World Service) team who built and maintained our website CMS over the last few years.  Consummate professionals all, they have given us a great platform to talk to you, and we are most grateful.

So, in a nutshell, here are the details- no more posts here on Movable Type.  The existing content (though probably not this post) will be migrated to the new platform, where our web site content and blog type material will co-exist.  No top level URLs will change, and we hope that all the existing URLs will map to the old stories migrated to the new platform.

This is a big step forward for us, and we hope for you.  To have a dedicated search feature will help enormously, and to have news on projects part of the same system as the details and publications offers us huge flexibility in how we present our work. Crucially this is NOT a cost cutting measure.  All these changes are specifically designed to get a more effective and better website for you- our users. 

So, see you on the other side,

Ant

Binaural sound itself is not a new idea, public performances were transmitted in some form of binaural stereo as far back at 1881. In the early 1970s dummy head microphones, like Fritz and the one pictured below, became commercially available. These are basically human manikins with microphones placed where the eardrums should be. The BBC made several pioneering radio dramas using these microphones at the time. Most notably The Revenge made in 1978. It was a 20 minute play written and performed by Andrew Sachs without a single word of dialogue. This programme is often played on BBC Radio 4 Extra, so look out for upcoming repeats. Since then there have been occasional experiments with these techniques in BBC programmes. In 2008 the Radio 4 documentary Bravo November, about a Chinook helicopter in the Falklands War, used dummy head recordings from inside the helicopter. The brilliantly imaginative interactive drama The Dark House, broadcast in 2002, used miniature microphones placed in the ears of the actors, so you could listen from each of their perspectives.

R&D's head and torso simulator microphone

So why aren't all BBC programmes available in binaural sound? These recording techniques can create great immersive effects when the right demonstrations are used. Rob da Bank showed some of the classic examples such as the virtual haircut which can be quite spooky to listen to. But often the quality of binaural recordings is not yet good enough. It is very difficult to create the impression of sounds coming from in front of your head, partly because of conflicting visual information and partly because of a mismatch between the shape of the dummy head and the shape of your own, which means that the auditory cues are not perfect. This mismatch of cues also affects the tonal quality of binaural recordings.

There are also practical issues with dummy head recording. They are not the most portable or inconspicuous of microphones. Another issue is that binaural sound cannot be reproduced over loudspeakers without special processing techniques, which are still developing. In 2011 there was a feature on the Today programme about a scientist at Princeton University who is one of the many people working in this area. (That piece inspired Today's April Fools joke that year, where Evan Davies tried to get the nation to put their hands in front of their face to block out a noise signal, much to our amusement.) Thanks to Internet distribution of programmes, it is now more feasible that different versions of a programme could be delivered, depending on whether headphones or loudspeakers are being used. It makes sense to develop production techniques that are independent of the reproduction method.

Our more recent experiments with binaural sound have used a different technique which I'll call headphone surround. This takes surround sound material produced for loudspeaker playback and using binaural techniques creates virtual loudspeakers around the listener. Instead of playing each programme over loudspeakers and recording this using a dummy head, a set of measurements (known as HRTFs) can be made and used in software to create headphone surround from any 5.1 audio signal.

Headphone surround has the advantage that existing programmes with 5.1 audio could be listened to with headphones without losing the surround sound impression. We have previously shared a radio drama and a carol concert that were made using this technique. Radio France have just launched a new website called NouvOson, which hosts a collection of programmes that contain innovative sound. Much of the content is offered in a binaural format, also made in this way. But why stop there? The use of HRTFs allows you to create a sound source in any direction, so it is feasible to go beyond established loudspeaker formats and create a rich 3D scene. We have previously discussed the concept of object-based audio, which potentially allows any reproduction method including 3D binaural sound.

There are still many open problems with binaural sound though. I've touched on the issue of incorrect cues caused by differing head and ear shapes between the recording and listener. If the HRTFs for the listener are known then binaural sound can be created specifically for them. This is possible in a research lab now and one day may be available for everyone through projects such as LocaPhoto. Work is underway to develop a standard file format for this kind of data.

Another issue is head movement. For a convincing impression of sounds coming from outside of the head, they should not move with your head but should stay fixed in place. This is only possible if a system can monitor your head movement and compensate for it dynamically. Again this is easily done in a lab these days but we do not yet have widely available technology to do this for BBC audiences.

There have already been a few mobile apps that have used binaural sound. Recently I worked with Neil Cullen, an MSc student at the University of Salford, to build a prototype that creates dynamic headphone surround on a mobile (see photo). It used a relatively cheap Bluetooth head tracking device that attached to the headphones. The quality and affordability of mobile technology for audio processing and head tracking will improve over the next few years, so this may be feasible in the future.

Dynamic headphone surround on an Android tablet

When head tracking is used, a dummy head recording no longer adequate. In Beck's recent reinterpretation of David Bowie's Sound and Vision, which last week was released online with interactive 360 degree video and binaural audio, a nightmarish head with eight ears was created to give four different perspectives depending on the chosen orientation of the video. Other approaches exist using microphone arrays and applying HRTFs to the captured signals, allowing free head rotation and individual HRTFs to be used. This is a hot area of research at the moment.

Last summer we evaluated a range of headphone surround systems in a listening test. Participants listened to a range of BBC programme clips on headphones, comparing these systems to a stereo down-mix of the 5.1 soundtrack, which is what we currently hear, and graded the sound quality of each. The test included a system that used individual HRTF measurements of the listener and dynamic head tracking. The results, which will be presented at the next AES Convention, showed that there are more improvements to be made before high quality binaural broadcasting is possible.

With colleagues in the EBU we are organising a workshop on binaural audio in May, where we hope to discuss the opportunities and challenges with the broadcast community. Meanwhile we will continue to explore the potential of this technology to create immersive headphone sound for BBC programmes.

A bit of a brief entry this week as a few of the team have been on holiday, a few have been struck down with the lurgy, and others have been organising and attending workshops. Aside from that, exciting developments in IRFS continue... but much of it confidential. Here's what we can tell you...

Chris Newell has developed a system which continually builds recommender models for news clips, following some promising investigations during the recent News Build Studio. We hope to use the models, which use collaborative filtering, to explore recommender systems for news.

Andrew Nicolaou has been planning an internal hack day for the team; writing a brief, talking to facilitators and pricing up kit. As well as this Andrew re-wrote some python code for the "tweet printer" 10% project in javascript.

In other news, Olivier Thereaux has been putting together a position paper for the upcoming open data on the W3C web workshop, with contributions from a number of interested people across the BBC. Joanne Moore has been writing a report on the privacy user studies carried out in 2012. Tristan Ferne had a good workshop with colleagues in radio, plus Alia and Zillah created a great idea on the bus back to the office. And finally, Barbara Zambrini has been writing, reviewing and refining reports for the first phase of FI Content.

Links of interest

• Our very own Chris Finch featured in a "Being At The BBC" film
• How the Cooper-Hewitt design museum link to and from Wikipedia
• VideoScribe: a tool to create stop-motion style hand drawing animations
• Versu: interactive storytelling using a mixture of branching narratives and AI
• Galatea: a classic piece of interactive fiction

When I asked people what they'd been doing this week I had a bumper crop of responses from the Snippets project team so I'll start there.

Snippets - tools for finding, snipping and sharing any moment

We have released a new version of our search page and genre filtering tool. This decision was based on user feedback and we believe the new design offers a much simpler experience. Chris F also noticed a lot of performance bottlenecks and the new release takes a much more efficient approach to displaying results in HTML. Anthony is finishing up on the keyframe tool, a new feature to make clipping really fast.

Matt H and Gareth are still working on indexing radio programmes and it is looking like we'll have close to 800,000 programmes available soon, doubling our current dataset size, so we are investigating how well it will perform and how we manage such a large set. James has been working on evaluating speech-to-text transcripts and looking into using some machine learning techniques to help us identify the most useful data in these automated transcripts.

Vicky and Jasmine, from our Salford lab, were in London for a couple of days and updated the team on their progress and plans. This week Jasmine has been "Testing and playing with accelerometers and other sensory hardware exploring the interfaces for our connected prototype. Also dabbling with a python interface for the Firmata protocol on Arduino" and exploring the "continuum of awe" with Lancaster University (no, me neither).

World Service experiment - using machine-generated and crowd-sourced metadata

Dan and Chris L attended an elasticsearch training course run by its core developer, Shay Banon. We're using elasticsearch on the World Service project to act as a fast index of the aggregated data from our users (stored in a MySQL database) and our automated tagging system (a 4store triple store). Pete's been doing "behind-the-scenes bits and bobs", Mark's been gathering some stats, I've been writing some words, one of our top users has been helping us plug some gaps in the archive and Michael's been trying to get a feel for the shape of the archive by looking at the data so far...

R&D Website - developing a sustainable and world-class R&D web presence

We are getting very close to making the beta site public. Olivier has been reviewing quotes for security and accessibility testing, adding and tagging content and talking with people across R&D, while Matt P and Chris Needham get the infrastructure ready to host the site on bbc.co.uk. Meanwhile, the team at Kite are helping us on the next stage of development.

VistaTV - using real-time audience data

Chris Newell has started a study of temporal changes in the performance of collaborative filtering - "The recommender models we build rely on having a representative sample of audience data. The question we need to answer is how long after broadcast is this achieved?"

Developing new ideas

There were at least three workshops, that I know of, developing new ideas and projects. Unfortunately I missed all of them because of other forms of meeting people. On Wednesday most of the team were at a workshop imagining future experiences and services based on metadata, facilitated by our friends at Superflux. Andrew N says "I learnt some great ways of structuring these sorts of exercises and how producing tangible models can be a powerful way of exploring and explaining ideas". And on Thursday there were two more, very nearly overlapping in time; Libby ran one on media technology for kids and Barbara ran one thinking about new forms of radio.

All the things that didn't fit...

Libby has been looking into getting more music data and doing some strategic things. Barbara and Chris Needham have been working on documentation as the FI-Content project wraps up. James says "The dashboard is coming". Andrew N got an interactive TV prototype, from colleagues in the TV Platforms team, working with a remote control gadget. And he got an LED to flash using a Raspberry Pi. And we shouldn't forget the impact of our older work. Last week's "6 Music Live" season used RadioVIS, which we helped invent and develop, to send out pictures of the bands to radios with screens.

Links

Lastly some links from the team. Well, actually no-one sent me any so they're just links from me...

I watched a programme on BBC Four about Google Books and discovered that in Barcelona they keep their supercomputers in deconsecrated churches.

Debate was sparked in the team after reading this article on how AMEE use Trello to run their projects. We tend to use a combination of Trello for tasks and Gitlab for issue tracking but end up with some redundancy. If and when we settle on a stable method, maybe we'll write something about our workflow for building protoypes.

And from Make magazine, the Offbot: "Offbot contacts you once a day and asks you what you've been up to. It's not a task management tool. It's more a tool to reflect on your own work, and how your team works together. And because it contacts you out of the blue - it emails you at random times of the day - most of the time, people reply with the first thing that comes into their minds.". Look out weeknotes, here come the robots.

Thus, volunteering to work on our weeknotes was a rather selfish way to know a little more about what the team had been up to: new radio functionality in snippets, the world service archive taken over by radio drama enthusiasts, internet of things, conferences, website, and more!

The problem with squirrels

Along with bug fixing and improvements to the existing site, Team Snippets has started the first phase of our new Radio functionality. This is the ability to quickly search radio shows by programme description, contributor and episode title and then facet those results.

Matt H has been collecting data on the project, and James has been working on the transcode service.

The big internal debate this week was on the best way to present this functionality - do you force users to pre-select or do you combine search results from all the datasets? Next week we’ll be roughing out the work in anticipation of prototyping effort in an upcoming sprint.

... through the archive

World Service Radio Archive Prototype

Tristan and Michael have been watching with fascination our World Service archive prototype being picked up by the radio drama community, starting with someone creating a curated collection of the radio drama and plays in there.

Andrew N, hacking on the website

R&D Website

Work is continuing on the R&D website. Akua has been patiently editing content, while Chris Needham authored pages for some of our past projects, notably RadioVIS.

Meanwhile, Andrew has been refactoring the CSS on the R&D website and making use of Microsoft’s recently upgraded testing options.

The Internet Of Cables

Internet of Things, Radios and other devices

Jasmine sent me an enthusiastic summary: "This week", she writes, "we have been scoping, exploring, building. We are in an initial phase of research into building a connected sharing and saving device for radio listeners."

This has sparked the exploration of how a single-touch contextual interface (with feedback) could be embedded in BBC programming, and as a route to network/ brand engagement.

We've also been testing and networking microcontrollers via bluetooth.

Also in hacking-devices-land, James is off to the European Broadcasting Union’s Radiohack event in Geneva on Monday to talk about an open source radio contribution link tool (OpenOB), Raspberry Pi radio development, and to learn more about some other projects being undertaken by the maker/hacker community to make radios.

Talks and Conferences

on Thursday, I joined Jory of html5audio.org on stage at the AES “Audio for Games” conference.

We kept our talk about the history of audio on the web to a minimum, preferring to spend most of our hour slot showing cool demos such as Step Daddy, the Infinite Jukebox or Fist Pumper... Most of which currently only work in Chrome by the way, and the latter on iOS6 devices - but the fact that these are using an upcoming standard still being implemented by other browser engines was, well, the point.

Pete has been helping to prepare a presentation for the British Library ‘Opening Up Speech Archives’ conference, which he is attending with Chris Lowis and Theo today.

Lest we forget

In Other News...

• Matt Haynes has also been helping BBC News stabilise their triplestore for the next round of “Labs” prototyping.

• Andrew N has been thinking about tools to wireframe TV-like experiences using laptops, web browsers and remote controls.

• Tristan, Michael and others had a meeting with the Netherlands Institute for Sound and Vision (Nederlands Instituut voor Beeld en Geluid): lots of crossover with their work on making archives available and useful online.

• Barbara, ChrisG and others have been working on the preparation for phase 2 of the FI-Content mega-project. This apparently involved a marathon-like teleconference. Oh, European projects have all the fun...

Links

Our curator of links this week is... Tristan:

• Dan Hill writing on the design process of Little Printer

  "That's material reality!", Schulze laughs. "You realise that, you can't just say "In the future all televisions will speak to you, because they'll recognise your face', because it turns out that if you've got a beard, or the resolution of the eye placement is too coarse, it won't work."

• The Economist explains why they like federated approaches to authentication to government services

• On the non-availability of radio drama

Vista-TV - exploring applications of real-time audience data

Status: building infrastructure

Chris Newell's exploring the capabilities of the ViSTA-TV project stream engine, based on Storm, an open source distributed real-time computation system. The stream engine gives us the ability to process audience logs and subtitle streams in real-time, rather than offline as in previous work, to create interesting live services and applications. Libby's been getting to grips with the Vista-TV code and fixing a few bugs.

World Service archive and ABC-IP - archives with machine-generated and crowd-sourced metadata

Status: researching

While Chris L has been busy fixing bugs and improving the search, Pete has been showing the World Service Radio archive prototype and some wireframes of new potential improvements to some volunteer test-subjects and this has helped identify user-experience issues, inform next steps and inspire future features. Meanwhile, Tristan has been doing some research and writing with Michael around tagging, controlled vocabularies, participation and agency.

Also, Yves deployed our first try at a topic extraction service for web pages.

Status: polishing

Rob and Chris F have been focusing on perfecting the new Genre Filter as part of the Snippets search faceting upgrade that the Snippets team have all been working on. It's going well although some parts might need design tweaking.

FI-Content - exploring identity and personal data on connected TVs

Status: quantifying

Barbara, Chris G and George were thinking about revenue streams on the user data dashboard and authentication use cases that we prototyped during the project itself. They were trying to quantify how we'd exploit the work we've done so far and what the possible market impact of such things might be. Not easy.

R&D website - re-building the R&D website

Status: bug fixing

Chris Needham and Andrew N were completing the last few features and bug fixes on the beta version of the R&D website. Olivier's been pushing the project through some of the last process hurdles between us and the release, planning the next steps, and trying to get more content in before we launch the beta in public.

Internet of Things - researching how the IoT works for media, entertainment, storytelling, games and toys

Status: exploring

Vicky and Jasmine are continuing to explore and prototype ideas framed on our research questions: - How can we give non-digital objects behaviours to make stories more captivating and absorbing? - How can we make media management more physical? - How can we bring characteristics of theatre into the home? - How can we get our audience to shape and build their own experience?

Cefn Hoile of Shrimping.it will be helping us to prototype some of these use cases.

Out and about

Status: talking and showing

Bristol: Theo and Libby attended Digital Bristol technology day - showing off the team's work to others in the BBC and beyond.

Paris: Olivier participated in a workshop organised by Europeana and INA on opening data and content for cultural institutions. It was great to see that the BBC had a lot to share about how we've dealt with opening our data and APIs, and a lot to learn from the vanguard of open museums and libraries.

London: Theo and Tristan went to the UCL HCI MSc project day to present some project briefs to the students. The challenges were: Authenticating TV, Navigation with Noisy Data, Agency and Participation and Exploring the Mood of Programmes.

Interesting links

• Bletchley Park Podcast
• Our own Chris L's article on storytelling R&D at The Literary Platform
• On Google and design
• Tim Bray's review of OAuth
• A bunch of links from the Europeana workshop: OpenGLAM , thoughts about a prototype/user test by the Danish SMK , open images/video archive from Netherlands Sound & Vision institute
• Netflix culture

We had an enthusiastic contingent at the iPlayer Connected Studio too, where Denise, Jana and Sam's mood metadata formed the basis of two pitches and the Snippets team pitched a number of product ideas.

Vicky and Jasmine have been planning for the next expeditions, making a short video pitch and application to the Nesta Digital Makers Fund. George pitched FI-Content phase 2 to the commission, and on Thursday we got the splendid news that our proposal was being funded.

In between planning the next World Service Archive prototype sprints, Tristan has been experimenting with generative logos:

Meanwhile, in the 'making ideas reality' department, ChrisL and Dan have moved the World Service Archive search from Solr to Elastic Search. From Chris: "We did this primarily due to Elastic Search's support for nested documents which match our use of tags for episodes and our desire to include user votes on tags to influence ranking in the search engine. A neat side-effect has been easier faceting and cleaner integration into our Rails app thanks to the Tire gem".

Pete has been preparing materials and organising informal user testing to help clarify the UX thinking around search, and Yves has been creating a service extracting named entities from web pages, using the same disambiguation algorithm as used in our tool for automatically tagging speech audio.

Chris Needham and AndrewN have been working on the R&D website. Chris has been implementing the new blog, which is built using Locomotive CMS. Andrew spent a lot of time listening to the sound of the site (what a lovely thought) to fix accessibility issues with its structure.

Chris Newell added an embedded web server to the ViSTA-TV stream engine for monitoring and demonstration purposes. He says: "I chose Webbit, built on Netty, because it supports server-side events and Web Sockets - ideal for delivering our live data and results to browsers."

Rob reports that "Snippets finally went live with its new transcode formats. This should make programme makers lives easier by offering a range of smaller file sized files perfect for rough cuts when you're on a slow network. In addition we've gone live with our audio-only transcodes which allows Radio producers to easily create Snips from TV shows for broadcast use. Quite a few shows on Radio 1 are already using Snippets as a way to talk about last night's TV, so we're pleased to offer this new functionality." James, meanwhile, has been working on evaluating new tools for extracting information from radio programme archive material, and spent a little bit of time on a prototype application to extract locations from topics extracted from subtitle data.

Contributing back to the idea commons were Jana, preparing of datasets (TV programmes plus metadata) to share with universities for research collaborations; Olivier, who is speaking at a workshop on open data and APIs on the 29th of Jan, and co-presenting a talk on web audio at the AES "audio for games" conference in early February; and me: I'm on a panel on Tuesday at Digital Bristol Week, and Theo and I will be demoing some of our prototypes there.

Links for the week, both from Olivier:

• I'm learning R, so some of my chosen links are about that: Google's R Style Guide and RStudio: Software, education, and services for the R community

• Tabula, a "PDF table liberator"

To generate the initial ideas, a few of months ago we held a workshop involving a mixture of BBC and non-BBC people. Four proposals came out of it:

• 'Heartbeat EPG' - a heatmap showing what people are watching now
• 'Infinite Trailers' - a 'channel of continually playing clips of available content on iPlayer, which learns what you like, presenting you with better suggestions depending on whether you skip or like an item.
• 'Radio Dashboard' - a dashboard of real-time radio-related data for the Audio and Music team to quickly see any problems
• Taking iPlayer to you - a variety of techniques for integrating iPlayer with your online life more closely.

Of these, the first three each seemed to have a core idea that we could develop into a prototype in a short period of time. Dan Nuttall, Chris Newell, Ant Onumonu, Andrew Wood and I got together as a team, set aside a week for each idea and created a prototype for each. In practice things took a little longer than we had hoped, but we now have prototypes that we can demonstrate to project partners and within the BBC. We hope to develop some of these into public prototypes in the future.

I want to thank the workshop participants, and the people we've talked to within the BBC who've offered very helpful advice on the directions to take.

In the rest of the post, I've asked the team to describe what we've made in a little more depth, including some of the technologies we've used.

Chris provides some background to the underlying data: "As the ViSTA-TV system is still under development we needed a simple way to extract interesting real-time statistics from our media server logs. Luckily we had an existing system which maintains a table of open iPlayer streams in memory from which we could collate statistics for channels, programmes and user platforms. The only limitation we had was time - it can take 20 seconds to download the minute-by-minute logs and update the table during busy periods, leaving 40 seconds to collate the statistics. This was enough but the ViSTA-TV system should provide a more efficient approach."

Heartbeat EPG

In some ways this was the simplest of all the ideas, but also very powerful: show in real time what people are watching on iPlayer - both 'simulcast' (programmes watched online at broadcast time) and 'catch up' (programmes watched after broadcast).

During our initial discussions we thought it would be interesting to show updates without reloading the page, and this became the core of the prototype. Andrew decided to take a clean, simple 'list' approach to the design, and included a way to show the proportion of the audience watching each programme, highlighting the most popular item.

Andrew says: "The design pattern for this was essentially a table with the top 20 results. First quick sketches were made and discussed with the team, this was a very quick and iterative process due to time scales. Once the team had agreed on the route I passed mocked up Photoshop assets to Anthony for the front end build. I then had some time before our next sprint to look at a more conceptual route for the UI. Using colour and scale I mocked up the "ball interface" We revisited this for the Radio Voyeur sprint. I also met with Chris Kimber to discuss a version that would work as a dashboard for his team. This is more specific as it shows 2 columns - one for Live content and one for On demand which I have art-worked. The rest of the design team within IRFS have seen this work and a possible outcome is to develop it as a 10% project for responsive design experimentation."

The underlying APIs from Chris that we needed for this prototype were already in place. Dan says:

"The backend needed to be able to handle multiple incoming data streams and to push that data out to clients as quickly as possible. Node.js was the perfect match for this. The application polls Chris' API for new data points. When new data is found, we search for applicable metadata using a combination of BBC Programmes' API and our own programmes' Solr powered datastore. Once the dataset is complete, connected clients receive the new information over WebSockets, using Socket.IO. Socket.IO has a very straight-forward API, allowing us to pass a simple JSON object to Ant's client-side code each time the data changes."

Infinite Trailers

The idea for this prototype was a continuously playing channel of BBC TV videos available on iPlayer, which the user could choose from to play now, or indicate 'like' or 'dislike' to get better suggestions. The initial idea from the workshop was to have skip or play, but this didn't give the recommendation engine enough information - just "dislikes".

The team decided to create a mobile application for this prototype: Ant in particular was interested in looking at HTML5 video for mobile. This turned out to more difficult than we initially thought, as browser vendors on mobile have different interpretations of some aspects of HTML5 - but we still managed to create a prototype that works on Android phones, iPads and as a bonus, in Chrome, and Ant says he "learned a lot".

Andrew says: "My starting point with this sprint was a smart phone interface - the concept is a passive experience for the user - flicking through trailers - so the design needed to be simple and uncluttered. A simple title screen labels the content then fades off, the user has the option to move on, or add the trailer to their playlist. Once the front end was in development mode I had some time to look at other options - the egBox platform (a two-screen prototyping system) seemed appropriate to the concept - so I mocked up the 2 screens (trailer and playlist) - as with the EPG longer term development would implement responsive design templates."

Dan says: "Ensuring that each recommendation was backed by a video clip was the key to the liveliness of this application. Chris' API was polled every few minutes to keep the pool of available programmes up-to-date. As new programmes are found, their identifiers ('PIDs') are entered into a multi-state system, backed by a SQLite database. First of all, metadata is applied using similar techniques to the EPG application. With this data, we then query the Snippets API to see if the programme has been captured internally. If this is successful, we schedule a snippet to be taken from the first minute of the programme, of around 30 seconds in length. As these clips become available, they are downloaded and put onto the web server. When all the clips are downloaded, the recommendation data is copied to the server, refreshing the available programme list."

Chris: "The recommendations were obtained using the client-side recommendation engine we use in an existing prototype recommender system called Sibyl. The engine provides a list of available programmes which is ordered to reflect the preferences of the user, based on programmes they like and/or dislike."

From Ant: "Since the videos from the Snippets Transcoder service are provided in h.264 we could deliver the trailers to users with a HTML 5 Video player that worked on the Desktop and Mobile. We also made use of Local Storage to store the user's playlist. Video playback currently only works in Chrome, Safari and Internet Explorer. Support for h.264 in Firefox is in the pipeline and can be accessed via nightly builds. We wanted the videos to auto play once the application had initialised but this only worked on the Desktop. We also noticed on the iPhone that the video played only in the native player. This wasn't an issue for the iPad."

Radio Dashboard

This prototype was designed for internal BBC use - to give a technical or production team an overview of audio IP streams, but also allowing them to dig into the data in detail to identify potential causes of increases or decreases in traffic. The design goal was therefore to show as many pieces of information as possible in a way that allowed the user to make sense of them.

Andrew comments: "The starting point for design was to implement the "ball interface" from the EPG design sprint earlier. The "balls" are vector graphics of the Radio brands - which allowed Anthony to scale them. The intention was to make something that looked and responded dynamically - so when the page updated - the balls would grow / shrink and move. A scale had to be set for the smallest ball size based on legibility."

"The user can click on any of the "balls" (radio programmes) for more contextual information - the design pattern used here is a simple lightbox which displays a graph showing viewer figures from the last hour on desktop and mobile."

The third screen shows even more detail - the number of users joining and leaving the channel in a five minute period and where they go, the number of tweets in that period, and what track was playing, if any.

Dan says: "The backend of this system was a frankenstein-build of the previous two backends. We used the Node.js server from Radio EPG and added Chris' additional data streams. We complemented these with data from other BBC APIs; social media information from Buzz and music metadata from our internal playout system. It was important for data points to be emitted only when fully-formed with data from all the available sources. We added the multi-state system from Infinite Trailers in order to ensure all sources are integrated before updating the client in real-time. Every five minutes, an aggregation process iterates across the recent data points and emits the collated data, producing the 'blobs' in the activity graphs. Like Infinite Trailers, the data is stored locally in SQLite. This allows us to offer the previous 1 hour of data immediately on connection, while keeping users up-to-date as new information is received."

Ant: "The frontend for this application consisted of three charts built with SVG and HTML. Initially we used a JavaScript library called D3 but switched to Raphaël because it was easier to work with the vector assets once they were converted into Raphaël JSON. One challenge we encountered was displaying the bubble chart in a random manner without any overlapping. This would have been easier if the logos were the same size. We ended up using a technique called circle packing to ensure the logos didn't overlap whenever an update occurred. It should also be noted that we found animation performance was greatly increased by animating parent divs and not the SVG elements directly."

Conclusions

Due to the rapidity of the prototyping process we chose, we cut some corners and made some assumptions, but as a team we feel like we learned a lot because we made working systems. We'll take this new information as an input to the decision on what to build for our main application in ViSTA-TV, but we can also show these prototypes to colleagues within the BBC to see if they are interested in the approaches we have taken; and use the technical knowledge we've gained from this work in other projects too.

Matthew Postgate on stage at the R&D In Session event

In a day-long event at the BBC’s Radio Theatre in New Broadcasting House, chaired by BBC News correspondent Nick Higham, we staged and filmed seven lectures and interviews that aimed to ask where broadcasting is heading from a technological and societal point of view. The lectures were loosely grouped around storytelling and the year 2020, were kept deliberately open to encourage debate.

Today, we are publishing these lectures and interviews online to allow them to be viewed by an audience who weren't present, and to encourage debate, spark ideas, inspire collaboration, and help us gauge what viewers and listeners think about some of the topics discussed on the day.

Frank Rose, author of The Art of Immersion and West of Eden and longtime contributing editor at Wired.

He was followed by my colleague, Graham Thomas, who asked ‘What's next? Beyond HDTV and 5.1 audio’ giving examples of BBC R&D's contributions to various initiatives, from visual effects and high resolution television, to content metadata, 3D audio and the newly established Audio Research Partnership.

Graham Thomas covered a wide range of R&D work

After a short break, Lieven Vermaele, Technical Director of the EBU, explored how the EBU and the research and development community could work toward a worldwide digital TV standard. We also mixed these live interviews with a recorded one from historian and filmmaker, Dan Snow, allowing him to share his interpretation of history as metadata while he was half way around the world. Louisa Heinrich, Director of Strategy, Fjord, then spoke about the choreography of personal devices, how services are influencing our behaviour and how interaction between users and content is evolving.

Lieven Vermaele, Technical Director of the EBU

Historian and filmmaker Dan Snow appeared on video link

Louisa Heinrich, Director of Strategy, Fjord

In the final talk, I rounded the day off with an overview of the day’s key themes and ideas expressed, and a discussion about the role of R&D in the information age and the evolution of broadcast systems.

We hope that the ideas expressed encourage dialogue within the industry, and help to set the agenda on the challenges and opportunities of digital technology.  We work best when we collaborate with others, but sometimes our work at early stages takes place behind closed doors (while we work up our hypthoses before taking them to a wider audience) and so we are keen to explore opening up discussion with people around possible areas of future interest at an early stage.

We're really excited by the format and aim to make BBC R&D In Session a regular event, and would love to hear from you about what works and doesn't work.

The videos have been licensed under a Creative Commons licence to allow creative reuse, and to encourage sharing and discussion. Thanks to all speakers and everyone involved in putting the day together. I'm keen to hear your thoughts in the comments.

Your position paper should attempt to address one or more of the following themes:

• Remotes
• Content Discovery and Engagement on Companion Screens
• Discovery of Devices
• Social Network and User Generated Content
• Personalisation and Recommendations

Our workshop seeks to increase the volume and quality of HCI research and innovative practice around user interfaces for television. Internet connectivity is driving a rapid increase in the range and scope of interactive experiences on the TV platform and it represents an exciting new opportunity for developing new HCI practice and methodology, as well as innovative forms of user experience. Through this workshop we hope to:

• Share current research and practice to foster and strengthen a community of researchers in this field
• Develop an ambitious research agenda to enhance user experience and HCI/User-centred practices in this field
• Promote TV experiences and interfaces as a distinctive and important theme for HCI researchers by presenting exciting work from this field

Further details on the themes of the workshop, method of submission, important deadlines and organisers can be found at the workshop website. We look forward to seeing you at CHI this year.

Radio drama production workflows have developed over the last 60 years, but the productions are almost always a mixture of microphone recordings of live actors' performances, recorded Foley effects, sound effects from libraries, and music either from a composer or a music library. These sources are then mixed together to create a stereo (two channel) file which is played out for the broadcast. Recently a few radio dramas have been produced in 5.1, for example earlier this year Private Peaceful was mixed to 5.1 and then rendered to binaural using virtual speakers. This involves the creation of two mixes, a stereo (two channel) file and a 5.1 surround (6 channel) file. The creative process of mixing to these formats had to be performed twice, once in stereo and once in surround.

 

Mixing Pinocchio in BBC R&D's Listening Room

 

For Pinocchio we did something a bit different. Steve, the sound designer, and I mixed the recordings, treating the sources as "audio objects". This means that rather than making a stereo mix by panning sounds somewhere between left and right speakers, we forgot about speaker locations and positioned audio objects at locations in space. This means the final mix, rather than being a stereo or surround sound file, is actually a set of audio objects, each with accompanying metadata to describe things like the source level, azimuth, elevation, distance, etc. This data is then rendered to speaker channels before broadcast (in the case of this experiment) or at the listener's home/device (potentially the case in the future).

There are a number of potential advantages to describing audio scenes in this way:

1. Speakers/listening devices become independent of the mix. This means listeners can put as many or as few speakers as they want, wherever they want. Or they can listen using headphones or a mono tablet speaker. The client system knows the listening environment and can render the scene in the way that provides the highest quality of experience.
2. An object based scene representation can be rendered differently for different people. For example, people with different hearing abilities may want a different balance between foreground and background sounds. There is also a lot of potential for applications like Perceptive Media when describing scenes using an object based audio approach.
3. Interactivity can be enabled. When audio scenes are comprised of different object, those objects can be fully interacted with in order to provide computer game like experiences. This opens up a lot of user experience research questions.

The microphone set up for some of the recording of Pinocchio.

This approach has the advantage that we could mix Pinocchio in 3D, placing sounds wherever we wanted (above, below, in-front behind etc.). For exmaple, there is a scene where Pinocchio was swallowed by a shark. Steve and I were able to position underwater sound effects, as audio objects, all around the listener, creating a highly immersive experience.

We were able to monitor the mix in our listening room, which was equiped with 26 loudspeakers. This set up has more loudspeakers than are likely to be available to the average listener so we used a rendering system created by IOSONO to create the stereo mix, which was broadcast on Saturday, a 5.1 mix and the 24.2 mix that the production team used to monitor the mix during post-production. So although you can't yet hear a full 3D surround version of Pinocchio, you can download a (AC3 or MPEG surround) 5.1 version to hear it in surround sound. Instructions for how to do so are given here.

This is still very experimental and we'd love to see your feedback in the blog comments below.

As you know, the halfRF Radio Camera was used for the first time for a live broadcast of The One Show yesterday evening. JohnB, Adrian, Tuck, Kevin and I spent a very cold day setting up the equipment in the area around the Stage Door of TVC and in the SIS Live outside broadcast studio OB7. There was a great deal of interest in the system from everyone who saw it – particularly the SIS Live crew and cameramen.

SiS live cameraman using the halfRF rig in preparation for a live link at TVC on The One Show

As expected, some of the cameramen said it was a bit big but were happy to understand that it is only a prototype and a properly engineered solution would be much smaller and ergonomically better. They did say however that the balance on the Sony camera was in fact quite good. Martyn, our cameraman for the evening, seemed to be quite happy with it.

The system was completely stable with no breakups at any time and the batteries, which were powering the transmitter, camera and lens and the camera control receiver lasted about an hour.

Whilst we’ve initially released this based on iPlayer content, our goal is that it will help people find new entertaining content from the BBC Archives. Users can search for a programme that they know of and then find programmes from the archive which have similar (or opposite) moods. We hope to further extend this by adding topic similarity too – allowing users to find programmes with not only similar moods but also about similar things.

For detailed technical info on how we do this, I’ve listed some of our published papers at the bottom of this blog. If you have any queries please email us at

multimedia.classification@rd.bbc.co.uk

Further reading;

• Affective Classification of Large Scale Broadcast Archives; BBC R&D White Paper 201 
• A Large Scale Experiment for Mood-based Classification of TV Programmes; BBC R&D White Paper 232
• An Affective Interface for Mood-Based Navigation; BBC R&D Blog 

Some people may feel that they have been hearing Jingle Bells on a constant loop since 2003, but at the BBC the festive season began on Tuesday night on the One Show with the switching on of the BBC Christmas lights. At Television Centre Alex, Matt and their guests were having a merry old time at a traditional Christmas market and hanging out with Rudolph and his reindeer pals. Meanwhile, up here in Salford, BBC R&D got together with the One Show team and an army of volunteers to create an amazing 150ft long Christmas tree light painting live on-air.

The One Show Christmas Tree Light Painting

Video and more after the bump!

View from the roof of torch-wielding volunteers

R&D's Brendan Crowther has made a short film about the project: 

The light painting effect was generated using our software system and sent down to the outside broadcast van outside where the local production team were based. An HD satellite link was set up to transmit the footage down to Television Centre in London. Shortly after 7pm, a well choreographed and rehearsed team of volunteers set off around the MediaCityUK piazza with their torches, as I took on the highly complex task of pressing the space bar at the correct time. We were all a little nervous as live television is not something we often get involved in at Research and Development, but it went off without a hitch.

R&D's Christmas Bauble Quality Assurance Unit

To create the light painting effect we used a simple piece of software that I'd written in the summer. It takes in an HD-SDI camera feed and builds up a painting in real-time using the brightest pixels in memory. The software was written using Cinder, a free and open-source C++ library designed for creative projects. It's a very straightforward process (the more advanced part of Little Sun was the object-tracking and the stunning WebGL-rendered online experience), but it can create a remarkably dynamic and engaging experience for the light painter using it.

Over the last ten days we've done some development and testing to make sure that the software is broadcast-stable. We spent an evening or two experimenting with a variety of torches and manoeuvres on the ground. In order to get suitably diffuse coloured light from LED torches we taped ping-pong balls over the end of the lights, which was a good few hours work for the One Show team, but looked fantastic. There were one or two dodgy runs in R&D's first experiment but on the night it was looking great. Well done to all from BBC R&D who worked on this, especially Max Leonard, Anthony Churnside, and Matt Shotton, and also congrats to The One Show for such a spectacular show all round. I'd also like to thank all the torch-bearers who braved the icy winter conditions for the cause. I hope you were as pleased with the result as we were. Finally huge congratulations to the R&D team responsible for the halfRF MIMO HD Radiocamera which was used for the coverage outside Television Centre during the show.

UPDATE: The more attentive One Show viewers will have learnt that all of the reindeer must be female, because the males lose their antlers in Winter. Therefore my statement in the first paragraph should not read "Rudolph and his reindeer pals" rather "Rudolph and her reindeer pals". I offer my sincerest apologies to any reindeer who I may have offended. Thanks to AJM for pointing out the mistake. Merry Christmas!

We have been itching to  try out the ‘halfRF’ MIMO HD Radiocamera outdoors for a while , but bad weather and other commitments had so far been in the way.  It is one thing to know that your system ought to work well outdoors, but it’s much better to have tried it and imminent programme use really concentrates the mind, so last Thursday several of us braved the cold and did some outdoor tests at TVC.

R&D has a rather elderly transit van which was previously used by our Spectrum Planning Group to check coverage of transmitters. The van is equipped with useful things like benches and mains power making it an ideal base for outdoor radiocamera tests.

We initially set up just behind security at the front of the car park. The antennas were mounted on lighting stands about 8 feet off the ground and 20 feet apart. We walked the camera through the covered walkway next to Studio 1 and around the doughnut (the open circular area in the middle of TVC). We didn’t expect this arrangement to work brilliantly as there were number of obstructions that we thought might cause problems. We were in fact very pleasantly surprised, as there was only one small break up with this arrangement. We then rearranged the receive antennas so that one was much higher up and had a clearer view of the area to be covered. We repeated the walk and this time the coverage was absolutely perfect with no breakups at all.

In the spectacular  there will be a stage in the upper car park and we thought we might be able to mount our antennas on some of the lighting rigs, so we moved the van to the upper car park and set our antennas up there. The performance again was perfect around the car parks and in the doughnut. 

UPDATE: Ant here- I've just been over to see the team setting up as the last of the daylight fades- all going well- the radio camera tech is generating lot's of interest among the SiS-Live OB team.  Signal strength is excellent in all the filming locations- this should be good!