Recognizing commercials using the Alphonso API
Liberty Global organized the Hack & Play Appathon in Ziggo dome on September 15th and 16th. More than 20 teams of hackers, designers and programmers were invited to create an app or a game for the Liberty Global product Horizon set-top box. Team Trifork joined with Dennis de Goede (Design & Frontend), Tony Abidi (Devops) and myself (Front & Backend).
Alphonso added another challenge to the appathon: Create the best integration with the Alphonso platform. Integration challenge? Sounds like a Trifork challenge to me.
Who enjoys commercials on television? Hardly anyone we spoke to, including ourselves. On the other hand, who dislikes funny kitten movies? Again. Hardly anyone, right! So we created our concept: KitMercials.
We would use Alphonso to listen for commercials and trigger our app to display funny YouTube videos during the commercial. The live TV feed would still be displayed in the top corner and extra information about the commercial could be displayed as we received it from Alphonso.
The frontend would be responsible for the creation of the Room, the websocket connection. Then, we would create an Android app that must discover the room and connects to it. The app then listens to the sounds from the microphone and sends it to Alphonso for analysis. When Alphonso recognizes a commercial (which it does very good I must say!) the app gets the extra data like brand, description and optionally logo, website info etc. The app sends this data to the frontend which can trigger the KitMercial interface and starts playing the YouTube videos selected.
Within 24 hours after the start we were live. Live stream on one of the Dutch commercial channels and automatically the kittens were on screen during commercials. And the crowd loved it.
With the excellent support from John Gee and Naren from Alphonso the first price was ours to get and Alphonso agreed: the first price in the integration category went to Team Trifork!