Working with Giga-Pixel Photos and Websockets

Published on March 10th, 2014 Link

This is a post written for Native Digital regarding the work I performed for them for our client Rekorderlig Cider

The Rekorderlig Gigaphoto app was a unique technical challenge for our team. A high resolution photo such as the image featured on this campaign requires careful planning for delivery to the many competition players.

These challenges are similar to what large technology organisations such as Google, Microsoft and Apple face when proving maps to their users. High resolution images need to be cut up into squares, called titles, at a variety of zoom levels, so that the entire image does not need to be loaded all at once. Imagine trying to download satellite images for not only your local street, but the entire world. The Rekorderlig campaign uses a very similar tiling technique, allowing our visitors to explore the intricate details of the Stockholm panorama by zooming and panning around to discover the hidden clues and characters without downloading the entire multi-gigabite high-definition photo.

The concept behind the Rekorderlig Gigaphoto promotion involved releasing the name of the new cider flavour only once it had been found in the panoramic photo. There were also many other hidden objects which when found gave the visitor an opportunity to claim a prize. To prevent anyone from cheating – wrongfully discovering the location of the new flavour, or any of the other hidden objects – our team devised a method of transmitting prize data which did not reveal any locations. This involved maintaining a list of prizes and their locations on our server, and coding the application to send the coordinates of any click on the photo back to this server for comparison. In this way, no one could artificially discover any of the locations of hidden objects without clicking on every part of the photo.

To provide a user experience that appeared responsive and fast, our development team chose to use a relatively new technology known as Web Sockets. If you were to imagine a telephone conversation, the older, more common method of communication back and forth between a visitors browser and your server would be like calling the other party on the phone, delivering a sentence and listening to a reply before hanging up again. Web Sockets would be more like a constant conversation, sentences going back and forth as required, and holding the line while nothing is being said as opposed to hanging up. Since we required the coordinates of every click or tap on the photo to be sent to our server, dialling in, then hanging up each time wouldn't be terribly efficient or responsive, resulting in users having to wait for a second or two before they were told if they had discovered a prize. Using a Web Sockets setup gives our visitors a near-instant response as they can tell us where they have clicked and we can respond telling them the details of a prize if they discovered one without "redialling" each time.

Another benefit of using WebSockets is that our server can send messages to any visitor, which allowed us to provide live updates when prizes were discovered. This creates a sense of urgency to the visitors, seeing that other people are actively discovering prizes, enhancing the campaign concept.

Functionality like this is usually only found in large multiplayer games which rely on other, less robust technologies such as Flash, which isn't supported on many handheld and mobile devices. Web Sockets, being a relatively new technology provided quite a unique challenge to our development team, requiring careful research, experimentation and testing.

* * *

At the time, the hosting PaaS our company used – Heroku – did not support Websockets. A secondary server was deployed on RedHat's OpenShift PaaS for this campaign. Additionally, performance issues with older browsers using XHR fallback were not addressed resulting in a somewhat frustrating experience for some users.

When our client revisited the concept for another hunt challenge the following year, I developed messaging system ground-up, which is based around Server Side Events, Rails Active Record and Postgres Listen statement, along with EventMachine to process events in the background and pass messages between processes living on a multi-server environment.

Server Side Events are a great alternative to Web Sockets where most of the data is being pushed from the server rather than the client. The functionality can be replicated with standard XHR and long lived requests. Thus a far greater range of browsers can take advantage of it. Unfortunately, client-initiated requests had to be handled by XHR for all browsers in this system.

Since Heroku has now announced support for Web Sockets, the system can be updated for support, again using SSE as a fallback