Problem:

Virtual Reality has failed. Sales of VR headsets have flatlined, never moving at the logarithmic or exponential growth pattern that the technical giants like Facebook had expected - rendering massive $4b acquisitions like Oculus obsolete. While there is a small, but dedicated, group of users who interact with Virtual Reality, this market of 18-25 year-old gamers has neither grown in the years since the launch of Oculus nor has the percentage of users grown to include the other 96% of Facebook users.

In 2015, Facebook Singapore launched an innovation acceleration program studying solutions to the limitations of VR, including ways to enable higher user-engagement in capturing and sharing 360° VR-ready content on Oculus and the main Facebook platform. The problem facing the platform was that, outside of a niche early adopters, most Facebook users did not own a 360° camera or interact with the format.

Enabling the average Facebook user to capture this ‘SocialVR’ content was critically important to expanding the user-base of VR and 360° content. This VR innovation research group looked for solutions across the technical spectrum, including low-cost disposable 360° cameras, embedding mobile 360° cameras into popular smartphones, as well as strategies to integrate 360° cameras into the world around us as part of Facebook's push into the Internet of Things (IoT).

The specific hardware requirements of 360° cameras such as extreme wide-angle lenses and heavy graphic processing requirements limited the development of both the low-cost and mobile-embedded 360° cameras. Efforts to design cameras that fit inside common smartphones floundered as the wide-angle lenses, large graphics processor, and high battery consumption would ruin the pocketability and price point of the phone. These specific limitations continued to underline a key user behavior: the only camera we carry is the smartphone in our pocket, but this smartphone couldn’t contain a 360° camera.

By 2016, and with Facebook’s blessing, the department lead and SVP at Facebook Singapore, Peter Riley, spun off the only remaining research area, IoT enabled 360° cameras, into an independent company. As Co-Founder, Chief Technology & Product Officer, I joined him in launching Glorious Labs with a goal to improve the world's ability to capture and share memories in this new immersive format.

Challenge:

Designing a new product from the circuit board upward that enables all users from every demographic to not only easily capture this immersive content, but also to understand and trust the device. The product requirement was to design a camera that could easily be embedded into the types of places where memories were made, such as sports games, concerts, clubs, and weddings and would enable anyone to easily capture 360° video and photos. Managing a budget of $1.4m, the project would require the coordination of a wide variety of subject matter experts. These experts would be entirely freelancers, consultants, and specialists hired for a brief period as needed.

Hardware challenges:

The unique requirements for this product included:

• Low light sensors for use in nightclubs and music venues.
• Low light video, no slower than 1/15th of a second exposure to limit frame blurring.
• Due to cost per unit restrictions, no more than 3 camera sensors per unit, therefore requiring small aperture, ultra-wide-angle 130° lenses.
• In-unit processing and stitching of photographs and video, as close to real-time as possible.
• A battery life of at least 6 hours.
• A unique and novel IoT framework to connect devices to the internet without the use of local WiFi networks, or 3G/4G/5G chips or connections.

Mechanical Engineering & Industrial Design Challenges:

Importantly, the camera had to be intuitive to anyone, even those unfamiliar with IoT devices. Designing around many people’s inherent fear of cameras, we used a tinted dome to simultaneously hide the electronics, while having the additional benefit of offering a ‘live preview’ of what the final photo or video would look like using purely analog means.

User challenges:

Our target audience, we joked, was somewhere between a drunk 25-year-old and a drunk 85-year-old. This broad target meant that we could not rely on anything complicated or complex, and was aware that for many of our users this could be the first time they’ve interacted with IoT or 360°. It needed to be natively intuitive to anyone, regardless of age or capacity.

We also firmly believed that it was unlikely for this demographic to download new command and control applications to their smartphone. Therefore a (progressive) web application would be the only solution to enable easy access for all, although we understood that to date, no IoT device had been controlled solely through a web app.

Business challenges:

How to monetize 360° content was a question that I'd love to say we had nailed before we started work; however, our start as a Facebook incubated entity primarily offered us the freedom to design without the limitations of a business plan in mind. The goal was simple at the start: “If 4% of Facebook users interacted with Virtual Reality today, we needed to increase that to 5%.” Or 6%. Or 20%. The goal was 'more.'

Setting off as an independent company caused us to revisit the business plan. We were aware that the content we were building was valuable to marketers, and that many brands would be interested in their audience or consumers posting moments of brand interaction in this new format. As an example, posting a ‘360° Selfie’ at a night club while drinking Absolut Vodka, would be an ideal example of Peer-to-Peer SocialVR for the Absolut brand.

Development:

The camera development required contract employment for a diverse group of people including software, hardware, design, engineering, and manufacturing teams spread across Hong Kong, China, Russia, and the UK.

Software Development:

Our product had three distinct software development layers:

• The core camera functionality captured individual photos from each of the three lenses simultaneously, with auto-corrected Gain, Exposure, White Balance, and Gamma Correction, driven by an array of sensors.
• The image stitching functionality, which used Computer Vision, accelerated in hardware using OpenCL.
• The command and control functionality, driven by an on-board progressive web app, hosted on-board the camera by NGINX.

Hardware Development:

Hardware development involved the custom design of the entire electrical system from the Processor-On-Chip upward.

The camera had to capture low light video and photographs with reasonably priced sensors. It also required exceptionally small aperture wide-angle lenses, without the ability to expose shots for more than 1/5th of a second, all positioned behind a tinted and mirrored dome. These unique, oft-conflicting requirements, required the use of prototype low-light camera sensors, supplied by Sony's R&D department, as well as custom glass lenses by the German lens manufacturer Sunex.

Mechanical Engineering & Industrial Design Development:

To keep the camera unit from overheating without the use of moving parts, such as fans, we positioned all heat-causing components on the bottom side of the main PCBA, which attached directly to the product stand, functioning as a single large heat-sink. This dissipated heat, while maintaining a seamless design without mechanical cooling.

User Development:

One of the requirements of our platform is that it would be easily controlled by a web application, as well as connecting to the internet without a 3G/4G/5G cellular connection or the need to sync with local WiFi networks - a feature not allowed by our clients for security reasons. Our solution to this was a proprietary tool called the 'WiFi Bridge.'

WiFi Bridge Development:

I envisioned a platform in which users could connect to the camera as simply as connecting to any WiFi network - one of the few IoT behaviors with which the average user is comfortable. This strategy would already allow for a hosting local web application on the camera, as well as commands like "take a photo" to flow down to the device, and data like resulting photo to flow up. The difficulty is that when a user connects to a device using WiFi, their smartphone promptly loses their 4G/LTE connection and is effectively cut off from the internet. The network disconnection is a problem faced by other IoT devices, like GoPro cameras or most drones, which use WiFi as their primary network link.

Business Development

By positioning our product in the Ad-Tech space, rather than trying to sell units outright, we were able to compete with other social platforms with established metrics and client budgets. With a focus on inserting our camera into locations with obvious brand connections - sports games with Budweiser/AB Miller brands, night clubs with Diageo or Pernod Ricard brands, weddings with Moet, etc. - we were able to encourage users to capture and share immersive 360° memories in conjunction with specific brands.

Across the spectrum, brands from Grey Goose Vodka to McDonald's spend their advertising budgets to buy a presence on users' Facebook feeds. Stating the obvious; not all content is equal:

• If content about a brand is posted on a user's Facebook page by a friend, this content is trusted more than content posted on their page by a brand. This type of social post is called 'P2P (Peer to Peer) Social'.
• Content-type is equally essential. Video content is viewed over 4 times more than text content, and 360° content (both photo and video) viewed over double to any other type of content.

Any content shared by the Glorious Labs cameras would be shared Peer-to-Peer (from one user account to another), and highly viewable (all 360° content). This offered brands a stark choice: spend their ad dollars on 'traditional Facebook marketing,' involving text posts and traditional images, shared in a traditional way, or to match the same dollar value with Glorious Labs' CPM, which offered a higher engagement for the same cost.

Result:

After a year and a half of development, spread across two continents and five countries, Glorious Labs completed the development of our Gloria 'G1' IoT 360° Camera, as well as our initial pre-production manufacture run of units from our factory in China.

View 360° video on mobile.

In addition to our core business model as a media platform for social content, we realized that our unique position as the owner of a large number of 360° spherical panoramic/equirectangular images and videos. This content library was valuable for tech companies training their Computer Vision algorithms to recognize objects in this new format. Similar to retraining algorithms to understand images in fisheye photos, these algorithms must be entirely retrained to recognize objects in the extreme distortion present in flattened spherical panoramic images. As a consequence, we partnered with Blippar to form a joint venture equirectangular algorithm, as well as to produce custom filters and augmented items for use in our spherical photos and videos.

The crowning achievement of Glorious Labs development was our invitation to Tech Crunch Disrupt 2018 and our award at the event as 'Top Pick' in the AR/VR category.