by Jonathan David Martin & Christopher Maxey

towAR is a competitive tower building game. Adventure outside with a friend and compete to build the tower with the most points before time runs out. Built on Lightship’s semantic segmentation and multiplayer templates, players gather their building materials from the environment around them, use those materials to buy building blocks of different point values, and then place those blocks to construct a unique AR tower. Points are awarded based on tower height and the value of the blocks used. Whoever gets the most points when the timer runs out, wins! Behind the scenes, towAR takes advantage of Lightship’s networking and semantic segmentation apis. Our app features a potential for a large number of interacting objects modeled with Unity’s physics engine. For consistency amongst players, all physical interactions are handled by the host and pose updates for blocks are sent to the other player. To keep things performant, towAR relies on compressed sparse message passing, only sending pose updates when a block’s pose changes. Pose information is sent in half precision, effectively cutting the message sizes in half compared to using Unity’s standard 32bit float representation of poses. At the scales used in game, half precision is more than enough to keep a consistent experience on the side of the non-host peer. Networking functionality is contained within the GameController and MessagingManager scripts. TowAR’s use of semantic segmentation focuses on getting a player moving and interacting with their environment but linking resource gathering to segmentation. In gather mode, pressing on the screen will query the semantic buffer to see what semantic class, if any, is located at that screen position. If held or tapped, resources are added to an inventory system. Currently two classes, foliage and ground, are used for placing blocks but future iterations would gather from other classes as well. The QuerySemantics script contains the functionality for segmentation and the inventory system that interacts with other scripts.