Dec 2020

What the Flock?

Combining embodied games and active pauses to create a collaborative game experience that benefits children's learning abilities.

Project Details

Project Type: Product Design

Purpose: Academic Team Project

Timeline: 18 Weeks

Methods: Web Design Game Design Interaction Design Brand Design Interface Design Sprite Design Sound Design

My Contributions

Visual Design: Developed the visual design language for our solution & presentation materials, including logos, colors, fonts, and interface elements.

Interaction Design: Assisted with the conception refinement of the interaction and system design throughout the development process.

Research: Assisted with background academic & user research, as well as presenting findings during checkpoints and the final demo.

01 The Problem

What Happened to Recess?

COVID-19 has affected a majority of schools over the past year, resulting in many children having to switch from in-person teaching to online learning. This transition has been difficult for many students who are now physically isolated from their peers and miss playful moments of interaction at school. Additionally, children may also struggle with having to sit at their desk for long periods of time. This lack of informal embodied social interactions between children suggests that this problem area is in need of intervention.

Problem Statement

How can we design and develop an informal embodied social interaction experience for students currently struggling with socially-distant, online learning environments?

02 The Solution

Active Pauses in Distanced Learning Environments

To address this, we developed What The Flock?, an online game for Active Breaks for children in primary and elementary school who may not be feeling as engaged and connected to their classmates in a virtual classroom. This game begins to explore how we can create cooperative and competitive embodied games for Active Breaks that children can play together online without needing to purchase or install additional hardware and are easily embedded in lesson flows. The design allows children to adapt the controls to their own level of comfort with movement, promoting accessibility. We employ a narrative and visual design with original art inspired in bird migrations to foster metaphors of bodily movement and collaboration throughout the interaction. Our game can be played in conjunction with videoconferencing technologies, leveraging their affordances for verbal communication. Additionally, each player also controls the volume of a sound track, resulting in a unique song for every instance of game play.

Skip to Project Demos

03 Related Work

To meet the needs of social and physically engaging activities for virtual education, our game weaves together four areas in games and scholarship: Embodied games, IO games, Active Breaks for education and Online Multiplayer Games.

Active Breaks for Education

Active Breaks (ABs) are short and structured physical activities frequently used by teachers during class in K-12 education. Research suggests these activity breaks can foster concentration and improve performance besides promoting a positive attitude towards exercise in children^[2]. Similar benefits have been found with computer-based active breaks^[12]. Teachers may rely on existing online media to carry out Active Breaks during class, such as instructional videos on YouTube that encourage active pauses. Another platform marketed to families and educators is GoNoodle^[4], which produces movement-based mini games in addition to videos meant for single or collocated players. Sanford Fit is another program to encourage healthy lifestyles through accessible tools^[13]. Compared to GoNoodle, Sanford Fit’s videos are more focused on exercise for healthy lifestyles, rather than moving and exercising for entertainment. In contrast, What The Flock? promotes a fun active pause that fosters verbal communication in a cooperative context.

IO Games

Dislocated players use the web to connect in a variety of multiplayer online games. Among these dislocated multiplayer experiences, IO Games–such as Slither.io^[9] are a genre of short and casual games that can be played in real time without installing additional software. While multiplayer games like MMOs have drawn research interest even in the context of education^[8], IOs are relatively understudied despite their potential benefits. Because they are easy to join and relatively simple, IO Games have been very popular during social isolation. While IO Games may compensate for the lack of informal sociability caused by the pandemic among school children, they do not promote physical activity because they often rely on traditional controls such as a keyboard and mouse. What The Flock? shares the ease of use and the casual interactivity of IO Games but promotes an embodied collaborative activity for virtual classroom enrichment.

Embodied Games

Embodied Games are played using full body movement. They have become commonplace with commercially available gaming consoles such as the Nintendo Wii and Microsoft Kinect. In addition to their popular value as entertainment, research has pointed to their cognitive and affective benefits when they are used in the classroom for learning mathematics and science^[1]^[7]. Some of these games are played with cooperative collocated players. Such is the case of Kinect Adventures!^[15] where players complete tasks like stopping holes to prevent water from entering through a wall. The platformer game FRU [10] provides an asymmetrical experience where one player uses their body to create bridges or reveal parts of the scene, while the other player navigates the space using a traditional controller.

Collaborative and Embodied Music

We were also inspired by a variety of examples of technology-based embodied musical platforms, and associated research that demonstrated their benefits for informal education. The TuneTable is a project by researchers at Georgia Tech that has studied how co-creative interactions controlled by groups of museum visitors through tangibles can promote interest formation in computer science^[15]. Media artifacts such as Body Synth^[11] and Shared Piano^[5] are two online projects developed by Google Creative Lab that are either collaborative or embodied ways of making music online. Body Synth allows for one person to play chords on a variety of instruments by moving their entire body. Shared Piano lets multiple people online play the same virtual piano together using a keyboard and mouse for controls. Unlike these examples, our design allows for simultaneous embodied music making and collaboration without any tangible controllers.

04 Development

Initial Prototypes

We started by paper prototyping and software prototyping (p5.js, ml5.js, node.js) multiple interaction design ideas for multiplayer full body engagement for children in elementary school. We went through several rounds of brainstorming followed by two cycles of individual paper prototyping of the user interaction. These ideas were then shared with the rest of the class and we iterated based on their feedback. At this stage we discarded any ideas that could pose accessibility issues due to the interaction design or the technological demands on users, such as games using body poses with ml5.js PoseNet that required a lot of computational power or strict requirements for moving to control the game. We also iterated through narrative designs whose metaphors were inconsistent with the system design such as an initial prototype of a dung beetle rolling balls through hills.

Prototype using ml5.js PoseNet library

Initial Paper Prototype

Sketching the Interface Design

Game Development

We recombined elements of our prototyping phase to create an interaction design where amounts of movement would control third person players. We created a basic software prototype with p5.js and node.js to test the ‘calibration phase’, where individual users would adjust the controls to a desired amount of movement. This is achieved by calculating the average of pixel differences in the images during a given phase of calibration. With this functional script in hand, we then explored different metaphors that would match the embodiment of the system while fostering a collaborative and competitive balance. The bird migration metaphor emerged then as an ideal candidate to correspond the amounts of movement to the height of the individual bird character, while allowing for a collective score and goal through. Another advantage of this metaphor was its precedent in other games, such as Flappy Bird^[3]. Finally, we needed to create visual feedback that promoted the collaboration through coordinated movement. To emphasize the need for cooperation as a flock through individual actions, we created targets that randomly spawn in one of the four levels of the ’score’ for each player. This requires the players to vary their amounts of movement to reach their individual target. Initially the targets were abstract locations signaled with circles drawn in p5.js. We decided to create a more playful narrative by incorporating magic mushrooms with quirky faces. This rounded up the narrative so that the collective flocking would be reinforced by the mushroom reward.

Migratory Bird sprite sheets

Magic Mushroom sprite sheets

Visual Design

We hand drew and digitally colored the bird assets based on ten migratory birds, including Canada geese, cardinals, pigeons, eagles, mudpies, parrots, etc. We created sprite animations using p5.js and the bird animation sprites. This allowed us to control the speed of the bird characters flapping their wings in relation to the player’s movement. We also created simple backgrounds representing four environments: the forest, the city, space and under water. We added helmets to our birds when they were either in space or underwater. We created semi-transparent backgrounds to allow players to see their camera feeds, promoting engagement through third person and self-visualization. The mushroom assets, that represent the targets, were created using license-free assets that we recolored and added faces to for aesthetic and narrative purposes.

Background Designs: Forest, City, Space, and Underwater

Branding Design

When developing the visual design language for our branding and user interface, we had a few key goals we wanted to keep in mind. The first is that we wanted the design to be appealing to both children and adults. To do this, we studied the designs employed by brands of children’s toys, children’s board games, and children’s clothes which have a similar goal of appealing to both demographics. Many of these brands utilized 3 to 5 bright, engaging colors in combination with 1 or 2 neutral colors for contrast. The second goal we had was to consider modern logo & branding design trends when creating our logo. To do this, we kept the style simple with clean lines and solid colors, while using a hand-written sans-serif font to keep the logo from feeling too “adult”. Finally, the third goal we had was to keep the interface simple and understandable by young children who may be playing the game. To do this, we used large buttons, simple instructions, and icons with clear affordances to direct children through the website, the tutorial, and the game.

Branding & Interface Design

Audio Interface Design

Music encourages players to dance at various speeds to control their avatars.We designed a set of interactive soundtracks to emphasize the relationship between players’ movements and game play. Each round in What The Flock? features a unique song. Each song consists of four stereo tracks that adjust in volume with each player’s movement– the faster a player is moving, the louder their instrument plays in the arrangement. The songs were composed using Apple’s Garageband and Soundtrap by Spotify. Because of the randomness that prompts the mushrooms to appear, this effectively means that every time the game is played, there is a unique version of the song.

Testing

We conducted several iterative rounds of testing among us throughout the development, helping us correct bugs and basic usability issues. However, we needed to test the functionality of the interaction and the robustness of our software prototype with a larger group to simulate the conditions of a regular class that would use What The Flock? for Active Breaks. To do this, we requested to do a demonstration with a class of 12 graduate students. Part of the preparation for the activity regarded the creation of groups. While during a class a teacher may want to group together specific children, or children may want to create their own groups, initially we thought it would be useful to create random teams that would form as students joined in. This implied that students should go to a break out room after the flock formation. We realized this could complicate the flow of the class, resulting in difficulties for the teacher or the students if they needed to figure out who was in their group after calibration. To better leverage the affordances of organized breakout rooms, we decided to organize the teams in advance and sent individual room links to the groups of students. For our testing session we had 3 groups of 4 students counting one of us in each group to observe what happened. Our testing was successful in creating the fun and active pause we had designed for. We collected feedback such as creating additional constraints for the collection of points. One of our users pointed out that having to sustain a position for a few seconds, rather than having to simply reach the target, would foster the sense of collaboration. Our testing also revealed that a common scoreboard across all teams would provide a competitive incentive in addition to the collaborative dynamics.

05 Final Game Design

Starting splash screen

Paginated instructions

Countdown to game start

Premise

What The Flock? is an online multiplayer game that encourages children to get up from their desks and move for a few minutes during remote classes. What The Flock? aims to create fun, social and Active Breaks that encourage movement and collaboration in small groups. This full body game only requires a computer with a web camera and access to the internet. Players become a flock of migrant birds collecting magic mushrooms along their migration. Players need to coordinate movements with the group to reach proper flock configurations and maximize their chances of survival in their very long trips through different landscapes. The intensity of movement of the child controls the bird’s position and the volume of a musical track, providing multi-sensory complexity.

Motivation

The game surged out of the combined interests of three graduate students taking a class about social distance. One of the team members, who worked as a school teacher, was very aware of the shortcomings of virtual classes and the distress expressed by her students. This motivated our initial desire to create a physically engaging social game that would be easily accessible to children taking virtual classes. Another motivation for our design was to build a narrative and a game structure that could inspire children to think beyond their environments of confinement and connect them to natural processes. The migratory bird theme relies on embodied metaphors of bodily movement and the collective nature of flocking. We hope this connects children to the process of seasonal bird migrations, providing contrast to the confinement caused by the pandemic while culturing sympathy toward natural migratory processes.

Gameplay

The game slots players into flocks of up to four migratory birds which are randomly assigned to each player as they join. The game should be played synchronously with voice or video call platforms like Zoom. This allows us to harness the affordances of existing classroom technologies, especially when it comes to promoting communication and encouragement between players of the same flock. When players join the game they are prompted to enter their name. Players then calibrate their levels of movement while waiting for all players to join. They are given the instructions “Don’t Move”, “Move Slowly”, “Move Regular”, and finally “Move Fast.” Allowing the player to determine their own desired amount of movement during calibration makes the game more accessible to players with various levels of mobility.

Calibration stages allow the player to control their bird with their preferred amounts of movement

The amounts of movement correspond to how high or low their bird avatar moves in the game scene. Each player controls their own bird and is able to see every other player’s bird on the screen. The targets, which are magic mushrooms with quirky smiles, will spawn in random configurations on the screen. Each player must reach their individual target. Once every player has reached their mushroom, the flock accumulates a point. The tricky part is for all players to reach the target at the same time! Each round lasts a minute and the flock’s goal is to earn as many points as possible during the round. During each round, birds traverse various landscapes such as the forest, the city, outer space and the ocean. Each of these scenes corresponds to a different musical combination.

Flow diagram illustrating game interaction

06 Demos

Video Demo

Live Demo

Feel free to check out a live version of our game as well!

Game Demo

07 Conclusions & Future Work

While we have a functional prototype that we tested among ourselves and others in our remote class, future work should evaluate and refine the game in naturalistic contexts. We plan to add complexity to the game dynamics and a thicker narrative connection to bird migration. An important motivation to our narrative design was Pepler et al.’s scholarship on educational games with wearable computers to teach children about the complex behaviors of honeybees^[6]. While the short and casual nature of What The Flock? is very different from the deep engagement promoted by Peppler et al., we considered carefully the interplay between scoring mechanisms and collaboration, hoping to further study these mechanisms in the future. These studies will allow us to study the dynamics of Active Breaks with online tools and IO games. Additionally, we plan to develop systems to make it easy to divide classes into small teams and provide leaderboards to show score rankings of multiple teams within a class. We plan to conduct user studies with children to evaluate how this game promotes collaboration through remote full body interaction. We hope to learn more about the benefits of collaboration and competition through socially cognitive and physically stimulating pauses for classes.

Tools Used:

HTML/CSS JavaScript/jQuery p5.js ml5.js PoseNet Library Node.JS Adobe Illustrator GarageBand Soundtrap GIMP Heroku

The Team:

Jack Towery, Jordan Graves, Sara Milkes Espinosa

References:

[1] Ivon Arroyo, Matthew Micciollo, Jonathan Casano, Erin Ottmar, Taylyn Hulse, and Ma. Mercedes Rodrigo. 2017. Wearable Learning: Multiplayer Embodied Games for Math. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM, Amsterdam The Netherlands, 205–216. https://doi.org/10.1145/3116595.3116637

[2] Joanne Delk, Andrew E. Springer, Steven H. Kelder, and Megan Grayless. 2014. Promoting Teacher Adoption of Physical Activity Breaks in the Classroom: Findings of the Central Texas CATCH Middle School Project. Journal of School Health 84, 11 (Nov. 2014), 722–730. https://doi.org/10.1111/josh.12203

[3] Dong Nguyen. 2013. Flappy Bird. Game [Mobile].

[4] GoNoodle. [n.d.]. GoNoodle Games. Game [Mobile]. https://gonoodle.com

[5] Google Creative Lab. 2020. Shared Piano. Online. https://musiclab.chromeexperiments.com/Shared-Piano

[6] Kylie Peppler, Joshua A. Danish, and David Phelps. 2013. Collaborative Gaming: Teaching Children About Complex Systems and Collective Behavior. Simulation & Gaming 44, 5 (2013), 683–705. https://doi.org/10.1177/1046878113501462 arXiv:https://doi.org/10.1177/1046878113501462

[7] M. J. Silva, E. Ferreira, V. Andrade, O. Nunes, and M. Da Luz Carvalho. 2015. Embodied education: Senses, emotions, and technology. In 2015 International Symposium on Computers in Education (SIIE). 32–37. https://doi.org/10.1109/SIIE.2015.7451644

[8] Constance Steinkuehler. 2008. Massively Multiplayer Online Games as an Educational Technology: An Outline for Research. Educational Technology 48, 1 (2008), 10–21. http://www.jstor.org/stable/44429539

[9] Steve Howse. 2016. Slither.io. Game [Online]. http://slither.io/

[10] Through Games. 2016. FRU. Game [Xbox One].

[11] Use All Five and Google Creative Lab. 2018. Body Synth. Online. https://creatability.withgoogle.com/body-synth/

[12] F. S. Uzunoz, M. K. Chin, M. M. C. Mok, C. R. Edginton, and H. Podnar. [n.d.]. The Effects of Technology Supported Brain-Breaks on Physical Activity in School Children. 87–104.

[13] Suzanne E. Williams and Donna Hardie. 2019. Introduction to Sanford Health Children’s Health & Fitness (fit) Initiative: A Physical and Emotional Health Promotional Intervention. American Journal of Health Education 50, 3 (2019), 159–166. https://doi.org/10.1080/19325037.2019.1590257 arXiv:https://doi.org/10.1080/19325037.2019.1590257</p>

[14] Anna Xambó, Brigid Drozda, Anna Weisling, Brian Magerko, Marc Huet, Travis Gasque, and Jason Freeman. 2017. Experience and Ownership with a Tangible Computational Music Installation for Informal Learning. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction (Yokohama, Japan) (TEI ’17). Association for Computing Machinery, New York, NY, USA, 351–360. https://doi.org/10.1145/3024969.3024988

[15] Xbox Game Studios. 1994. Kinect Adventures! Game [Xbox 360].