Mystery Bottle
Math encoding and decoding practice
We participated in a game jam with the Learning Games team at Curriculum Associates, a K-8 education publishing company, to develop a math game that would incorporate speech detection as a primary means of interaction, as younger students have more difficulty typing out answers.
Our goal was to draw on our experience building fluency games and experiment with speech detection to build a playable and intuitive fact fluency minigame.
Our contribution
Web game build
Game design
Technical strategy
The team
1 × engineer
1 × artist
1 × UX designer
Year
2022
Mystery Bottle
Math encoding and decoding practice
We participated in a game jam with the Learning Games team at Curriculum Associates, a K-8 education publishing company, to develop a math game that would incorporate speech detection as a primary means of interaction, as younger students have more difficulty typing out answers.
Our goal was to draw on our experience building fluency games and experiment with speech detection to build a playable and intuitive fact fluency minigame.
Our contribution
Web game build
Game design
Technical strategy
The team
1 × engineer
1 × artist
1 × UX designer
Year
2022
Mystery Bottle
Math encoding and decoding practice
We participated in a game jam with the Learning Games team at Curriculum Associates, a K-8 education publishing company, to develop a math game that would incorporate speech detection as a primary means of interaction, as younger students have more difficulty typing out answers.
Our goal was to draw on our experience building fluency games and experiment with speech detection to build a playable and intuitive fact fluency minigame.
Our contribution
Web game build
Game design
Technical strategy
The team
1 × engineer
1 × artist
1 × UX designer
Year
2022
Process
Research-backed fluency practice
We were inspired by recent research on building reading fluency by improving cognitive flexibility, during which participants' reading skills improved after completing tasks that exercised their cognitive flexibility with words. See the example below — a task would be to identify an appropriate word to fill the empty quadrant (e.g. frog).
Because understanding the relationship between fact families (e.g. how 1 + 2 = 3 is related to 3 - 2 = 1) is crucial in building basic understanding of math facts, we designed gameplay that would encourage learners to use their knowledge of fact families to answer problems and say the answer out loud to complete the question.
Speech detection for different learning levels
Upon playtesting the initial version of the minigame, we also discovered that speech detection was a useful interaction for basic encoding and decoding of numbers. We created additional levels that enabled learners to explore these different means of recognizing and expressing numbers:
Numbers written as words (e.g. "nine")
Numbers written as sumbols (e.g. "9")
Numbers spoken aloud
Technical nitty gritty
The WebGL game was built in Unity/C# using the UnityWebGLSpeech open source plugin (enables speech detection in Unity via the Web Speech API).
Process
Research-backed fluency practice
We were inspired by recent research on building reading fluency by improving cognitive flexibility, during which participants' reading skills improved after completing tasks that exercised their cognitive flexibility with words. See the example below — a task would be to identify an appropriate word to fill the empty quadrant (e.g. frog).
Because understanding the relationship between fact families (e.g. how 1 + 2 = 3 is related to 3 - 2 = 1) is crucial in building basic understanding of math facts, we designed gameplay that would encourage learners to use their knowledge of fact families to answer problems and say the answer out loud to complete the question.
Speech detection for different learning levels
Upon playtesting the initial version of the minigame, we also discovered that speech detection was a useful interaction for basic encoding and decoding of numbers. We created additional levels that enabled learners to explore these different means of recognizing and expressing numbers:
Numbers written as words (e.g. "nine")
Numbers written as sumbols (e.g. "9")
Numbers spoken aloud
Technical nitty gritty
The WebGL game was built in Unity/C# using the UnityWebGLSpeech open source plugin (enables speech detection in Unity via the Web Speech API).
Process
Research-backed fluency practice
We were inspired by recent research on building reading fluency by improving cognitive flexibility, during which participants' reading skills improved after completing tasks that exercised their cognitive flexibility with words. See the example below — a task would be to identify an appropriate word to fill the empty quadrant (e.g. frog).
Because understanding the relationship between fact families (e.g. how 1 + 2 = 3 is related to 3 - 2 = 1) is crucial in building basic understanding of math facts, we designed gameplay that would encourage learners to use their knowledge of fact families to answer problems and say the answer out loud to complete the question.
Speech detection for different learning levels
Upon playtesting the initial version of the minigame, we also discovered that speech detection was a useful interaction for basic encoding and decoding of numbers. We created additional levels that enabled learners to explore these different means of recognizing and expressing numbers:
Numbers written as words (e.g. "nine")
Numbers written as sumbols (e.g. "9")
Numbers spoken aloud
Technical nitty gritty
The WebGL game was built in Unity/C# using the UnityWebGLSpeech open source plugin (enables speech detection in Unity via the Web Speech API).
Outcome
After a week of experimenting and building, we presented Mystery Bottle to the rest of the game jam teams and received incredibly positive feedback about the ease-of-use of the voice recognition interaction.
Once we demonstrated the capabilities of leveraging speech detection technology in a playable math minigame, the Learning Games team added implementation of this interaction to their roadmap for their new Fluency Flight product.
Outcome
After a week of experimenting and building, we presented Mystery Bottle to the rest of the game jam teams and received incredibly positive feedback about the ease-of-use of the voice recognition interaction.
Once we demonstrated the capabilities of leveraging speech detection technology in a playable math minigame, the Learning Games team added implementation of this interaction to their roadmap for their new Fluency Flight product.
Outcome
After a week of experimenting and building, we presented Mystery Bottle to the rest of the game jam teams and received incredibly positive feedback about the ease-of-use of the voice recognition interaction.
Once we demonstrated the capabilities of leveraging speech detection technology in a playable math minigame, the Learning Games team added implementation of this interaction to their roadmap for their new Fluency Flight product.