Guide Pishi Through an AI Voice-Controlled Maze in Scratch
Speak a route, guide Pishi through two maze missions, collect the optional battery, and reach the goal before time runs out!
Use Speech Recognition to guide Pishi through a maze with your voice. Say directions and numbers to move across the grid, avoid obstacles, and reach the goal. Complete both missions and see how high you can score!
Difficulty
INTERMEDIATE
Estimated Time
40-50 MINUTES
Extension
SPEECH RECOGNITION
📖 What You Will Learn
- How AI Speech Recognition converts spoken language into text
- How Speech Recognition turns spoken phrases into text
- How to read a phrase one word at a time and follow several commands in order
- How direction words and numbers control movement on a grid
- How to use custom blocks and broadcasts to organize the game
- How colors detect obstacles, the battery, and the goal
- How to add a timer, score, and feedback sounds
- How to customize commands and improve recognition when speech is unclear
🧰 What You Need
- A computer with a microphone and an internet connection
- Chrome, Edge, or another browser that supports Speech Recognition
- Pishi.ai Scratch (start a new project)
- Speech Recognition extension (view the help guide)
- The Pishi sprite and two maze backdrops
- Distinct colors for paths, obstacles, boundaries, the battery, and the goal
- Sounds for the countdown, collisions, battery, and goal
🛠️ Build Your Project
1. Create the Two Maze Missions
Create two maze backdrops named mission 1 and mission 2. Draw each as a grid with the centers of neighboring squares 56 steps apart, matching Pishi’s movement.
Give the safe path, obstacles, outer boundary, battery, and goal distinct colors. Place Pishi on the starting square. The battery is optional: collecting it adds 10 points, but Pishi can finish without it.
Tip: Use the eyedropper to select the exact backdrop colors in your touching color blocks. Even a slight color mismatch can prevent detection.
2. Initialize the Game and Voice Commands
Create the variables shown in the blocks, then make a custom block called: “initialize”.
Use it to select English (American) for Speech Recognition, switch to mission 1, place Pishi at the start, set “step” to 56 and “countdown” to 60, and reset the “score” and other game values.
This lets you change a word later if Speech Recognition has trouble understanding it. For example, you could change “right” to “east”.
This lets you change a word later if Speech Recognition has trouble understanding it. For example, you could change right to east.
Each command must be one word. The project checks recognized speech one word at a time, so a command such as “go right” cannot match a single command variable.
3. Start the Mission and Process Spoken Commands
Start the green-flag script by running “initialize”, announcing the mission, and broadcasting “start timer”. The timer script will count down and broadcast “start recognition” when Pishi is ready to listen.
In the “start recognition” script, keep listening until “countdown” becomes “FINISH”. After each phrase, save the transcript in “message”, then use “next word [final]” to send each confirmed word to “run command [text]” in order. Clear “last command” before listening for a new phrase.
Use [final] because these are the words Speech Recognition has finished recognizing. [live] words can still change while the player is speaking, so Pishi might act on a word that was only a temporary guess.
For example, in “move right four times,” the game ignores move and times. “Right” moves Pishi one square; “four” adds three more squares. Keep the number immediately after its direction—“move right please four times” will not repeat the movement.
Testing tip: Show the “message” variable while testing to see what Speech Recognition heard.
4. Turn Direction and Number Words into Movement
Create the custom blocks:
- run command [text]
- check repeats [num]
The first block checks whether the current word is a direction.
If it is, Pishi moves one square:
“up” and “down” change “y”, while “left” and “right” change “x”.
“check repeats [num]” handles “two”, “three”, and “four”. A direction has already moved Pishi one square, so the number adds the remaining moves.
For example, “right three” moves Pishi three squares in total.
After each square, run “calculate score” to check where Pishi landed. The short wait lets the player see each move.
The script also remembers the previous direction in “last command” so a number can repeat it.
5. Calculate the Score and Change Missions
Create the custom block “calculate score”. After each move, it uses “touching color” blocks to check what Pishi has reached:
- Outer boundary: Subtract “10” points and return Pishi to the start.
- Obstacle: Subtract “5” points; Pishi can continue.
- Battery: Add “10” points if “battery collected” is “0”, then set it to “1” so the bonus is awarded only once.
- Goal: Add “15” points plus the remaining whole seconds.
Play the matching sound when each event happens. At the goal of “mission 1”, reset “last command” and “battery collected”, switch to “mission 2”, return Pishi to its start, and broadcast “start timer”.
At the goal of “mission 2”, set “countdown” to “FINISH” to end the game.
6. Add the 60-Second Countdown and Finish the Game
Create a script beginning with “when I receive [start timer]”.
It says “3”, “2”, “1”, plays the whistle, resets the Scratch timer, and broadcasts “start recognition”.
This same sequence runs at the start of both missions.
While the Scratch timer is below “60” seconds, update “countdown” to show the time remaining.
When time runs out, set “countdown” to “FINISH”.
This stops the listening script too.
Hide the countdown display when the mission ends.
Pishi then announces “Your Score: [score]”.
🚀 Try Your Project!
Click the green flag and allow microphone access. Wait for the countdown and whistle, then speak when Pishi says “Listening…”. Try a sentence such as “move right three times”, followed by a route of your own.
Test both missions. Can you reach each goal before time runs out? Try collecting the optional battery, then try a shorter route without it.
Which route earns the higher score?
If Pishi moves unexpectedly, show the “message” variable to see what Speech Recognition heard. Try speaking in a quieter place or changing a command to a different one-word alternative.
🤖 How Does It Work?
The mission starts with a countdown and whistle
↓The player speaks a route
↓Speech Recognition turns it into confirmed words
↓Scratch reads the words in order
↓Directions move Pishi; numbers repeat the previous direction
↓Scratch checks the color after each square
↓The game updates the score and mission
Other words are ignored. Broadcast messages coordinate the countdown and listening, then start the same sequence again for “mission 2”.
💡 Make It Better!
1. Test New Command Words
Change the one-word commands stored in the “command …” variables. Try several alternatives and record which ones Speech Recognition understands most reliably. Does a complete phrase, such as “move right three times”, work better than “right three”?
2. Build a Multilingual Mission
Change the Speech Recognition language and replace the command words with words from that language. Keep each command to one word, then test whether the maze still responds as expected.
3. Add an AI Hint
Use the AI ChatBot extension to give the player a hint when they get stuck. Give it Pishi’s position and nearby obstacles, and ask for one suggested direction rather than the entire route.
4. Make the Next Mission Adapt
Track obstacle hits or how much time the player needs. Give a player who is struggling more time in “mission 2”, or offer an extra challenge to a player who finishes quickly.
🔗 Share Your Creation
Finished your maze?
Change the theme or design a new route, then share your project with a friend or classmate. Ask them to play both missions and see if they can beat your score!
