Build an AI Object Hunt with Object Detection in Scratch
Let AI discover the objects around you, then race against the clock to find them again!
In this project, the Object Detection extension first watches the camera and collects the names of objects it can recognize. Then Pishi chooses four of those objects at random and challenges you to show each one before time runs out.
Difficulty
INTERMEDIATE
Estimated Time
25–35 MINUTES
Extension
OBJECT DETECTION
📖 What You Will Learn
- How to use Object Detection to recognize real-world objects
- How to read the list of objects currently detected by the AI
- How to build a custom list from detected object names
- How to count the unique objects in that list
- How to randomly choose an object from a detection list
- How to check whether a specific object is currently visible
- How to create a timed AI challenge
- How object detection can behave differently depending on lighting, angle, distance, and the objects around you
Unlike Image Trainer, you do not need to train the AI first. Object Detection uses a pretrained model that already knows many common object categories.
🧰 What You Need
- A computer or laptop with a camera
- Pishi.ai Scratch (start a new project)
- Object Detection extension (view the help guide)
- Pishi sprite
- A few everyday objects around you
- Good lighting
You will also use these variables:
- score
- time
- all objects found
- object to show
The project also uses the extension’s my detection list to store the unique object names discovered during the first part of the game.
🛠️ Build Your Project
1. Prepare the Object Detector
When the green flag is clicked:
- Set all objects found to 0
- Set score to 0
- Set time to 0
- Turn classification off
- Clear my detection list
Then play the starting sound and say:
Let’s see what the AI can recognize around you! Show me at least 6 different things, one at a time.
Turn the detection boxes on, then turn classification on.
💡 Hint: Turn classification off and then on at the start of the project to reset detection events. This helps events such as “when any object detected” work correctly even for objects that were already visible before program started running.
This lets the player see which objects the AI is recognizing in the camera view.
2. Collect Recognized Objects
Repeat until enough different objects have been found.
Inside the loop:
“add [list of current detections] to my detection list”
Note: my detection list is normally used to filter which object labels the extension should look for. In this project, we use it differently: as a convenient way to store the objects the AI has already found, because the list keeps only unique object labels and does not add duplicates.
You can also show the current list with:
“Detected: [my detection list]”
When the number of items in my detection list becomes greater than 5:
- Set all objects found to 1
- Turn detection boxes off
- Remove person from my detection list
Removing person prevents the player from getting an automatic point just because they are already in front of the camera.
Then say:
Great! I found these objects: [my detection list]
Play a success sound.
3. Get Ready for the Hunt
Give the player a short moment to prepare.
Say:
Now let’s see how fast you can find them!
Then say:
If possible, move those objects out of the camera view.
This makes the game more interesting because the requested object will not already be visible when the next challenge starts.
Now the game has a list of objects that are both nearby and already recognized successfully by the AI.
4. Choose a Random Object
Repeat the challenge 4 times.
At the start of each round:
- Set time to 0
- Say Ready?
- Count down: 3…, 2…, 1…
Then choose a random item from my detection list and store it in: “object to show”. For example:
“my detection list item # [pick random [1] to [number of items in my detection list]]”
Remove that object from my detection list immediately after choosing it.
This makes sure the same object is not selected again in a later round.
Then say:
Show me the: [object to show]
5. Race Against the Timer
Show the time variable and reset the Scratch timer.
Repeat until:
time = FINISH
Inside the loop, continuously set:
time to timer
If the timer becomes greater than 5:
- Hide the time variable
- Set time to FINISH
- Play the failure sound
- Say: Time’s up! Let’s try another one.
Otherwise, keep checking whether:
“[object to show] #[1] detected in [current] detections?”
If the requested object is detected:
- Change score by 1
- Hide the time variable
- Set time to FINISH
- Play the success sound
- Say: Found it!
The round ends as soon as the object is found or the five seconds run out.
6. Finish the Game
After all four rounds are complete, show the final result.
Say:
Your final score is: [score]
Then play the ending sound.
You can also make the result a little clearer by saying:
Great job! Your final score is [score] out of 4.
Now the player can restart the project and get a different set of objects and challenges each time.
🚀 Try Your Project!
Click the green flag and allow access to the camera.
First, help the AI discover at least six different things around you.
Try showing objects such as:
- bottles
- cups
- books
- backpacks
- chairs
- phones
- laptops
- toys
You do not need to use any particular object. Just experiment and watch the detection list grow.
Once enough objects have been found, move them out of view if possible.
Then get ready!
Pishi will choose four of the detected objects one at a time.
You have 5 seconds to find each object and show it to the camera.
How many can you find?
🤖 How Does It Work?
This project has two main stages.
First, the AI explores what is available:
Camera sees the objects around you
↓
Object Detection recognizes the visible objects
↓
Scratch reads the current detection labels
↓
The labels are added to my detection list
↓
The game waits until enough different objects are found
Then the game begins:
Choose a random object from my detection list
↓
Ask the player to show that object
↓
Start the 5-second timer
↓
Object Detection checks the current camera view
↓
If the object is detected, add 1 point
↓
Repeat with a different object
Unlike a fixed-object game, this project builds each challenge from objects the AI has already recognized in the player’s environment, making it more flexible and reliable.
💡 Make It Better!
- Check AI Confidence
Use the confidence value to show how sure the AI is about each detection.
For example:
“I found the bottle with 91% confidence!”
Try the same object in different conditions and see how the confidence changes. - Accept Only Confident Detections
Make the game more reliable by only accepting an object when its confidence is above a threshold, such as 0.7.
If the confidence is too low, Pishi can ask the player to show the object more clearly. - Challenge the AI with Different Views
Test the same object from different angles, distances, backgrounds, and lighting conditions.
See which changes make the object easier or harder for the AI to recognize. - Add Position Challenges
After the correct object is found, ask the player to move it to a specific place, such as
“Move it to the left!” or “Put it in the center!”
Use the object’s x and y position to check where it is. - Test AI Reliability
Show the same object several times and count how often the AI recognizes it correctly.
Compare different objects and discover which ones the model detects most reliably. - Use Object Size as a New Challenge
Use the detected object’s width or height to ask the player to bring it closer to the camera or move it farther away.
This shows that Object Detection can understand more than just what the object is.
🔗 Share Your Creation
Finished your project?
Make it your own! Change the rules, visuals, sounds, difficulty, or AI behavior. Add a new challenge, create a different theme, or combine ideas from other projects.
Then share your version with a friend and see what they create!
