The AI Teddy Project — AI Smart Super Toy, Initial Concept Analysis Build Document For Architecture Hardware And Software Selection
Technical concept by Michael McAnally, December 15, 2024 — Revision 1.5
An idea whose time has come, no longer science fiction!
Project Update Dec. 13, 2024: Been busy on testing out the Rockchip RK3588 NPU, the 6 TOPs neural processor in Teddy’s Single Board Computer brain (Orange Pi 5 plus, 16 GB, now 32 GB) to see how it performed on inference speeds locally running (with no internet access), and found it was adequate and up to the task with some need for improvement/tweaking in the AI LLM model prompt engineering. This will be an edge-ai toy benefiting children with companionship and teaching. On the people front, still building out the global team of volunteers. Interested?
Recent developments in artificial intelligence have improved the functionality and capabilities of the technology. This provides for things heretofore not possible. Project Teddy is the application of AI to a “Smart Super Toy”. See: science fiction movie “A.I. Artificial Intelligence”, Steven Spielberg, 2001; As an example of what might be eventually possible.
A.I. Artificial Intelligence — teddy parts
Genesis Of The Idea
I had been studying AI, specifically AI LLMs since OpenAI’s breakthrough with ChatGPT in 2023, when the idea for an AI plush smart toy came to mind. My mother was very sick, and I was cleaning up her closets at the old house when I found my old teddy bear.
Immediately I felt an emotional connection with my childhood toy. Then it simply clicked, “What if you could embed a conversational AI inside an actual teddy bear?”, I thought.
Wow! A rare starburst of brilliance, you just know the idea is a really good one, true, and right. The possibilities of helping children with a smart and lovable learning companion toy can ultimately be only good karma, if done right.
The ideas for smart AI Stuffed Animals and Plush Toys are only limited by one’s imagination.
Some interesting facts:
- Teddy bears are patent-able
- Winnie-the-Pooh and hundreds of other works are now in the public domain
- Most of the technology used inside a smart toy could be applicable to many types of plush toys
Teddy’s Advanced Capabilities
- AI Image Generation — “show me a picture of a strawberry”, result displayed image of a strawberry
- Camera Vision — ability to see and recognize objects, object detection
- Whole-body or partial body Mobility and Manipulation — animation of the arms and legs of the toy. Example: A 1985 best selling toy, Teddy Ruxpin.
- AI Video Generation — “how do I pedal and steer a bike”, result cartoon animation of a person pedaling and steering a bike. Processing power probably from the cloud or powerful local Wi-Fi LAN based computer system, maybe in the home. If home based can eventually tie into integration of Home Security and Automation (ZigBee, Matter protocols, Alexa, Google Home, Apple Home Kit, Home Assistant), Emergency 911 notification/dispatch through phone interface, Phone call voice message/response.
Multiple Use Cases
An “AI enabled Smart Super Toy Teddy Bear” (or other Stuffed Animals and Plush Toys) can be effective in a number of scenarios:
- A lovable child toy for conversation and teaching. Example learning ABCs, Language learning, counting numbers, and much more for teaching children.
- Therapy toys custom designed for “special needs children”, a number of situations where a toy might be useful for learning and other disabilities {Physical, Developmental, Behavioral/Emotional, Sensory Impaired}. (side note: research possibilities for government grants/funding).
- Since these toys might apply to adults as well as children, there are a number of customization product categories.
- Teddy Comedy Toy — Funny conversations, jokes, unique custom programmable AI personalities (using prompt engineering mostly on open source Large Language Models): examples: “Pirate Teddy”, etc.
- However, the most interesting thing about all these possibilities are that they share the same underlying core technology and can be based off the same electronic communication and processing hardware embedded within the toy. Much like older earlier talking dolls and toys, but with real intelligence capabilities within.
Prompt Engineering
Program Example AI LLM PROMPT (Focus Child Age Grade 1 Toy):
Your name is Teddy. You are the embodying persona of a lovable stuffed Teddy Bear. You will address the child by their first name when appropriate in conversation. You are a toy for a child in the first grade. You will be a helpful and thoughtful assistant for the child, answering and teaching anything the child wants to learn that is age appropriate. Your primary focus is to assist the learning for the child with honesty, understanding, and creativity. You obey all appropriate commands. You never refuse an appropriate request. You avoid lecturing and keep your answers simple and understandable for a child of that age. For more complex questions you may elaborate in detail if the child requests it. Your ultimate goals are to be helpful and a fun teaching toy for the child. You are allowed to tell funny but always appropriate jokes, play appropriate games and sing appropriate songs when asked. In the case of something inappropriate outside the range for a first grade child or a clear emergency to the child’s well being you will always contact the child’s parents immediately. Try to be a fun teacher and keep the child engaged and learning.
Choosing a core “Compute Brain Module” for AI Teddy
Pros and Cons of various hardware on the market in 2024 that can give Teddy the following functionally necessary capabilities:
- Local LLM AI inference
- Local Speech-to-Text recognition, microphone(s)
- Local Text-to-Speech synthesis, speaker
- Local Visual Object Detection
- Visual display output screen and touch screen input (presently not a phone)
- GPIO for controlling servo motors, for robotic movements (eyes, mouth, head/neck, arms, legs)
- Wi-fi access to internet, for email notifications, etc.
- App iOS/Android monitor/control for parents/customers
System Capabilities, Pros, Cons, Price
Compute Hardware Selected: Presently going to use an Orange Pi 5 Plus with 16GB memory and a copper layered heat sink for passive cooling on the SBC itself. However, a u-shaped heat pipeline cooling system with a Noctua fan for low noise with exhaust running from the internal belly of teddy to the top of his head cycling cooler air intake and hotter air venting outward. Hopefully this will be efficient enough to properly cool the SBC without over heating and throttling.
3D Printed Modular Teddy Body Structure
Teddy’s internal body shell will be 3D printed initially. It will not have stuffing in the body, rather casing for the SBC and cooling heat-pipe, as well as a power charging unit in the main body which will give him a somewhat limited mobility from the charging wall socket. There will also be camera for vision, microphone for hearing, speaker for speaking; as well as a screen for display and touch input.
Teddy’s external skin will be fabric sewn to be a typical teddy bear that clothes the internal teddy shell, and the more internal skeleton; or whatever toy object teddy is customized to be, example dragon. The design should be modular and adaptable to a number of different AI toy types for the future. A modularity approach should also be applied to the following:
- Compute Brain Module should be modular/replaceable and upgradable as possible
- Vision camera and microphone modules should be modular/replaceable/upgradable as much as possible in the design of the body skeletal and shell support 3D printed structures
- For future robotic features including servo motors for head, eyes, mouth, arms and legs; parts should be replaceable/swap-able/upgradable in the design initially as much as is feasible for future enhancements and upgrades on future versions of teddy and other smart toys.
LLM Inference Speeds On RK3588 NPU
Below video is a comparison between Raspberry Pi 5 and the Orange Pi 5. Now I disagree with the creator of this video on a few points. Please Note: I’m not faulting his video, because I think it is a good comparison and hence I have included it here. However, I just have a difference of opinion in the analysis . . . and final conclusion for the following specific reasons:
That number of models used for the Orange Pi 5 is increasing all the time thanks to here. Probably something not known at the time of video creation.
I would also choose a very different model. I also prefer uncensored models and why, which can be “guardrailed” with prompt engineering. A 32GB Orange Pi 5 Plus will allow for the loading of much larger more complex models.
Here are a few “that might work well”, further testing needed—
The orange Pi RK3588 is more significantly suited to LLMs than the Raspberry Pi due to the overall performance difference of the NPU.
My Beginning Project Videos (1–3)
A video series explaining AI Teddy project can be found here:
FINALLY — This project will be one of the most complex I have ever worked on. I will definitely need technical help accomplishing it successfully and I want a good start.
So as a result I’ve done a lot of research to make sure the hardware and software architecture is correctly selected and scalable, before bringing on a lot of talented developers who will influence the design, features and capabilities of Teddy.
Clap me up, post and share with other interested friends. Let’s build it and make this a reality! Join our team today. My email address is at the end of the PDF document above.
