Table of Contents

• About This Article
• Part One: Hardware Configuration of the Raspberry Pi Zero 2 WH
• Part Two: Voice Processing and Response Pathways
• Part Three: Integrated Features and Design Principles
• Overall Summary
• Supplement: Technologies Mentioned in This Article

About This Article

The AI agent discussed in this article is a voice-interactive system that uses a Raspberry Pi Zero 2 WH as its device,
with its main processing performed on a server on AWS Lightsail.

The device is responsible only for recording, playback, and screen display.
Transcription, response generation, and integration with external services, including a database, are handled on the AWS side.

This article explains the hardware configuration, the pathway from voice input to response, what the system can actually do, and its design principles in a single cohesive flow.

Terms not explained in detail in the main text are briefly introduced in the “Supplement: Technologies Mentioned in This Article” section.

Updated as of October 2026
Gemini is used to generate conversational responses, and Groq Whisper is used for transcription.
Requests that require tools, such as web searches and file operations, are passed on to OpenClaw.

Part One: Hardware Configuration of the Raspberry Pi Zero 2 WH

Synopsis

The first Raspberry Pi I ever got my hands on was, for me, a story from a little while back.
I bought a Raspberry Pi 4 out of sheer curiosity, but chose the 2GB memory model to keep costs down.
As a result, I ended up doing little more than playing around with individual recognition that combined a webcam with a neural network,
and it was quite some time before I could reach a point where it could be called practical use.

Even so, my “Pi urge” remained.

This time, I obtained a Raspberry Pi Zero 2 WH, which is even smaller than the Raspberry Pi 4,
and decided to try making something with it.

This article is a memo summarizing the trial and error.

Motivation for Creating It

When you live alone, you become increasingly forgetful, even about yourself.

Momentarily becoming unsure of your own age
Forgetting the very things you had intended to do
Even when trying to make a shopping list, forgetting the very act of “adding it to the list”

What came to mind at such times was Iron Man. In the story, the protagonist, Tony Stark, carries on fighting while conversing with J.A.R.V.I.S., an AI agent installed inside his powered suit.

“I don’t have to fight—I could just entrust my daily life to this AI agent.” That was the thought that led me to start building an AI agent.

Goal for This Project

Implement an SF-style HUD (head-up display) on the screen
Use the words spoken via PTT (Push To Talk) as a prompt, and implement the entire sequence from speech recognition and AI-generated response text to speaking through synthesized speech
Equip it with external APIs that can answer weather questions as part of everyday conversation and control smart home appliances
Save information judged important during conversations (such as physical condition and personal details) in a database and use it in future responses to help manage diet and health
Finish off a high-tech gadget that makes every man’s heart race

This article focuses on the hardware it runs on, while briefly introducing the technologies used as well.

To put it plainly, the AI agent I built this time far exceeded my imagination and proved extremely useful.

Raspberry Pi Cooking Recipe

First, let’s introduce the hardware. Even with all the basic parts combined,
the total comes to ¥10,000–¥19,999 (if everything is bought at list price).

1. Raspberry Pi Zero 2 WH

Although it comes later in the order, the main unit that serves as the brain is the Raspberry Pi Zero 2 WH.
The Zero 2 is a single-board computer smaller than a business card, equipped with a quad-core Cortex-A53.

Item	Specifications
CPU	Quad-core Arm Cortex-A53 1GHz
Memory	512MB
Wireless	Wi-Fi (2.4GHz) / Bluetooth
Size	65mm × 30mm
Price	Approximately ¥4,000 (list price)

The notable point is that the memory is 512MB. This determined the design of this project.

...That said, the memory is 512MB.
This can be described as less than one-tenth the capacity of a modern smartphone, so
to be honest, it is inevitably limited when it comes to hosting a speech recognition model or running an LLM locally.
However, even 512MB is considered a generous amount compared with single-board computers of the same size.

For this project, however, we have limited the processing entrusted to this “brain” to sending audio captured by the microphone and playing back synthesized speech.
With this configuration, we expect it to work sufficiently well even with modest specifications.

In reality, the heavy processing is handled by the API on the cloud side.
The Pi itself serves as the interface that “sends, receives, and plays.”

Put another way, the Pi is acting as the receptionist in a large corporate office building.
Running the company is the executives’ job, but a company also needs an indispensable receptionist to receive appointments from outside and pass them along internally.

The Raspberry Pi Zero 2 WH is considerably smaller than a credit card, so it will not get in the way even when placed on a desk.
There are versions of the Raspberry Pi Zero 2 with built-in wireless LAN, designated W, and versions with a pin header, designated WH,
and if you choose the WH, you can connect it to Wi-Fi and use it as is, without any soldering.

Image: Official stock was unavailable this time, so I purchased it from Amazon at a premium price. (I’d rather not mention the price...)

Because it was shipped inside a paper bag, I was nervous until I had checked it for defective memory and broken pins.

2. Whisplay HAT

The Whisplay HAT is a HAT (expansion board) sold by PiSugar.
HAT (Hardware Attached on Top) is the standard name for expansion boards that plug into the top of a Raspberry Pi.

Image: Since the LCD appears prone to cracking, it is being used with a simple protective film made by cutting a transparent plastic sheet to fit.

This HAT is the key component for this project.

Features	Details
LCD	ST7789P3, 240×280, SPI connection. A compact 1.69-inch LCD for displaying responses and animations.
Physical Button	A PTT (Push-to-Talk) trigger that records only while held down
Microphone and Speaker	Recording and playback via the WM8960 audio codec
RGB LED	For status indication. The way it lights changes during processing.

With a standard Raspberry Pi, additional wiring, a microphone, a speaker, and buttons are required.
In other words, this single Whisplay HAT consolidates all the features needed to create the voice assistant this time.
If you use a Raspberry Pi Zero 2 WH, it is ready to use simply by inserting the pins.

That said, it must be inserted straight into the pins while one side remains elevated.
It is a delicate task, much like placing a strawberry on a shortcake that is beginning to collapse.

By combining this Whisplay HAT with a Raspberry Pi 4, 5, or Zero-series model, you can relatively easily create a portable smart gadget.

3. PiSugar 3

PiSugar is also a battery HAT designed exclusively for the Raspberry Pi Zero and sold by PiSugar. It attaches magnetically to the back, and installing it frees the Raspberry Pi Zero 2 WH from the constraints of a power cable.

Image: The Raspberry Pi Zero 2 WH board with PiSugar 3 placed on top and secured with screws. The magnets cling together strongly even without any conscious effort.

One advantage of PiSugar is that it is not merely a mobile battery; the remaining capacity and charging status can be read by software via I2C.
This allows an AI agent to perform operations such as the following:

• Notify that “the battery is running low”
• Detect and report the start of charging and a full charge
• Safely shut down when the remaining capacity enters the danger zone

Even without it, the intended functions can still be achieved.
However, if you are going to use it as a location-independent “always-with-you☆going-out AI agent,” it is an indispensable component.

About the Case

Image: From left: Whisplay HAT (display, buttons, microphone, speaker), RaspberryPi Zero 2 WH (main unit), PiSugar 3 (battery)

The basic setup is complete with these three parts.

However, with the circuit board and wiring exposed as-is, it is not exactly hygienic to carry around.

Image: RaspberryPi Zero 2 WH, Whisplay HAT, and PiSugar 3 stacked together.

If I take this outside, by the time I get home, it is certain to be a sweat-soaked, utterly worthless piece of junk.

...So, while searching for various cases, I learned that PiSugar had published the case data.

Reference: PiSugar official repository (case STL files)

If you 3D-print this STL data, you can make a case that fits this setup perfectly.

However, I had given away my 3D printer when I moved several years ago, so I had no choice but to look for a 3D printing service.
This time, I will use DMM's 3D printing service

.

With this service, all you have to do is upload the STL file, and they will print it for you using a wide range of materials, from nylon, silicone, and resin to metal.

For the material, I chose PA12GB, a relatively inexpensive, strong material made by blending glass beads into nylon.

It is an unfamiliar name, but it seems to be a material used in brake tubes, industrial hoses, sports shoe soles, and similar products.

As for the price, I had assumed it would not cost much because the parts are small. However, shell-1 was ¥5,019 and shell-2 was ¥2,595, making a total of ¥7,614.

That said, considering that it would be finished flawlessly using a high-performance industrial 3D printer, I think the price is reasonable.

Since this is for the AI agent that will handle the troublesome chores from now on, I ordered it without hesitation.

Here is the 3D-printed case (PA12GB) that arrived.

Image: Fastening screws are not included, so you will need to obtain them separately.
On GitHub, it says “M2×3 countersunk screw × 4,” but the M2 screws spun without catching, so I prepared four M2.6×3 screws.

I would have preferred black, but I forgot to specify the option, so it ended up gray, the material's natural color. The layer lines are also inconspicuous, and the finished product is sturdy and does not bend easily even when pressed with a finger.

The texture feels like smooth concrete, and even the flat surfaces feel slightly rough.

As an aside, it took the following number of days from ordering each part until its arrival.

• Pi Zero 2 WH and PiSugar 3: Arrived the day after ordering (purchased from Amazon.co.jp)
• 3D-printed case: Approximately one week (ordered and purchased through DMM)
• Whisplay HAT: Approximately two weeks (overseas shipping; purchased from PiSugar)

Having successfully combined all of these, that concludes the hardware configuration section.

Next, I will explain the architecture built into this small AI agent.

Part Two: Audio Processing and Response Routing

Overall Flow

This diagram shows "what happens where," from microphone input to response.
The small Raspberry Pi handles recording and display, while resource-intensive processing and external tools are handled on the AWS side.

Jev: Determining Whether External Processing Is Necessary

Jev is not an AI for generating response text.

In this AI agent, Jev is assignedan auxiliary rolein which, after converting speech into text, it probabilistically determines whether “this request requires external information or tools.”

For example, small talk and general questions proceed along the conversational path.

Requests that seem to require web searches, file operations, memory retrieval, and so on will also take Jev’s assessment into account and be routed to OpenClaw or LangChain.

The important point is that Jev does not replace the existing decision logic.

If it cannot connect to Jev or the time limit is exceeded, the system falls back to the conventional local decision logic.
The design therefore ensures that a malfunction of Jev alone does not stop the entire conversation.

For a detailed explanation of Jev, see the separately published article
What Is Jev? An Explanation of an AI That Excels at “Decision-Making,” Rather Than Text Generation

The AI agent being created this time routes requests according to their type.

It is not designed to send every utterance to the same model.

1. The Pi receives the audio
3. AWS normalizes the audio format
5. Transcribe it using Groq Whisper
7. Determine the type of request
9. Route it to direct processing, LangChain, Gemini, or OpenClaw
11. Return the result via audio and on screen

By dividing the flow in this way, responses to short questions can be made faster.

Requests that do not require external tools also do not go through unnecessary processing.

Input Processing

Wake word (for example, the “ア⚪︎クサ” or “Hey, S⚪︎ri” thing)

The Raspberry Pi detects phrases specified in Snowboy’s personal model and sends only detection events to AWS.
It never continuously uploads audio while waiting.

After startup, it plays a sound effect and transitions to recording, while also managing the playback state of the sound effect and backchannel acknowledgment.

Audio Format

The AI agent’s recordings are in WAV format, while browser PTT recordings may also use the WebM format.
The server converts the received audio to WAV and uses it.

By standardizing the input format, downstream processing can be unified, while also preventing failures caused by mismatches between the audio format and MIME type.

Transcription

Transcription prioritizes speed and uses Groq’s free Whisper implementation.
The result includes not only the transcript, but also confidence scores and the probability of silence.
If the speech could not be heard, it asks the user to repeat without answering.

Response Routing

Request	Route
Time, weather, current location, earthquake information	Direct response
Calendar, home appliances, notes	LangChain
Casual conversation, general questions	Gemini
Web search, file operations, memory	OpenClaw
Meeting minutes, voice memos	Dedicated continuous recording process

Requests that can be answered directly, such as greetings, bypass the model to shorten response time.
Calendar and SwitchBot requests are first routed to lightweight processing.
Only requests that combine multiple tools are passed to OpenClaw.

OpenClaw

OpenClaw is an execution environment that enables AI to carry out requests while using multiple tools. It handles requests involving multiple steps, such as searches, file operations, memory retrieval, and development work, rather than merely returning text.

In this AI agent, short questions and predetermined operations are handled first through lightweight routes. Requests are passed to OpenClaw only when multiple information sources or tools need to be combined. For development and modification requests, OpenClaw connects to Codex CLI and follows a workflow for checking the working branch, tests, and reviews.

Just as humans respond immediately to greetings and use memories or notes for research, the processing route is selected according to the content of the request.

Memory Handling

At the end of each conversation, the AI agent extracts information worth recalling later, such as preferences, health, family, and habits. It then decides whether to save it and records only the necessary information in the memory database.

This is the part I especially prioritized for my personal AI agent.

The storage destination is SQLite. The policy is to manage schedules and one-time commitments in a calendar rather than in memory.

The process of deciding whether an AI agent should save conversation content and using only the memories needed for the next conversation

In the next conversation, only memories relevant to the question are searched.
It searches not only for matching keywords but also for semantically similar content, so it can handle paraphrases such as “favorite thing” and “favorite food.”
The memories found are used as context for the response, but unrelated memories are never brought into every conversation.

Memories are assigned numbers as a mechanism to prevent another memory containing the same word from also being deleted.
By specifying a number, you can delete only that single target memory.

Model selection

Gemini is the primary model for conversational responses.
Groq Whisper is the primary model for transcription.
Code generation and self-repair use Codex CLI, signed in through a ChatGPT subscription.

If Codex CLI reaches its usage limit, the setup switches to Aider and the Gemini API as a fallback.
There is also another fallback in OpenClaw’s response path for when Gemini models are unavailable.
That said, the primary route for code generation remains Codex CLI.
(This is for the personal reason that my main subscription is currently ChatGPT Plus, as of October 2026.)

Response speed

As for the response time from a spoken instruction to vocalizing and returning the answer, because this is an AI agent designed for real-time conversation, it is important to return a response even slightly faster, so the following optimizations have been implemented.

1. Separate transcription into Groq Whisper (instead of sending audio directly to Gemini, transcribe it quickly with Groq) → approximately 0.27 seconds measured
2. Direct response path
   Information such as the time, weather, and routine details is returned immediately without going through OpenClaw.
3. Making OpenClaw lightweight
   Even when operations such as web searches or memory manipulation are required, Gemini Flash Lite is used first to streamline response generation.
4. Routing simple requests to different models
   Short questions are routed to a lightweight model, while complex questions are routed to an accuracy-focused model (Gemini Flash Lite).
5. Streaming text-to-speech
   Instead of waiting for the full response to be completed before speaking it, audio playback begins at sentence boundaries as soon as the first sentence is generated.
6. Masking waiting time with backchannels and sound effects
   At the start of processing, a sound effect or a backchannel such as “Checking that now” is returned first, reducing perceived waiting time by minimizing silent periods.
7. Various caches
   • Responses to the same small talk as before
   • Responses that include the conversation history
   • Memory prefetching
   • Reusing Gemini responses before they reach OpenClaw
     …and the like as a cache, so it answers immediately without routing through the deeper path.
8. Pre-Send Compression for OpenClaw
   It removes filler words and hesitations such as "Um," and "Oh, that..." as well as redundancy, reducing the amount of text sent.

As an example of a conversation, we tested it with the spoken prompt "How much is the exam fee for the AWS Certified Machine Learning Engineer – Associate?"
It replied, "For the AWS Certified Machine Learning Engineer, at the Associate level, the fee is 150 USD, which comes to about 22,000 Japanese yen including tax," and it took just under three seconds from the end of prompt recognition to the display of the answer.
At this level, I think the interval is not far off from that of conversation with a human.

This AI agent's design breaks processing into fine-grained parts and makes use of small devices.
The Raspberry Pi handles input and display.
AWS handles speech recognition, responses, and integration with external services.

Increasing the number of models is not the only solution.
Choosing the appropriate route for each request greatly affects speed and stability.

Finally, I'll introduce how to use the AI agent built in Parts One and Two in real daily life.

Part Three: Post-Integration Features and Design Principles

Naming the A.I.R.I.S

This is the system I have introduced up to this point as an "AI agent."

From here on, I will call the completed AI agent the
A.I.R.I.S (Artificial Intelligence Responsive Interaction and Solution).
A.I.R.I.S is a name meaning "conversational autonomous problem-solving artificial intelligence," but the real reason is that it just sounds kind of cool.

Daily Assistant

If you ask "What's the weather today?", it replies directly.
If you ask "When is the next train?", it routes the request to a web search.
If you say "Check tomorrow's schedule," it checks the linked Google Calendar and answers.

Shopping lists, memos, and health condition records can also be handled by voice.
The policy is to use saved memories in responses only for related topics.
If connected to SwitchBot, it can also operate compatible home appliances, so you can turn off the lights.
It can also notify you of battery status and earthquake early warnings, depending on the conditions.

Meetings and Research

When you say "Take the meeting minutes," it starts continuous recording.
After recording ends, it performs transcription and summarization, and finally creates a page in Notion and keeps it as a record.

You can also specify the depth of the research.
Short checks are answered by voice, while research that needs to be recorded is output to Notion.
For in-depth research, it searches from multiple perspectives and compiles the results into a report.

Self-Growing Development Tools

A.I.R.I.S is a development tool that also handles modifications to itself.
For example, if you ask it to “add a notification feature to ⚪︎⚪︎,” it examines its own current code and prepares a proposed change.

The usual change procedure is as follows:

1. Make the changes on a new Git branch
2. Run syntax checks and tests
3. Send the changes to Discord
4. Have the user review the diff
5. Merge into main after approval
6. Deploy to AWS and Pi

Even if a request is made by voice, deployment to the physical device requires user approval.

Changes are never deployed to the physical device before approval.
Because the changes can be reviewed in the pull request after they are made, it also prevents the system from unexpectedly behaving differently based only on conversation.

Current limitations

• Always-on wake-word detection consumes CPU and battery.
  If battery life is the priority, it can be disabled in the settings.

• Speech recognition and external services are subject to network conditions and usage limits.
  A fallback is already in place for when the response model is unavailable.
  However, rather than hiding every failure and continuing to operate, the cause is recorded in the logs.

• OpenClaw’s response path retains an alternative model for when the Gemini family is unavailable.
  The primary path for code creation and self-repair is Codex CLI, used through a ChatGPT subscription.

• The battery life of PiSugar 3 is extremely poor. With the current configuration, it does not last half a day.

Overall summary

This article introduced a voice-interactive AI agent using a Raspberry Pi Zero 2 WH as its device, covering the hardware, processing paths, and operational design in that order.

Key points of the configuration

• The Raspberry Pi handles recording, playback, screen display, and wake-word detection
• AWS handles speech-to-text, response generation, and integration with external services
• Depending on the request, the processing path is divided among a direct response, LangChain, Gemini, OpenClaw, and Codex CLI

This division of roles avoids placing heavy processing on a small device, enabling quick responses to everyday questions while using only the necessary processing for requests that require tools.

What mattered in operation

Speech recognition and external services involve uncertainties such as misrecognition, communication errors, and usage limits. Therefore, transcription confidence is checked, a fallback is provided in the response path, and Git diff review and approval are incorporated into development changes.

A.I.R.I.S, named in Part Three, is an AI agent that consolidates daily-life management, recording, research, and development on a single device while allowing it to be safely refined through use.
We believe the value of this configuration lies in separating the roles of the small Raspberry Pi and the cloud, and continuing to improve the system while examining logs and empirical measurements.

Summary of Creating the AI Agent

This configuration is the first prototype of a compact AI agent using a Raspberry Pi.
Although there are limitations, it achieved the intended functions and is fully satisfying as a first prototype.

There is still room for improvement, but it became a valuable learning opportunity and a foundation for future AI agent development.

Supplement: Technologies Mentioned in This Article

Groq Whisper

Groq Whisper is a speech recognition model used to convert audio data into text. In the configuration described in this article, the AWS side receives the audio, transcribes what was said, and then passes it to a route according to the type of request.

LangChain

LangChain is a framework for combining a language model with external tools. This AI agent uses it to handle requests with relatively well-defined processing, such as calendar, home appliance, and memo operations.

Aider

Aider is a coding agent that assists with modifying source code. In this article, it is positioned as an alternative route for when the Codex CLI cannot be used. Before applying changes to the actual device, the Git differences are checked and approved.

Gemini

Gemini is the generative AI primarily responsible for conversations and responses to general questions. Requests that do not require external tools are first handled through this conversational route.

Codex CLI

Codex CLI is a command-line tool for creating and verifying code in response to development and improvement requests. Changes are made on a working branch and are applied only after passing tests and review.

Jev

Jev does not generate the response text; instead, it assists in determining whether external information or tools may be needed. If Jev cannot be connected to, the system returns to the conventional local determination, so this judgment alone will not bring the entire conversation to a halt.

AWS Lightsail

AWS Lightsail is a cloud environment for running the AI agent's server. In the configuration described in this article, it receives audio sent from the Raspberry Pi, performs transcription and responses, and integration with external services.

Snowboy

Snowboy is a wake-word system that detects a designated call. Rather than continuously sending standby audio to the cloud, it detects the wake word on the Raspberry Pi and then proceeds to recording.