Boards Guide 2025: AI at the Edge

Raspberry Pi 5

• CPU: Broadcom BCM2712 quad-core 64-bit Arm Cortex-A76 @ 2.4GHz
• GPU: VideoCore VII @ 1GHz
• RAM: 2, 4, or 8GB
• Flash: Your own SD card or SSD

The Raspberry Pi 5 is an accessible, low-cost single-board computer ideal for edge AI applications. It offers a powerful CPU and the ability to run full versions of machine learning frameworks like PyTorch and TensorFlow. Additionally, it supports some GPU acceleration when performing inference but not for training. Object detection (YOLOv8n) for images with 640×640 resolution runs at about 5 FPS, and small LLMs (Llama8:3B) can generate responses at about 2 tokens per second. USB and PCIe slots allow for the addition of various AI acceleration hardware, which can dramatically increase the speeds of the above metrics.

Notes:

• Easy to use, acts like a cheap, low-power computer
• Runs full version of most ML frameworks (e.g. PyTorch, TensorFlow)
• Fast CPU
• No GPU support for training (yet)
• Some GPU support for inference
• Training is possible, but will be slow
• Runs object detection at a reasonable rate, local LLMs very slow
• USB and PCIe = possibility for external accelerators

Nvidia Jetson Orin Nano

• CPU: 6-core 64-bit Arm Cortex-A78AE @ 1.5GHz
• GPU: 512-core (16 tensor core) or 1024-core (32 tensor core) Nvidia Ampere @ 625MHz
• RAM: 4 or 8GB
• Flash: Your own SD card or SSD

The Nvidia Jetson Orin Nano is a very powerful single-board computer with an integrated Nvidia GPU that enables fast inference. While training on the device is feasible, it remains slow. The Nvidia-specific frameworks present a longer learning curve compared to simpler devices like the Raspberry Pi, and the Orin Nano is significantly more expensive, starting at around $500. Object detection (YOLOv8n) for images with 640×640 resolution runs at about 30 FPS, and small LLMs (Llama8:3B) can generate responses at about 4 tokens per second. Like the Pi, the Orin Nano has USB and PCIe slots for potential hardware acceleration accessories.

Notes:

• Integrated NVIDIA GPU for very fast inference
• Nvidia-specific frameworks, more difficult to use than RPi
• Training is possible but will be slow
• Runs object detection very fast, local LLMs slow
• Expensive (~$500)

Seeed Studio Xiao ESP32S3 Sense

• CPU: Xtensa dual-core 32-bit LX7 @ 240MHz
• RAM: 8MB
• Flash: 8MB
• Hardware optimizations: SIMD, DMA, FPU
• Software optimizations: ESP-DL, ESP-DSP

The Seeed Studio Xiao ESP32S3 Sense is a compact and powerful IoT microcontroller board capable of handling many small AI tasks, such as audio, vibration, and image classification. It can’t handle full object detection (e.g. YOLOv8n), but can use Edge Impulse’s FOMO model to perform constrained object detection at about 8 FPS with 96×96 resolution. Additionally, the onboard WiFi/BLE radios offer enhanced connectivity for your projects. The S3 model relies on dual-core Xtensa LX7 processors with AI acceleration whereas the C3 model has only a single RISC-V core with no such acceleration. The Xiao Sense kit has an impressively small form factor and a stackable add-on board with a camera and microphone.

Notes:

• Great for accelerometer, vibration, audio, image classification, light object detection (e.g. FOMO)
• Integrated WiFi and Bluetooth
• Accessory board has microphone and camera

Raspberry Pi Pico 2

• CPU: Arm dual-core 32-bit Cortex-M33 @ 150MHz
• RAM: 520kB
• Flash: 4MB
• Hardware optimizations: SIMD, DMA, FPU
• Software optimizations: CMSIS-DSP, CMSIS-NN

Raspberry Pi’s newest microcontroller, the RP2350, is a significant upgrade from the previous RP2040. The dual-core Cortex-M33 offers specialized hardware optimizations like SIMD, DMA, and an FPU, making it effective for processing time-series data and performing inference on simple machine learning tasks, such as audio, vibration, and image classification. While it lacks the connectivity options of the ESP32S3, the official Raspberry Pi Pico 2 board is cheaper than similar ESP32 boards, and it should perform similarly well on most AI tasks. Raspberry Pi’s comprehensive reference documentation also aids in custom board layouts for specific projects or products.

Notes:

• Basic microcontroller board (no Bluetooth/WiFi)
• Reference docs–easy to lay out a board for own project/product
• FPU and DSP instructions for time-series data (e.g. vibration, audio)
• Dual-core (one for general app, one for inference)

Arduino Nano 33 BLE Sense Rev2

• CPU: Arm 32-bit Cortex-M4 @ 64MHz
• RAM: 256kB
• Flash: 1MB
• Hardware optimizations: SIMD, DMA, FPU
• Software optimizations: CMSIS-DSP, CMSIS-NN

The Arduino Nano 33 BLE Sense is built around Nordic Semiconductor’s nRF52840 module. It offers less computational power than the ESP32S3, which means less power consumption. The onboard BLE radio offers some connectivity for IoT options, such as communicating with a nearby smartphone. The single-core Cortex-M4 processor is capable of performing light AI duties, such as audio and vibration classification. However, it struggles at processing image data due to its limited CPU speed and available RAM. Expect to see 1–2 FPS for basic image classification tasks with monochrome 30×30 resolution input. Even constrained object detection with FOMO will likely be too slow to be useful. The built-in microphone, IMU, temperature and humidity sensor, and gesture sensor offer a variety of data for AI tasks.

Notes:

• Lots of built-in sensors
• Still a great low-power platform to learn on
• Good for time-series data (accelerometer, audio, etc.)
• Not good for image classification (slow, need external camera)

Off-the Shelf Edge AI Boards

Thanks to the increasing popularity of edge AI, several companies have started offering premade, off-the-shelf AI boards. These come with pre-trained machine learning models and the necessary sensors to perform the desired task, such as classifying images and keyword spotting. They are intended to work as intelligent, stand-alone sensors that can be connected to a microcontroller board or single-board computer. Here are a few boards we recommend checking out:

• Seeed Studio SenseCAP Watcher – The Watcher looks for a pre-defined object, keyword, or gesture on the device to enable it, much like saying “Alexa” or “Hey, Siri.” It then sends subsequent images and audio to a more powerful, internet-connected LLM for further processing. It’s like a tiny, hackable Amazon Echo Show.
• Seeed Studio SenseCAP A1101 – The SenseCAP A1101 is an industrial camera with built-in processing to perform image classification and object detection. It includes an IP66-grade waterproof enclosure for outdoor use.
• Useful Sensors Person Sensor – The Person Sensor looks for faces in the camera’s frame and sends a notification via I2C.
• DFRobot Gravity – The Gravity is an offline voice recognition module that responds to 121 preprogrammed words and can be customized with up to 17 user-created command words.

This article appeared in Make: Vol. 91.