This is my electronic artwork inspired by Peter Vogel’s and Walter Giers works. And I am pretty happy with how it came out. It looks cool and it has artificial intelligence (to a limited extent). It recognizes when you are taking a picture of it with a mobile phone. And it uses only a microcontroller to do it. All thank to TinyML.
It consists custom-made PCB (made by PCBway) where plugged MCU (Arduino Nano PLE sense), a camera (OV7670) module and some LEDs. The largest part is the old soviet doorbell module, made in 1987. And then the speaker is connected there. I also added some status LEDs, so I will know what the MCU is currently doing.
The camera takes a picture every few seconds and MCU analyzes it. It detects things in three categories: 1) humans, 2) when someone takes a picture of it with a cell phone 3) and all other scenes.
There are also some sensors that can sense the environment:
- 9 axis inertial sensor LSM9DS1
- humidity, and temperature sensor HTS22
- barometric sensor LPS22HB
- digital microphone MP34DT05-A
- gesture, proximity, light colour and light intensity sensor APDS-9960
And there is a Bluetooth module. But I have not used it so far.
Most of my effort and time have been spent building a Machine Learning model to process images and recognise scenes. So far I have collected almost 700 images with this camera. And labelling all of them. I use Edge Impulse and it makes the process to build an ML model and library to embed it very easy.
This camera can take colour images in sizes 160×120 px, but the model uses size 96*96px. And it is quite small. You can imagine how few pixels take a mobile phone on these images. It’s just a black blob.
There is a big difference in model performance whether to use an unoptimised (float32) or the quantised (int8) model. The microcontroller’s small resources force us to use the quantised (int8) model. I have experimented with model parameters and found optimal results that work for me.
Pin layout may be different on different camera modules!
The Soviet doorbell electronics board
- My Code on GitHub
- TinyML Cookbook: Combine artificial intelligence and ultra-low-power embedded devices to make the world smarter, Gian Marco Iodice, 2022, Packt, ISBN9781801814973
Erik, this is so cool. Edge Impulse has an event in silicon valley on Sept. 28. Would it be possible to showcase this magic there? email@example.com