This is my electronic artwork inspired by Peter Vogel’s and Walter Giers works. And I am pretty happy with how it came out. It looks cool and it has artificial intelligence (to a limited extent). It recognizes when you are taking a picture of it with a mobile phone. And it uses only a microcontroller to do it. All thank to TinyML.
It consists custom-made PCB (made by PCBway) where plugged MCU (Arduino Nano PLE sense), a camera (OV7670) module and some LEDs. The largest part is the old soviet doorbell module, made in 1987. And then the speaker is connected there. I also added some status LEDs, so I will know what the MCU is currently doing.
The camera takes a picture every few seconds and MCU analyzes it. It detects things in three categories: 1) humans, 2) when someone takes a picture of it with a cell phone 3) and all other scenes.
There are also some sensors that can sense the environment:
- 9 axis inertial sensor LSM9DS1
- humidity, and temperature sensor HTS22
- barometric sensor LPS22HB
- digital microphone MP34DT05-A
- gesture, proximity, light colour and light intensity sensor APDS-9960
And there is a Bluetooth module. But I have not used it so far.
Most of my effort and time have been spent building a Machine Learning model to process images and recognise scenes. So far I have collected almost 700 images with this camera. And labelling all of them. I use Edge Impulse and it makes the process to build an ML model and library to embed it very easy.
This camera can take colour images in sizes 160×120 px, but the model uses size 96*96px. And it is quite small. You can imagine how few pixels take a mobile phone on these images. It’s just a black blob.
There is a big difference in model performance whether to use an unoptimised (float32) or the quantised (int8) model. The microcontroller’s small resources force us to use the quantised (int8) model. I have experimented with model parameters and found optimal results that work for me.
Details
Schematic:
Pin layout may be different on different camera modules!
The Soviet doorbell electronics board
Links
- https://taunoerik.art/2022/07/13/one-pcb-designs/
- My Code on GitHub
- TinyML Cookbook: Combine artificial intelligence and ultra-low-power embedded devices to make the world smarter, Gian Marco Iodice, 2022, Packt, ISBN9781801814973
- https://en.wikipedia.org/wiki/Peter_Vogel_(artist)
- https://en.wikipedia.org/wiki/Walter_Giers
Tauno Erik 
Erik, this is so cool. Edge Impulse has an event in silicon valley on Sept. 28. Would it be possible to showcase this magic there? adam@edgeimpulse.com