Artwork that knows when you are taking a picture of it

This is my electronic artwork inspired by Peter Vogel’s and Walter Giers works. And I am pretty happy with how it came out. It looks cool and it has artificial intelligence (to a limited extent). It recognizes when you are taking a picture of it with a mobile phone. And it uses only a microcontroller to do it. All thank to TinyML.

It consists custom-made PCB (made by PCBway) where plugged MCU (Arduino Nano PLE sense), a camera (OV7670) module and some LEDs. The largest part is the old soviet doorbell module, made in 1987. And then the speaker is connected there. I also added some status LEDs, so I will know what the MCU is currently doing.

The camera takes a picture every few seconds and MCU analyzes it. It detects things in three categories: 1) humans, 2) when someone takes a picture of it with a cell phone 3) and all other scenes.

There are also some sensors that can sense the environment:

  • 9 axis inertial sensor LSM9DS1
  • humidity, and temperature sensor HTS22
  • barometric sensor LPS22HB
  • digital microphone MP34DT05-A
  • gesture, proximity, light colour and light intensity sensor APDS-9960

And there is a Bluetooth module. But I have not used it so far.

Most of my effort and time have been spent building a Machine Learning model to process images and recognise scenes. So far I have collected almost 700 images with this camera. And labelling all of them. I use Edge Impulse and it makes the process to build an ML model and library to embed it very easy.

This camera can take colour images in sizes 160×120 px, but the model uses size 96*96px. And it is quite small. You can imagine how few pixels take a mobile phone on these images. It’s just a black blob.

There is a big difference in model performance whether to use an unoptimised (float32) or the quantised (int8) model. The microcontroller’s small resources force us to use the quantised (int8) model. I have experimented with model parameters and found optimal results that work for me.

Details

Schematic:

Pin layout may be different on different camera modules!

The Soviet doorbell electronics board

Links

Pink PCB, why not

New and pink with gold! One of my experiments to combine electronics and illustrations. And actually, it will be a sculpture.
Assembly will be difficult because some holes and components are purely decorative. But others must be precise.

Final PCB made by PCBWay:

Pink PCB, Tauno Erik
Pink PCB, Tauno Erik
Pink PCB, Tauno Erik
Pink PCB, Tauno Erik
Pink PCB, Tauno Erik

Different PCB colours at different angles!

Design ideas and sketches. Although there is a lot of resistance from the software to deal with these complex graphics.

Golden Lion

I really like these golden lions. They have good contrast and blue with golden is a classic combination. There has a lot of coats of arms with this combination.

The board is just a Raspberry Pi Pico carrier board. I only needed to change the Pico footprint to remove all through-hole pins and pads from the centre of the plate. This blue colour is also different from the previous ones. It feels semi-transparent. All the fine details in gold look excellent. Manufactured by NextPCB.

Golden Lion, Tauno Erik
Golden Lion, Tauno Erik
Golden Lion, Tauno Erik
Golden Lion, Tauno Erik
Golden Lion, Tauno Erik
Golden Lion, Tauno Erik
Golden Lion, Tauno Erik

One PCB designs

Here are one PCB designs I have made recently.

In spring I bought TinyML Cookbook and there are a lot of detailed examples of how to use machine learning on microcontrollers. So I designed this board to experiment with it. On this PCB, with is made by PCBWay, is room for Arduino Nano 33 Ble sense and OV7670 camera. I also added some LEDs to illuminate camera scenes if needed. And on the side are all the rest of the pins that I can use.

So I plan to use the camera and TinyML and make one Peter Vogel style circuit sculpture/painting.

Project documentation is on GitHub.