HOWTO: Make a WLAN Python MIDI Controller with an ESP32
In this post, we’ll make a super nerdy and probably-totally-useless digital instrument.
If you’re not technical or just want to see the final product, skip to the end!
I’ve always been a software/cloud guy. The extent of my hardware experience is: I got a Boy Scout merit badge in soldering, had a few 12-year-old “do it yourself robotics!” Christmas presents, learned Ohm’s law in High School physics, and took an electrical engineering class in my Bachelor’s program - but even in that we barely touched hardware.
So when I saw that my new roommate has a self-built 3D printer, I was naturally intrigued. I asked how to learn more, and he showed me an ESP32. These things are badass. Think Raspberry Pi, but even more lightweight. If you don’t know what a Raspberry Pi is (not Pie), then think “it’s a super low cost chip with a USB port and WIFI chip which let’s you put a “cheap computer” anywhere for connectivity”.
Historically, one would have to write C/++ to interact with these chips. I haven’t touched C code in 10 years, so that’s not much of a turn on. But what’s cool is - there’s a “new” project out there called “MicroPython”. MP is a slimmed-down version of Python, and some smart people have created drivers for ESP32s (there are other boards/competitors to ESP32s, even a “MicroPython Board”). This makes the barrier of entry much lower, as Python is way more fun to write than C ;)
So my roommate hands me an ESP32 and a potentiometer (literally, a knob on a stick) and basically says “good luck, have fun”.
“Okay, what can I build with literally just a knob?”
Coincidentally, I just got a new Mac and opened GarageBand to check out the new features.
“hm, it would be pretty interesting if I could use this knob as an instrument”.
Flashing MicroPython on an ESP32
The first thing we’ll need to do is flash the chip with MicroPython, so we can write Python to interact with the chip!
There’s already a dedicated page to this here, so instead, I’ll just include some details I noticed:
On your computer, you’ll need to
pip install esptoolDownload the correct firmware for the chip that you have. I have an “offbrand” ESP32, so I just used the generic ESP32 firmware.
Follow the short instructions to complete the flash
The ESP32 is the successor to the ESP8266, so if you ever get stuck, you should also be able to follow those instructions. The getting started guide even says “From here you can now follow the ESP8266 tutorial, because these two Espressif chips are very similar when it comes to using MicroPython on them.”
In order to interact (write code) with the chip, you’ll need to use an interactive REPL prompt. There are different tools for this depending on which OS you use.
Typically: Windows - TeraTerm, Linux - picocom/minicom, Mac - ScreenScreen is not sooo beautiful to work with. The commands are kind of clunky.
Important notes regarding Screen and ESP32:I’m running OSX Catalina which apparently blocks ESP32 from automatically installing drivers. To check if you need the drivers, run
ls /dev/cu*and look forSLAB_USBtoUARTIf you don’t see it, you’ll need the driver(s).
Here’s a resource with links to the driver you need for which board
(or you can just install both 😬)When you run screen. make sure to use the SLAB_USBtoUART port and not the usbserial… port
When you run
screen /dev/cu.SLAB_USBtoUART 115200and don’t see anything, you might have to press Return for the prompt to display.I encountered a few online resources about “how to exit Screen” which didn’t work (1). Additionally, some of them resulted in me killing Screen but not stopping the REPL on the device, resulting in the error “Cannot open line … resource busy”.
The correct command to cleanly kill Screen for me is CTRL+A, k, confirm
MIDI and OSX
By default, Macs don’t come with a MIDI output port automatically enabled, so you’ll have to do that to continue. You can follow the instructions from Ableton here.
The important thing to note is at the end, you should have an active IAC Driver.
Connecting the Potentiometer to the ESP32
This is dependent on your potentiometer so hopefully the terminals are labelled or you have a manual to go with it. Here’s a pretty general write up. Make sure to ground it!
Python and MIDI
It was actually not so easy to find a Python 3 library which interacts with MIDI controllers. I tried a few but I’ll cut to the chase: Mido is the way to go, and it’s simple to use!
ESP32 has an internal filesystem of two files: boot.py and main.py, but I was cheating and just copying/pasting code directly into the Python REPL in order to save time and test fast. In order to copy/paste code using screen, press CTRL+E, paste your code, and CTRL+D to finish.
To begin, we’ll need to write the host (ESP32) code, then we’ll need to write the client (personal computer) code to ping the ESP32 and do the MIDI conversion.
Instead of walking through the code here, I commented directly on Github!
I suggest reading host.py first, and then client.py
Design Considerations
WLAN vs USB
The host<-> client communication takes place via WLAN.
Necessary? No.
Practical? Not at all (an instrument that first has to go through your router!?)
Badass? I mean, you can walk around your house and DJ while you make eggs…
So, sure, we could have the ESP32 talk directly to the computer via USB, but that would be boring.
MIDI Note vs MIDI Pitch
At first, I had every response from the ESP32 stop the current note, calculate a new note based on the position of the dial, and then play a new note, which allows for a greater range of melody, but in practice results in the start / stop of notes every 0.05s 😱
Instead, I play one note when the program starts, the knob adjusts the note pitch, and the note stops when the program stops. There’s actually quite a big range of melody in the MIDI pitch definition, so this was a reasonable trade off.
Garageband Setup
Garageband now has “AI Drummers”. As a drummer, I’m not sooooo happy about this 😅, but for this project, I just threw down a Deep Tech track and voilà..
For the Bass, I just put a simple two note track on repeat.
Garageband should automatically detect the MIDI input if it’s set up correctly.
The cool thing about the MIDI instrument is that you can choose any instrument, rerun the program, and have a new sound. I dig the “Classic Techno Lead”
Future Ideas
It would be relatively simple to connect a hardware button and do something with the input. This would have a range of uses - automatic tempo, note on / off, etc.
But then, why not just create a full blown midi controller?
Final Product 🔊
Here it is! Completely useless but fun to build!
Now it’s time to go DJ in the kitchen… 🤩