About

This app lets you play piano using a printable two-page layout. Just download the PDF, print and tape the pages together, then point your webcam at it and start playing. Watching the tutorial first is highly recommended to get the best experience.

This project has a fundamental limitation: the computer can't tell if you're actually pressing a key. If your finger appears within a key’s coordinates, it counts as a play, even without real contact (or touch). So the printed layout is mostly for visual guidance. That’s why I recommend using the dummy layout and simply playing in the air.

Huge thanks to the University of Iowa Electronic Music Studios for providing high-quality, publicly available recordings of piano notes.

How it works:

This app uses a machine learning model to detect small and large polygons (red and blue regions). It then maps three white keys and two black keys onto each small polygon, and four white keys and three black keys onto each large polygon. MediaPipe is used to detect your fingers and check whether they intersect with any key's coordinates.

An offline Python version is also available on the GitHub repo. Just install the requirements using pip, and you can play without an internet connection. It's even faster and more accurate than this web app.

FAQ:

1. Do I need to use your layout, or can I make my own?

The layout I provided was used to train the model, so custom layouts may not work reliably. You’re free to try your own, but be prepared for possible disappointment.

2. Will it work on mobile or other small-screen devices?

Technically, yes—but this is an unresponsive app, so the layout may appear broken or awkward. It's strongly recommended to use a desktop or a device with a resolution higher than 1024×720 pixels.

Privacy Policy:

This app uses your browser's local storage to store note sounds, model data, and some other variables. However, none of your personal data is collected. The app runs entirely in your browser with no backend—it's just a few static HTML files.

Third-party libraries are used to power the app: ONNX Runtime (for running the model), OpenCV.js (for image processing), and MediaPipe (for finger tracking). These are safe, widely-used, and industry-standard tools.