A web app that lets users upload songs and get easy-to-read guitar tab. Built on libraries and models from Meta Research and Spotify's Audio Intelligence Lab, the pipeline picks out the guitar from multi-track songs, processes the audio into musical notes, intelligently places them on the guitar neck, and displays the result as readable tab.

• Inspiration
• How It's Built
• What's Next
• Architecture
• Tab JSON Format
• Getting Started
  • Prerequisites
  • Development
  • Production
  • Compose File Structure
  • Docker Compose Usage
  • Docker Build Best Practices
• Running without Docker
• Updating Dependencies
• Screenshots

What inspired us to create Tab-it is our mutual interest in playing guitar. We realized that learning songs by ear is incredibly difficult for beginners, and finding accurate tabs for niche or new songs is often impossible. We wanted to build a tool that could listen to any song and give us a starting point to learn it immediately.

The Backend: We used Python for the heavy lifting. We utilized Meta's audio-separator library (powered by the Demucs model) to perform high-quality stem separation, which is then converted to MIDI by Basic Pitch from Spotify's Audio Intelligence Lab, then using the computer-aided musicology library music21 we transform this into well-organized musical notes, which are then run through a custom fretting algorithm to place the notes on the guitar neck. This pipeline was connected via Flask endpoints (one for audio separation, and one for the rest of the pipeline and file uploading from the frontend).

The Frontend: Built with React and TypeScript. We used a component-based architecture to handle the file upload and tab display. The frontend receives a JSON object from the backend containing every note in the song, our React script maps those into a 6-string grid, and renders it as a clean text block. This lets the user visually see what string and fret to play.

Tab-it works. We developed a complete, modular pipeline that takes in songs and provides guitar tab you can read and play along to. Since this project is of personal interest to us both, we intend to continue working on it. While the pipeline outputs readable tabs, along the way notes can be dropped, added or shifted. With some model tweaking, fine-tuning, and potentially employing custom algorithms, we think this could be mitigated — allowing for very easy playing.

There are many factors to consider when placing notes onto the fretboard efficiently, and identifying chord patterns, hand shapes, and positions are all key to producing helpful tab. This is a very large area to be explored, and is a pretty pivotal part of our product, so more research and development will be put into this.

We intend to add database storage to allow the storage and retrieval of tabs, so users can build a library, and share their tabs with others. Other integrations could also be added, such as allowing users to record their own playing for feedback, or to get the tabs for their improvisations and solos. Re-examining the scalability, cost, and hosting of the product is also on the roadmap.

The backend returns a JSON object describing the fretting for the entire song. The frontend consumes this to render tab:

{
  "notes": [
    {
      "duration": 0.5,
      "strings": [2, 3],
      "frets": [5, 7],
      "time": 0.0
    },
    {
      "duration": 0.25,
      "strings": [1],
      "frets": [3],
      "time": 0.5
    }
  ]
}

Each entry in notes represents a single moment — if multiple strings are played at the same time they share one entry with parallel strings and frets arrays.

• Docker & Docker Compose
• Node.js 20+ (for local frontend dev)

The two Python services run in Docker; the frontend runs natively on the host for instant Hot Module Replacement (HMR).

# Start backends
docker compose -f compose.yaml -f compose.dev.yaml up --build

# Start frontend
cd frontend/tab-it-frontend
npm install
npm run dev

The backend services bind-mount local code into their container, and Flask runs in debug mode. So you can save a file and it auto-reloads.

All three services run in Docker. The frontend is built and served via nginx.

docker compose -f compose.yaml -f compose.prod.yaml up --build

A full --build rebuilds every service. If you only changed one service's Dockerfile or dependencies, you can target just that service:

# Rebuild and restart only the backend
docker compose -f compose.yaml -f compose.dev.yaml up --build backend

# Rebuild and restart only the auto-separator
docker compose -f compose.yaml -f compose.dev.yaml up --build auto-separator

The other service(s) will keep running untouched.

Since the images may contain heavy ML libraries, to keep development fast when adding new dependencies, install new packages live for quick testing. Instead of rebuilding, install directly in the running container:

# Install into the running backend container
docker compose -f compose.yaml -f compose.dev.yaml exec backend pip install <package>

# Install into the running auto-separator container
docker compose -f compose.yaml -f compose.dev.yaml exec auto-separator pip install <package>

This is instant but temporary — the package disappears when the container is recreated. Once you're happy with it, add it to requirements.txt and do a proper --build.

Rebuild only the service you changed. If you updated backend/requirements.txt, run up --build backend instead of up --build so you don't rebuild the auto-separator for no reason (and vice versa).

If you want to run the services directly on your machine instead of Docker:

pip install -r backend/requirements.txt
cd backend
python app.py

# PyTorch must be installed first from the official CPU index
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

# FFmpeg is required (restart your terminal after install)
winget install Gyan.FFmpeg

pip install -r auto-seperator/requirements.txt
cd auto-seperator
python audio_splitter.py

cd frontend/tab-it-frontend
npm install
npm run dev

After adding or upgrading a package, freeze the new state into requirements.txt:

pip install some-new-package
pip freeze > requirements.txt

If you're using Docker, rebuild the images so the container picks up the changes:

docker compose -f compose.yaml -f compose.dev.yaml up --build

cd frontend/tab-it-frontend
npm install some-new-package

package-lock.json updates automatically. If you're running the prod compose, rebuild to pick up the change:

docker compose -f compose.yaml -f compose.prod.yaml up --build