⚠️ This project is still under development!
Web UI & tray: This repository includes an optional browser-based Web UI and system tray integration that wrap the main UltraSinger CLI. That layer was produced only with AI-assisted coding as an add-on to the core app; it is not part of the official upstream UltraSinger project and is unofficial. Use it at your own discretion; bugs or behavior should be reported in this fork/repo, not assumed to be maintained by the original UltraSinger authors.
UltraSinger is a tool to automatically create UltraStar.txt, midi and notes from music. It automatically pitches UltraStar files, adding text and tapping to UltraStar files and creates separate UltraStar karaoke files. It also can re-pitch current UltraStar files and calculates the possible in-game score.
Multiple AI models are used to extract text from the voice and to determine the pitch.
Please mention UltraSinger in your UltraStar.txt file if you use it. It helps others find this tool, and it helps this tool get improved and maintained. You should only use it on Creative Commons licensed songs.
There are many ways to support this project. Starring ⭐️ the repo is just one 🙏
You can also support this work on GitHub sponsors or Patreon or Buy Me a Coffee.
This will help me a lot to keep this project alive and improve it.
- UltraSinger
- Install Python 3.12 (older/newer versions may have compatibility issues). Download
- Also download or install ffmpeg with PATH. Download
- Go to folder
installand run install script for your OS:- Choose
GPUif you have an NVIDIA CUDA GPU. - Choose
CPUif you don't have an NVIDIA GPU or want CPU-only processing.
- Choose
- In root folder just run
run_on_windows.bat,run_on_linux.shorrun_on_mac.commandto start the app. - Now you can use the UltraSinger source code with
py UltraSinger.py [opt] [mode] [transcription] [pitcher] [extra]. See How to use for more information.
The Web UI is an optional, AI-only-coded integration layered on top of the main UltraSinger application. It is unofficial and separate from the core maintainers’ scope.
What it provides
- Browser UI for queuing jobs (URLs, playlists, file uploads), live logs, per-job ZIP downloads, settings, and cleanup.
- Optional desktop system tray mode (
pystray): run the server in the background with a tray menu (open UI, jobs, stop all, resume, exit).
Art & visuals: The Web UI look (layout, CSS, inline logo graphic, tray icon, and similar assets) is homemade—simple project-made styling, not a separate design system or stock artwork package.
For developers: This add-on is here to fill a gap—a friendlier entry point so more people can use UltraSinger without living in the terminal. It is not meant to stay as a permanent AI-scaffold forever. The end goal is for someone to overhaul the webui/ code: audit it, refactor it, and replace it with human-written, human-maintained code you stand behind. Treat what is here as a bootstrap; PRs and rewrites toward that outcome are welcome.
Install (from the repository root, with your UltraSinger venv active):
pip install -e ".[webui]"Linux/macOS convenience scripts:
./install_webui_linux.sh./install_webui_macos.sh./install_webui.sh(auto-select based on OS)
Run
- Web UI only: set
PYTHONPATHto the repo root, thenpython -m webui(default:http://127.0.0.1:8756). On Windows you can usestart_ultrasinger_webui.bat(orstart_ultrasinger_webui.bat --hiddento run without a visible terminal). On Linux/macOS usestart_ultrasinger_webui_linux.sh/start_ultrasinger_webui_macos.commandif.venvexists in the repo root. - Tray: set
tray_enabledtotrueindata/webui_config.json(or use the Settings page), then start withpython -m webuiagain.
Configuration and job data live under data/ (for example webui_config.json, jobs/). See the webui/ package for implementation details.
This subsection (Web UI settings, jobs/downloads behavior, environment variables, and the related CLI flags listed here) was updated with AI assistance to reflect newer options—double-check against the running Web UI and webui/ code if anything disagrees.
Settings are edited in the browser (Settings page) and stored in data/webui_config.json (under your configured data directory).
| Area | Option | Notes |
|---|---|---|
| System | Data directory | Blank = repository data/. |
| Listen host / port | After changing host or port, restart the app. | |
| Processing | Force CPU / Force Whisper on CPU | Passed through to UltraSinger. |
| Whisper model, compute type, batch size | Compute type is a dropdown (CTranslate2 / faster-whisper values); empty = UltraSinger default (float16 on CUDA, int8 on CPU). | |
| Demucs model, FFmpeg path, yt-dlp PATH | Extra directory for yt-dlp on PATH. |
|
| Home (queue from URL) | Use YouTube metadata only | Per queue action: skips MusicBrainz for that job; uses yt-dlp title/artist for folder names and tags. |
| YouTube metadata background screenshot time (%) | When metadata-only mode is used, captures one frame from the downloaded video at this % of total duration and writes *[BG].jpg (also updates #BACKGROUND in the UltraStar txt). Default: 30. |
|
| yt-dlp cookies | Paste or file path | Netscape cookies.txt; saved under data when pasted. |
| Default job | YARG mode | Omits writing .mid for WebUI jobs unless UltraStar folder export is enabled with a path (see below). With YARG mode on, ZIP downloads (per job and bulk) use the same export-folder-style names inside each song folder (guitar.*, background.*, notes.txt, album.*; no .mid in the archive). |
| Delete Demucs/Whisper cache after success | UltraSinger --keep_cache is omitted when this is on. |
|
| Downloads | ZIP: omit Instrumental/Vocals | Applies to per-job ZIP and “download all”; does not delete files from the job folder. With this on and YARG mode off, only the cover is renamed to <song folder>/album.<ext>. If YARG mode is on, stem omission still applies to which files are included, and the whole ZIP layout follows export-style renaming (see Default job). |
| Enable YARG folder export + path | After each successful job, writes a flat folder <path>/<Artist - Title>/ with fixed names (same inclusion rules as the YARG-style ZIP; no .mid): guitar.<ext>, background.<ext>, notes.txt, album.<ext>. |
|
| Enable UltraStar folder export + path | Copies full song output (.txt, .mid, instrumental/vocals audio, cover, video, etc.) into <path>/<Artist - Title>/. When this is enabled and the path is non-empty, the worker runs UltraSinger without --yarg_mode so .mid and stems exist even if “YARG mode” is checked. |
|
| Storage & cleanup | Job retention, cleanup on startup / interval | Background cleanup of old job metadata. |
| Debug | Verbose server logging | |
| Tray | Tray options | Require starting via python -m webui (or the start script). |
Jobs page: Stop all pauses the queue and cancels the currently running job. Cancel all jobs cancels every queued and running job without toggling pause. Resume unpauses; Clear finished removes finished rows. Per row: Cancel, Retry (clears that job’s output/ before re-queue), Clear.
Downloads page: Per-job ZIP saves as <song folder>.zip (same name as the UltraSinger output folder under output/, e.g. Artist - Title.zip). “Download all” builds one archive whose entries are grouped under each song folder name (not job_… prefixes); duplicate song names in one batch get a [job_id] suffix on the folder. Bulk download file: completed_songs.zip. Delete / Delete all completed, bulk ZIP prepare.
Environment: Jobs and the queue persist across WebUI restarts (data/jobs/<job_id>/job.json plus data/jobs/queue.json for order and pause). On shutdown, a rolling JSON export is written under data/history/job_backups/ (keeps the 5 newest snapshots). Set ULTRASINGER_WEBUI_CLEAR_JOBS_ON_START=1 to wipe jobs/ on startup (previous “fresh queue each launch” behavior). If YARG or UltraStar export paths (Settings) point at existing folders, the worker skips running UltraSinger when a subfolder already exists whose name matches the job’s Artist - Title (case-insensitive). Set ULTRASINGER_WEBUI_FORCE_REEXPORT=1 to disable that skip. WEBUI_NO_BROWSER=1 forces no browser tab on start (overrides Settings “open browser on start”). ULTRASINGER_WEBUI_HOST and ULTRASINGER_WEBUI_PORT override listen address and port (e.g. Docker); they take precedence over values in webui_config.json.
CLI (non-WebUI): --youtube_metadata uses only yt-dlp metadata for YouTube URLs (no MusicBrainz). --yarg_mode skips writing .mid. See UltraSinger.py -h for full help.
Not all options working now!
UltraSinger.py [opt] [mode] [transcription] [pitcher] [extra]
[opt]
-h This help text.
-i Ultrastar.txt
audio/video like .mp3, .mp4, .wav, youtube link
-o Output folder
[mode]
## if INPUT is audio ##
default (Full Automatic Mode) - Creates all, depending on command line options
--interactive - Interactive Mode. All options are asked at runtime for easier configuration
# Single file creation selection is in progress, you currently getting all!
(-u Create ultrastar txt file) # In Progress
(-m Create midi file) # In Progress
(-s Create sheet file) # In Progress
## if INPUT is ultrastar.txt ##
default Creates all
[separation]
# Default is htdemucs
--demucs Model name htdemucs|htdemucs_ft|htdemucs_6s|hdemucs_mmi|mdx|mdx_extra|mdx_q|mdx_extra_q >> ((default) is htdemucs)
[transcription]
# Default is whisper
--whisper Multilingual model > tiny|base|small|medium|large-v1|large-v2|large-v3 >> ((default) is large-v2)
English-only model > tiny.en|base.en|small.en|medium.en
--whisper_align_model Use other languages model for Whisper provided from huggingface.co
--language Override the language detected by whisper, does not affect transcription but steps after transcription
--whisper_batch_size Reduce if low on GPU mem >> ((default) is 16)
--whisper_compute_type Change to "int8" if low on GPU mem (may reduce accuracy) >> ((default) is "float16" for cuda devices, "int8" for cpu)
--keep_numbers Numbers will be transcribed as numerics instead of as words
[extra]
--disable_hyphenation Disable word hyphenation. Hyphenation is enabled by default.
--disable_separation Disable track separation. Track separation is enabled by default.
--disable_karaoke Disable creation of karaoke style txt file. Karaoke is enabled by default.
--create_audio_chunks Enable creation of audio chunks. Audio chunks are disabled by default.
--keep_cache Keep cache folder after creation. Cache folder is removed by default.
--plot Enable creation of plots. Plots are disabled by default.
--quantize_to_key Quantize notes to detected musical key. Removes pitch slides and out-of-key notes. >> ((default) is enabled)
--format_version 0.3.0|1.0.0|1.1.0|1.2.0 >> ((default) is 1.2.0)
--musescore_path path to MuseScore executable
--keep_numbers Transcribe numbers as digits and not words
--ffmpeg Path to ffmpeg and ffprobe executable
[yt-dlp]
--cookiefile File name where cookies should be read from
--youtube_metadata For YouTube URLs, use yt-dlp title/artist only (skip MusicBrainz)
[device]
--force_cpu Force all steps to be processed on CPU.
--force_whisper_cpu Only whisper will be forced to cpu
[export / Web UI related]
--yarg_mode Do not write a .mid file (UltraStar txt and audio outputs unchanged)
For standard use, you only need to use [opt]. All other options are optional.
default (Full Automatic Mode) - Creates all, depending on command line options --interactive - Interactive Mode. All options are asked at runtime for easier configuration
--interactive
-i "input/music.mp3"
-i https://www.youtube.com/watch?v=YwNs1Z0qRY0
Note that if you run into a yt-dlp error such as Sign in to confirm you’re not a bot. This helps protect our community (yt-dlp issue) you can follow these steps:
- generate a cookies.txt file with yt-dlp
yt-dlp --cookies cookies.txt --cookies-from-browser firefox - then pass the cookies.txt to UltraSinger
--cookiefile cookies.txt
This re-pitch the audio and creates a new txt file.
-i "input/ultrastar.txt"
Keep in mind that while a larger model is more accurate, it also takes longer to transcribe.
For the first test run, use the tiny, to be accurate use the large-v2 model.
-i XYZ --whisper large-v2
Currently provided default language models are en, fr, de, es, it, ja, zh, nl, uk, pt.
If the language is not in this list, you need to find a phoneme-based ASR model from
🤗 huggingface model hub. It will download automatically.
Example for romanian:
-i XYZ --whisper_align_model "gigant/romanian-wav2vec2"
Is on by default. Can also be deactivated if hyphenation does not produce anything useful. Note that the word is simply split, without paying attention to whether the separated word really starts at the place or is heard. To disable:
-i XYZ --disable_hyphenation
Pitching is done with the SwiftF0 model, which is faster and more accurate than CREPE.
SwiftF0 automatically detects pitch frequencies between 46.875 Hz (G1) and 2093.75 Hz (C7).
UltraSinger uses 60hz and 400hz
The vocals are separated from the audio before they are passed to the models. If problems occur with this, you have the option to disable this function; in which case the original audio file is used instead.
-i XYZ --disable_separation
By default, UltraSinger quantizes all detected notes to the detected musical key of the song. This helps to:
- Remove pitch slides and vocal transitions between notes
- Correct out-of-key notes from pitch detection errors
If you want to keep the raw pitch detection without quantization, set quantize_to_key = False.
For most songs quantization should produces better results.
For Sheet Music generation you need to have MuseScore installed on your system.
Or provide the path to the MuseScore executable.
-i XYZ --musescore_path "C:/Program Files/MuseScore 4/bin/MuseScore4.exe"
This defines the format version of the UltraStar.txt file. For more info see Official UltraStar format specification.
You can choose between different format versions. The default is 1.2.0.
0.3.0is the first format version. Use this if you have an old UltraStar program and problems with the newer format.1.0.0should be supported by the most UltraStar programs. Use this if you have problems with the newest format version1.1.0is the current format version.1.2.0is the upcoming format version. It is not finished yet.2.0.0is the next format version. It is not finished yet.
-i XYZ --format_version 1.2.0
The score that the singer in the audio would receive will be measured. You get 2 scores, simple and accurate. You wonder where the difference is? Ultrastar is not interested in pitch hights. As long as it is in the pitch range A-G you get one point. This makes sense for the game, because otherwise men don't get points for high female voices and women don't get points for low male voices. Accurate is the real tone specified in the txt. I had txt files where the pitch was in a range not singable by humans, but you could still reach the 10k points in the game. The accuracy is important here, because from this MIDI and sheet are created. And you also want to have accurate files
With a GPU you can speed up the process. Also the quality of the transcription and pitching is better.
You need an NVIDIA CUDA device for this to work. Sorry, there is no CUDA device for macOS.
For GPU support on Windows and Linux, the installation script automatically installs PyTorch with CUDA support.
It is optional (but recommended) to install the CUDA driver for your GPU: see CUDA driver. Also check your GPU CUDA support. See CUDA support
For manual installation, you can use:
uv pip install --index-url https://download.pytorch.org/whl/cu121 torch torchvision torchaudioIf something crashes because of low VRAM then use a smaller Whisper model.
Whisper needs more than 8GB VRAM in the large model!
You can also force CPU usage with the extra option --force_cpu.