Skip to content

fix: Handle Empty SSE Terminator in LM Studio/llama.cpp - #113

Open
deathwish wants to merge 3 commits into
mlhher:mainfrom
deathwish:BUGFIX-llama_empty_sse_terminator
Open

fix: Handle Empty SSE Terminator in LM Studio/llama.cpp#113
deathwish wants to merge 3 commits into
mlhher:mainfrom
deathwish:BUGFIX-llama_empty_sse_terminator

Conversation

@deathwish

Copy link
Copy Markdown

Description of Changes

Sporadically, reading from LM Studio or llama.cpp's OpenAI endpoint would hang with "streaming" status indicated despite inference having completed, likely due to MTP tearing. Google informed me that this was an expected behavior and the fix was recognizing an empty payload as an alternative terminator. I've been running this fix for several million tokens worth of generation and it's been rock solid, no hangs or early termination whatsoever.

Contributor License Agreement (CLA)

To accept your code, we legally need you to agree to our CLA so we can maintain the project's Business Source License (BSL) and future open-source transitions.

  • By checking this box, I confirm that I have read and agree to the terms of the CLA.md in this repository. (To check the box, put an x between the brackets like this: [x])

@mlhher

mlhher commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Thank you for your contribution. If everything looks and works fine this will be merged. Interesting though since I never ran into this.

@deathwish

Copy link
Copy Markdown
Author

Awesome, thanks much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants