Fixed: chunk boundaries plus metadata in chunked streams - #205
Conversation
|
Thanks a lot for fixing the issue for http stream and showing that I started work on fixing the metadata issue for chunked streams. |
|
I use as a test stream as it very quick to fail. |
|
Well, at a quick glance I see that you changed _nextChunkSize() a bit. Looks fine by me, although I have not seen it enter without enough available the last days, has been behaving quite nicely. But checking on stream->available() rather than counting, that can never harm. And to be honest, it things fail here, some disaster already has happened. But, yes, looks good. Also see that you moved all that metadata work to a seperate function. Makes it at least a bit more readable, I guess. I can't give any comments on the details, my mental model still is rather shaky. I'll try to implement your new approach, right now I am still chasing a stubborn Australian station that just stops sending data for no apperent reason. And you know, the more I am geared up for it, the less it drops out, not giving me a chance to make up my mind ;-) |
|
It is also tempting to move _playFromRingBuffer(); to a seperate task, rather than having it run as part of the big loop. |
|
I changed If there is not enough data available in |
For multiple reasons, yes. |
|
Maybe safer to replace all |
|
But it's a very nice fix of _nextChunkSize(); Good work. For some reason I think it is good to leave stream.loop() always empty handed. |
|
I just wanted to dump an idea I had before it vanishes. Combine This massively reduces the complexity around chunk ends if done right. Same idea for the rest, use -1 as a |
|
I'm lacking insight re sync and chunksize, protocol wise, but if that really always comes as /r/n/chuncksize/r/n/ then, yes indeed, why not lift that out of the stream as one entity... |
|
About vanishing ideas, inside loop, before a return, early or at the very end, check if available is 0, which would indicate, if >0, some oppertunity is missed. Sure, leftovers will still be there in the next round, but just a gut feeling to consume every byte availaible. |
|
There might be a full ringbuffer ofcourse. But the idea has merit. |
|
So true. Ideally, you would like to be able to always store in the ringbuffer, which, btw, is usally filled up almost completely. That sortof bothers me, and actually can't get my head around it (yet) how that give&take game works. Something is about to burst, if we receive data faster than we can play away. And vice verca, if the station runs with a slower clock, would eventually give tiny silent hicks, possibly unnoticable. . I already fooled around a bit with that, but it doesn't seem to work. Didn't investigate much further, got distracted by other things. Here some preliminary stuff I wrote but didn't work. Maybe I don't have the right patches loaded. The idea was to call this function on a regular basis, trying to keep the buffer from underrunning and overrunning. |
|
Then there is that mechanism where the station/server notices the client is hungry or not, and adjusts the bitrate. Some start somewhat high, and then you see it drop slowly. Or it starts somewhat low, and see it climb up. This you see in particular with ADTS. With MP3 you sometimes see it drop from 192 kbps to 128 kbps. I also suspect that some MP3 implementations drop an entire MP3 frame, when they notice the client can't keep up. Nobody would hear that, too short. Wild guesses here. |
|
Or make it a |
Grolloo radio works the first time, bugs out second time
|
By handling the metadata first thing in |
|
Very satisfied with how this turned out. Some logging from Grolloo: [799400][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799450][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799478][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799503][I][ESP32_VS1053_Stream.cpp:909] _handleChunkedStream(): chunksize not fully read
[799515][I][ESP32_VS1053_Stream.cpp:869] _handleChunkedStream(): next chunk size: 1400
[799555][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799582][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799609][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799659][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799687][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799714][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1
[799723][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799765][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799791][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799817][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[799870][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 7000
[800026][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800078][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800104][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 601
[800131][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 800
[800141][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800182][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800209][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800235][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800287][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400
[800315][I][ESP32_VS1053_Stream.cpp:919] _handleChunkedStream(): next chunk size: 1400Looks good. Notice the chunksize 1. |
|
Fixed! |
|
Well, forget about trying to adjust the playback speed. It's a nightmare. So I ditched that. To always have enough room in the ringbuffer, to be able to fully store a really large stream->available() when it comes in, I added something to keep the max fill level around 80 percent, in loop(); Not that it helped stopping that AU station from sometimes dropping out. But hey, at least I tried. Also tried a blunt auto reconnect, which 9 out of 10 gets it running again, but sometimes still keeps dropping out 5-10 times in a row with only a short blip of sound. Which is proof (for now and for me) it is simply an unreliable station. Also added a fill percentage displayed on my puck. Sort of fun to see it go up and down. Added the necessary calback to make that happen. Overall, it's definitely getting better. |
|
Nice, to see a chunksize of 1, keeps the station running. [EDIT] Oops, seems I have to remove all the old _checkSync() stuff too. |
|
I am listening to https://tcom-s1.tcom.net.au/2ten Runs ok now. |
|
Still some minor issues but this is fixing a lot of stations on my todo list. |
|
Looks good! I added that last |
|
Stations like these now run fine: |
|
Wow, that's a lot of small chunks there. What were they thinking.. |
|
@CelliesProjects You can remove |
|
I have added/changed this now: Also commented out all refences to _playFromRingBuffer(); to keep things safe. I think this helps a bit with these obnoxious streams that are not-so-steady, causing short stalls that freeze/halt the normal loop() from doing its work, which gives short audio gaps. |
|
Technically correct, but code also shows intent. |
|
Fixes yet another set of non working stations for me. |
|
Excellent work! Pasted in your changes. I was still entertaining myself with https://tcom-s1.tcom.net.au/2ten which is 320kbps chunked station in Australia. Still gives me buffer empty so every now and then. Also doubled the I like the move of _playFromRingBuffer() to a dedicated task. When this station consistently refuses to send more data, at least it dies more graciously now, without preliminary hicks/sputtering. I was not able to link this to other network activity I do, like display communication, getting time/geolocation/wheater/mqtt etc. |
|
At what timescales? I played the au station a while but it was not dropping at all. |
|
I do a lot of websocket in the same and other tasks, and do searches through an api then parsing 40kB without any hickups. I would just try to keep the psram buffer filled btw. |
|
With a 65K buffer it could end in 5-10 minutes, 130KB seems to give instant relief. But I have seen also run for > 1 hour with 65K, so difficult to decide. And I went back to buffer filling max, to keep it around 80% was just a silly experiment. Oops, it just stopped after 6 minutes.. It's another station, sorry |
No description provided.