Does my prompt get erased from the chat context/history? Asking in the sense that if I am in the middle of a chat, can I abort the last message that the ai is currently thinking about, and then write a different message as if the ai never saw the aborted message?
Thanks for this by the way! A llama.cpp chat client that is html + js only, is the only way to go! Not Open-WebUI which uses like 5gb of ram that I'd otherwise give to the llm, and needs hours to set up. Not some sketchy.exe you gotta download from a trust me bro repo and doesn't run in the browser. Plus I can tweak it myself!
PS: have you looked at hot swapping llama.cpp models via api call? https://www.reddit.com/r/LocalLLaMA/comments/1pk0ubn/new_in_llamacpp_live_model_switching/
Does my prompt get erased from the chat context/history? Asking in the sense that if I am in the middle of a chat, can I abort the last message that the ai is currently thinking about, and then write a different message as if the ai never saw the aborted message?
Thanks for this by the way! A llama.cpp chat client that is html + js only, is the only way to go! Not Open-WebUI which uses like 5gb of ram that I'd otherwise give to the llm, and needs hours to set up. Not some sketchy.exe you gotta download from a trust me bro repo and doesn't run in the browser. Plus I can tweak it myself!
PS: have you looked at hot swapping llama.cpp models via api call? https://www.reddit.com/r/LocalLLaMA/comments/1pk0ubn/new_in_llamacpp_live_model_switching/