Summarise the context when it grows beyond the context window

[kyteinsky]

How to use GitHub

• Please use the 👍 reaction to show that you are interested into the same feature.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

---

Feature request

Which Nextcloud Version are you currently using: 34.0.0

Is your feature request related to a problem? Please describe.
The title basically.
Although the context window is not known right now so we could try to use a fixed value for now.

Additional context
See #170

[TheBest-Name]

Okay here's the thought though from my original post if you're going to be squashing down the context and concentrating it, you might be able to still implement the short-term memory by putting some of the more important stuff to the front and the reason why I think that would be maybe better than just having it be summarized, is because this would allow the user themselves to specifically tell the model the things that are important for them the user for the model to remember.
Sorry that's a little confusing the way I said that, this would be more like a bullet pointed list that would be appended to the front and would allow the model to make sure it remembers the stuff that at least the user wants it to remember, even if some other things would get lost in summarization.

Sorry to go over the same old thing but I still think it might have some use, if you still don't think so I understand.

Thank you again though for hearing this out.

[TheBest-Name]

Also this doesn't have anything to do with my first comment but this should also be a tool.

It makes sense if you think about it because if the model didn't use up all of its context but it's still starting to go a little loopy because it's getting too high for what the model is set to then at least the user of nothing else could reset it or if the model starts to see that it's kind of getting too much just by hearing itself speak. Or if nothing else it would be able to look at its own context and see that it's getting messy and so it could basically clean itself out with the tool so it wouldn't be solely based on if the context is full just if the context is getting full or too messy or if the user says that it's time to clean out.

Just an idea or a thought.