#desktop

My chat app streams now, and half answers still never reach its memory

I made my desktop chat app stream answers from a local model as they're written, while its conversation log, which gets fed back to the model, only ever saves a finished question-and-answer pair. Plus the piece counter that keeps a fake stream from passing tests, and the gaps I haven't closed.