Hi, I'm starting to like the idea of an LLM providing me short texts for queries backed by links and references collected by Kagi. I'm currently trying to integrate FastGPT in Raycast, and while there already exists an extension, I'd like to have streaming support for a better UX (like the webpage has). I briefly peeked through the webpage JS code, and it uses SSE (using EventSource) from an endpoint called /stream_fastgpt, but explicit calls to that endpoint lead to 502s.
Would you please consider adding an endpoint for streaming FastGPT responses?
Streaming APIs would serve conventional LLM UIs very well. Although I love Orion, there are other places, where I might want to use FastGPT, like in Raycast or in iOS shortcuts.
Some examples of streaming responses are the OpenAI APIs. Although they also provide a JS SDK, you can pretty much do the same thing with axios. I can share some of my code, so as to convey what I'm trying to do.
