Would be really nice to have such a feature. The button (near the retry request) could just create a new chat starting from the last message, preserving the history up to that point. This would allow users to explore multiple directions from the same starting point.
Otherwise this feature can be also done in a simpler manner, in the list of all conversations for each item in menu can be a copy option, right below share option.
A more advanced and complete version could adopt a tree-like structure, similar to how the Orion browser organizes tabs, allowing users to see and manage different conversation branches visually.
Mostly to discuss different topics with the same context, instead of having one very long chat history, it might be N branches.
Benefits for the user:
- easier navigation
- mental clarity. trying one idea, backtrack, explore another.
- easy to experiment with "what if?" scenarios.
System benefits (assuming that branches are greatly shorter, fewer tokens, then one big prompt) :
- prompt evaluation time will be smaller
- there might be potential caching win overall.
It also scales better for exploring ideas.
For example: I have a chat with 20 messages - I can just branch out to the new one, and request will be with 21messages, and then branch out and make another request with 21 messages (the last one will be different), instead of continuing to work in one chat where first request will be with 21 messages, second with 22 messages.
So in general if I have N ideas to explore from some point for chat history with M messages, then with branching approach I will have N requests with (M+1) token load, and with one chat I will have the same N requests, but with N*M + N(1+N)/2 token load. So for branching it is linear increase vs quadratic for one big chat history.
Note regarding one chat requests: (M + 1) + (M + 2) + ... + (M+N) -> MN + (1 + 2 + .. + N) -> MN + N(1+N)/2
