Allow for LLM use through personal API token/pay per token - bring your own key

Vlad

pophob Lets expand this with concrete suggetsionf for UI:

How do you imagine this option would be surfaced to the user?
How do you imagine all available models from the provider would be shown for the user?

Pay per token is also interesting, basically passing on the API cost to the user, the problem is the requires additional billing.

rxzlion

I also agree with the option to pay per token.

Vlad like said in OldMan973471 post and like I said in discord I think the easiest way would be:

let ultimate users have the ability to load credit into their account
once you hit a certain usage threshold it would cost you credits
have a toggle in option to enable credit usage
have an option to set a max usage limit
having a table with price for each model token
having a graph that shows usage per model

I think something like in Anthropic API interface I know you probably know it but I added image bellow for others to see what I mean.

That would be my preferred way do to and not API key for the following reasons:

I have more trust in kagi then other companies
I want to show support for kagi
I will need to open an account with each of said companies and provide them with my data
Defeats the purpose of using kagi as a privacy respecting option

Your AI tools are very strong especially KI and having that power and a your respect of user privacy is something special.

also is it possible to merge this into here:
https://kagifeedback.org/d/6256-kagi-byok-bring-your-own-key

pophob

Would it be possible to give users a $5 monthly allocation for 'experimental' models such as o1 as part of the Ultimate plan? These models could be indicated with a little beaker icon in The Assistant UI. The $5 allocation could be communicated similarly to how search usage is currently shown on the Billing page for the Starter plan.

Maybe on the Billing page, a user could also specify an additional spending limit for additional use. I'd also be happy to pre-pay credit towards these 'experimental' models. (That's how I do it through the OpenAI API currently)

OldMan973471

This could solve the issue where some of us want to pay for our AI usage out of our own pockets without impacting Kagi. Similar to the API bucket, have an AI bucket where we can fill that with credit or billing information - and then check a box that says: "Use My AI Bucket" with the idea that limits don't apply - track them separately from the AI tokens when the "Use My AI Bucket" is unchecked.

When checking "Use My AI Bucket" - show a screen that allows API Keys from the popular LLMs to be put in so on top of allowing an AI Bucket of $, one can use their own API keys (pulling from that instead of Kagi or the AI Bucket of money).

dreifach

rxzlion Agreed, this interface and pre-billing would be my preference too.

mg127

rxzlion I agree with this, except I would prefer it not to be scoped only to Ultimate users. Now that I have access to Assistant as a Professional user (which was maybe made available after you made this post), I would like the ability to use the more expensive models on a pay-as-you-go basis, without having to upgrade to Ultimate.

This would be similar how I currently use OpenAI and Anthropic models myself - I pre-load billing credits and use the models via API. For my usage at least, this is significantly cheaper than $20+ monthly.

OldMan973471

That is a perfect example - wow 18.7 million tokens on some days, haha. I think the most I ever did was 5 million in one day. I would love to throw that $ Kagi's way - hoping they take a little off the top to keep business running. 😃

I don't even use LLM APIs to program and I can hit million tokens per day really quick with basic questions and follow up questions.

OldMan973471

Please consider allowing AI BYOK - bring your own key for the popular LLMs in Kagi Assistant. This would allow those of us who are heavy AI users not hurt Kagi's bottom line and use as many tokens as we want.

Allow filling of an BYOK Bucket with $ just like the API area. Show this amount on Kagi Assistant when the user chooses the BYOK LLM (an option?).

Usage of BYOK AI wouldn't count against the account's Token Count, but would be counted for user convenience.

For Ultimate - allow putting in an API key in the settings and selecting that BYOK LLMs from the Assistant drop down. Maybe allow Kagi Assistant with Fast GPT and BYOK LLMs for the Professional subscription tier.

rxzlion

This could be a good solution.

But like I said on the discord I would prefer to be able to pay kagi for extra token usage for the following reason:

I have more trust in kagi then other companies
I want to show support for kagi
I will need to open an account with each of said companies and provide them with my data
Defeats the purpose of using kagi as a privacy respecting option

OldMan973471

rxzlion Agreed, I think your idea is better. I wonder if that's easier for Kagi to implement than a BYOK option.

pophob

rxzlion

pophob

can we merge this 2 suggestions?

OldMan973471

I'm ok merging them if that's what Kagi / y'all want. More votes on this is a plus - I see more and more chat in Discord about people asking for more tokens so they can keep using the tools they love without hurting Kagi. 😃

afpco

No one has mentioned it, but there are dozens of models that are not in the Assistant. There may be people like me who want to use them.

scottu

I already have a subscription to Claude and I feel like a chump being made to upgrade my Kagi account to something that I arguably already pay for.

I'd like to be able to link my existing Claude account into Kagi Assistant as an option.

I think this is just something you'd add to settings.

RoxyRoxyRoxy

scottu Just to clarify on this point, this wouldn't be available for a regular Claude Pro subscription, you'd need to get a Claude API key as well.