I created a paid Kagi account without issue - I do not recall if I originally used OAuth/SAML/"Log In With Google" for my original account creation but I can log in that way just fine, this all works as expected.
I decided to add my wife to my account and upgraded to a Duo plan. I sent her the invite email, and she tried to log in but was dumped in to a non-duo trial account instead. We repeated this process several times over several months and seemed unable to get her account to register as a linked Duo account.
What finally worked was that we deleted her account and then used the email invitation again (technically it was a new invitation but each invite seems to generate the same URL so I'm guessing it's the same invite token on the back end) ... And this time she did not use Google to log in, but rather she used an email/password combination. This worked immediately, and as she was logged in a banner popped up saying she was successfully joined to the duo account.
I'm just taking a wild guess here, but I suspect that the OAuth login flow does not inspect whatever duo account linking header/flag/cookie/whatever, and so it fails to perform the link action when the OAuth IDP calls back to the Kagi login handler.
We expected her to be able to follow the account creation instructions, which after all invited her to click the "Log In with Google" button, and successfully join a paid Kagi duo account.