I've been looking for an API that leverages LLMs and is reasonably priced to convert a URL to Markdown (basically html to markdown) efficiently. It's not that easy to do this cost-effectively, and I've had somewhat bad/mixed experience with alternatives (I think the strongest contender is Jina AI Reader, but it still falls short massively on some URLs).
Would you guys at Kagi consider tackling this sort of issue? Considering you probably do something close for the search indexes and you probably can make it cost effective with your LLM APIs?
Something like this
import requests
base_url = 'https://kagi.com/api/v0/scrape'
params = {
"url": "https://www.bbc.com/news/articles/c4gp3ke6lzko",
"model": "google/gemini-2.0-flash-lite-001" # Probably unnecessary
}
headers = {'Authorization': f'Bot {TOKEN}'}
response = requests.get(base_url, headers=headers, params=params)
print(response.json())
and a response, sort of like this:
{
"meta": {
"id": "8b073f61-b508-42a4-52d3-0db17cd612d3",
"node": "europe-west4",
"ms": 217,
"api_balance": 3.985
},
"data": {
"content": content_from_article_in_markdown,
"tokens": 151235
}
}
