Multilingual search engine

snowynest

Feature Request:
A search engine feature that translates a search query into multiple specified languages and conducts a cross-language search.

Current Challenges:
Users often need to manually translate their search queries into different languages when searching for information in multiple languages. Most search engines only return results in the language of the query.

For example, while searching for programming knowledge, it's most effective to search in English. However, for non-English speakers, this requires translating the query into English, which is time-consuming.

en: search engine
ja: 検索エンジン
ch: 搜寻器
fr: moteur de recherche
de: suchmaschine
etc...

Proposed Solution:
Implement a feature that lists search results across multiple specified languages.

Utilizing advanced translation capabilities of Large Language Models for this purpose would be beneficial, especially for sentence-level searches. Using services like DeepL could also be effective.

Support for various language combinations:

Japanese and English
French, English, and Chinese
Various other combinations...

Query Syntax:
Specify languages using a syntax such as multi:fr,en,ch or lang:fr,en,ch.

Ex 1: search engine multi:en,ja
- This would search "search engine" in both English and Japanese ("search engine" and "検索エンジン").
Ex 2: what is the best headphone ? multi:fr,en,ch
1. Search "what is the best headphone?" in French, English, and Chinese ("Quel est le meilleur casque?", "what is the best headphone?", and "最好的耳机是哪一个？").
2. Summarize the search results with Quick Answer (triggered by the ?).

Considerations:
The number of searches would increase according to the number of specified languages.

Since a separate search is required for each language, it is similar to the user performing multiple searches themselves.

Vlad

How would this look like?

anonymous_coward

Vlad

So, I guess there are broadly four components to this feature: (1) how multilingual search results are presented to the user, (2) how the user opts into multilingual searches, (3) how the search engine interprets search queries intended to yield multilingual results, and (4) how websites are indexed.

and 2.

I speak English and French, so I'd be perfectly happy getting results in either of those languages in my day-to-day searches, even without considering modern automated translation. This would probably end up most similar to the lenses interface?

My browser can translate 16 languages, presumably more to come, but Kagi can't depend on every browser having that functionality. Power-users like me could create a lens that asks for search results in any of the 16 languages that my browser can auto-translate, but for most users, even if Kagi served an auto-translated search result, they would click through to a website in a foreign language. Orion could tell Kagi which languages it could translate, but until Google, Apple, and Mozilla agree to a standard for communicating that, that feature is going to probably have to be Orion-exclusive, except for advanced users who can configure it themselves on both ends.

and 4.

The languages specified in the lens would probably cause the user's query to be translated into the target languages. So an "English+French" lens with the search "bicycle repair shops montreal" would emit that search as well as "atelier de réparation de vélos montréal". One search becoming N-1 translations, N searches, and a merge might be intractably expensive? Then again, I just tried selecting "International" search, then concatenating the English and French search query together, and I got a reasonable mix of English and French results, so maybe it wouldn't end up making a big difference until people like me started being interested in seeing results from the top 50 languages. Maybe re-index the whole internet, keyed off BERT encodings 😛?

Anyway, I found this post because I saw that Firefox integrated local machine translation into the browser, and I thought it would be good if my search results could be made to incorporate results from any language I could "read". There are ~1.8B English+French speakers in the world. There are ~1.5B speakers of all the other languages Firefox can translate. Mandarin, Hindi, Urdu, Japanese, Arabic, Indonesian, Korean, and Vietnamese gets you most of the rest of humanity. We aren't far from getting the Tower of Babel back together.

I find I most often go to the trouble of running translated versions of my search queries from language-specific search engine front-ends, and then running web pages through a translator, because I'm interested in how things are being discussed around the world (Egyptian discussions of the Israel-Palestine conflict; Indian discussions of the Russian invasion of Ukraine; South Korean, Japanese, and Taiwanese discussions on the American CHIPS Act). I would do it a lot more if it were easy.

I would pay extra for this feature btw.

snowynest

I apologize for the delayed response! (I've been dealing with various real-life issues...)

And thank you for the use cases. You're right, if users often use specific language combinations, it would be convenient to call them up with Lenses.

I've outlined some information that I expect to see on the search results:

Display the "search query after translation into specified languages"
Maybe below the search box would be a good place? This helps users in further searches. Moreover, clarifying "what was actually searched" contributes to search transparency and enhances user confidence.
Issue: When specifying languages the user can't read, they can't understand the results
In this case, it becomes necessary to translate the search results upon display. Having at least the titles translated would be great! Also, it's crucial to display "which language the search result is in" using language codes next to each site's title.
If utilizing Quick Answer, I want to see the translated queries in each language (as mentioned in 1.). It would be helpful if the summary is paragraph-separated for each language's results. Also, I want the display language of the Quick Answer to match the language the user has specified for the UI. https://kagifeedback.org/d/2927-ai-output-language

Another concern is how to handle verbatim searches.
For example, should "search engine" be translated to "検索エンジン", or should it remain as "search engine"? Both cases are plausible, but I'm not sure how to design a UI for users to choose...