Use POST method to send search queries

SergioD

Instead of send the search query using GET HTTP method, it's preferable to use POST.
This adds more privacy to searches because queries travel encrypted (because of HTTPS) and there won't be any history about your searches in the browser.

This is a feature that Startpage has.
And some browsers, like Vivaldi, support searches using POST from the address or the search bars.
When a user enters kagi.com and performs a search, transparently to the user, the query should be sent using POST.
Or, if the browser supports the feature, the user could configure it to send queries using POST.

Vlad

SergioD We had this feature but removed as it gave false sense of privacy. Most people use browser address bar to search (99%), and this always uses GET, even if you turn on POST option.

If a browser supports POST natively then again no such feature should be needed on our end.

feedbackhax

The queries are encrypted eithet way, but yes POST has various upsides (hence why you rarely see GET parameters in banking apps)

feedbackhax

Vlad this always uses GET, even if you turn on POST option.

That's not true, just tested it with a searxng instance on firefox.
This has to be specified in the opensearch.xml, then browsers use POST.

Vlad

feedbackhax Can you provide more details? Does it work the same way with DDG for example when you enable POST in its settings?

Fernandez

Vlad For example, I've written an OpenSearch XML file for StartPage, shown below:

<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/"
                       xmlns:moz="http://www.mozilla.org/2006/browser/search/">
  <!-- Created on Sat, 30 Dec 2023 06:33:15 GMT -->
  <ShortName>StartPage POST</ShortName>
  <Description>StartPage Search with POST</Description>
  <Url type="text/html" method="post" template="https://www.startpage.com/sp/search">
    <Param name="query" value="{searchTerms}"/>
    <Param name="prfe" value="your-preference-string"/>
    <Param name="abp" value="-1"/>
    <Param name="t" value=""/>
    <Param name="lui" value="english"/>
    <Param name="sc" value="9r9AYWaSozCU00"/>
    <Param name="cat" value="web"/>
  </Url>
  <!-- <Image width="16" height="16">GENERATEDLATER</Image> -->
  <Developer>your-name</Developer>
  <InputEncoding>UTF-8</InputEncoding>
  <moz:SearchForm>https://www.startpage.com/sp/search</moz:SearchForm>
  <!-- <Url type="application/opensearchdescription+xml" rel="self" template="GENERATEDLATER"/> -->
</OpenSearchDescription>

Import this file to Mycroft Project and then add the search engine to Firefox.
Actually, I found one you can directly add to your Firefox.

After that, even though you search through address bar, Firefox will use POST to query StartPage.

feedbackhax

It surely won't work if you've added it as search engine using opensearch.xml and THEN change your settings.

Doesn't look like they do opensearch.xml at all.

You just have to change opensearch.xml depending on the users setting and tell them to remove and re-add the search engine if they toggle it.

SearXNG is open source and does this well.

EDIT: you already modify the xml per user to include the token. I think adding a flag there and have that appended to the opensearch.xml link depending on a users setting is the way to go.

SergioD

I tested this in Vivaldi, having Kagi added as a search engine. I configured it to use POST, but I get a 502 error.

KagiForMe

Now seeing that this is (or seems) possible to do on your end with a privacy improvement, I would love this feature to be implemented. Preferably opt-in (like DDG) manages it.

Vlad

@z64 do we still accept POST for search?

z64

Vlad For the initial search, yes it should work. For follow-up searches after the page has loaded, I believe our JS will insert a history element regardless.