Lenses, Bangs, and their implementation constraints

arizvisa

This might be user-error and not a straight-up bug report, but I seem to be having a hell of a time with lenses when I attempt to immediately use them using the "bangs" feature. Perhaps I'm exaggerating with "having a hell of a time", but when I create them and attempt to test out their results I tend to get zero responses to my query _unless I tinker around with the "Region" settings in the lens itself or on the results page. After a bit, however, it seems that the lens works as I had configured it to..which gives the impression that the previous time spent troubleshooting it was for no reason. This only seems to occur with a freshly-created lens.

So, is this because an index is being created specifically for that lens and so it takes a bit of time before the index is actively queryable? If this is the case (hence being intentional), I believe this is something worth documenting or at least making users aware of the conditions that cause it.

As this would be unavoidable, in the future it might be worth considering a sort of "preview" when the user is editing the lens so that way they can troubleshoot why their lens settings do not immediately give the results they expect. (I can create a "feature" request if it's easier for you guys to track. I didn't create this report as one because it is contingent upon the cause of said delay).

If the delay isn't actually intended, let me know and I can try to produce an actual bug report.

Vlad

arizvisa There should be no delay. Can you post exact steps to reproduce the issue?

arizvisa

Okay, so I came up with an example using a "blog.kagi.com/*" lens.

To confirm that search results will be returned, I used a common query, "site:blog.kagi.com search". This returns 19 relevant results. If this is a terrible test, let me know, as I'm only differentiating between the existence and non-existence of results. The search result quality is not something I'm concerned about.

Now when creating a lens, I set the following fields:

Name of the lens: test
Describe what the lens does...: this is a test
The style of how results...: Default
Lock search results to a...: Choose Region (left untouched)
Include only these sites in results...: blog.kagi.com/*
...
Set a keyword to use as a bang...: !test

After clicking "Save", it takes me back to the "Active Lenses" page. If I then create a new tab (less than 10 seconds on the "Active Lenses" page), navigate to https://kagi.com/ and use the query !test search (https://kagi.com/search?q=search&l=13 for my test lens). It returns no results. The host that the request is being submitted to is: 34.111.242.115.

Another symptom that might help is that it doesn't seem to occur when editing an already-existing lens...It explicitly needs to be a lens number that hasn't been used yet. In my case it was 13, but while re-testing it eventually started to work and I was not able to reproduce with that same lens. I created another lens, this time using kagi.com/* with the bang set to !mytest which resulted in the lens id of 14. Then afterwards, the query !mytest kagi successfuly (or unsuccessfully) returns the "We haven't found anything" page.

Also, I can probably record an HTTP Archive if there's still the issue of reproducing. It's probably worth noting that I'm also using the Kagi Search plugin, but the tests themselves were executed by explicitly using the https://kagi.com frontend page.

Vlad

arizvisa I just created a new lens, and tried 'search' query with it and got expected results.

https://kagi.com/lenses/7ce4dlMecpsvOWic6orUNv13DjMfeEAP

Perhaps you can record a vieo of what you experience as steps to reproduce?

arizvisa

In the example I described, the Lens I created for "blog.kagi.com/*" was used to attempt finding the word "search". This was the purpose of verifying with the query "site:blog.kagi.com search" that results were returned. Do you agree that the word "search" should be found in "blog.kagi.com/*" and should return more than 0 results? (As prior mentioned, if you believe that this is a terrible example, let me know).

What fields did you set in the lens that you created? I can test it on my end to see if I can reproduce.

Vlad Perhaps you can record a vieo of what you experience as steps to reproduce?

I hope that you're familiar with an HTTP Archive, it contains a capture of _all_ of the http traffic exchanged between the browser and your webserver. There are also ways to replay this traffic to include within your unit tests.

I looked around to see if you had published a PGP key somewhere in order to encrypt it (because this archive contains personally identifiable information inside it), but was unable to find one. Nonetheless, when you're done with viewing it, I would appreciate the removal of the archive or this issue being hidden to prevent it from being shared with anybody who can browse kagifeedback.org.

arizvisa

...and here's a walkthrough of the archive just in case.

At 17:11:23.415: the https://kagi.com/settings page is requested. This is to create the lens.
At 17:12:01.566: the parameters for creating the lens are being POST'd to https://kagi.com/lenses/create
At 17:12:09.705: the https://kagi.com/search page is requested, This is to make the query "!test search" using the recently-created lens. This returns error 302, a redirect.
At 17:12:09.777: the https://kagi.com/search page is requested as https://kagi.com/search?q=search&l=13 from the redirect. This inserts the user's lens id into the query, and contains the nonce (heh) used in the XHR
At 17:12:10.476: the nonce returned from the prior request is used to make a query using a websocket. in the serialized json response, the data with id: 1 is tagged as "error".

arizvisa

..and because my wife is telling me that some people might not understand this stuff and hence I need to be patient, here's a link to something that processes an HTTP archive capture: https://toolbox.googleapps.com/apps/har_analyzer/

But again, if you believe my test is terrible. Seriously, _let me know_. That is literally why I asked about it in the example.

Vlad

arizvisa use blog.kagi.com instead of blog.kagi.com/*

Not clear why the latter does not work now (it should, we will look into it)

arizvisa

Vlad Okay. Good to know. I will re-test tomorrow without globbing just for the sake of providing more information for you guys to troubleshoot with. The usage of asterisk has the symptom of not working initially, but then maybe like a few minutes later it takes effect. This is why I had the thought that maybe it was doing some type of indexing rather than just a basic filter.

Will update this ticket tomorrow with some more tests.

arizvisa

So, after tinkering with it a bit under the assumption that the asterisk cannot be used within the path component of a url, here's some additional findings:

Multiple asterisks used by a single pattern (that match against the domain component of a url) appear to be consolidated into one asterisk. (i.e. *.*.*.comwill include results from *.com)
The asterisk seems to only produce results if it's used as a domain component (i.e. blog.*.com and not blog.k*gi.com)
The asterisk seems to only match against a certain number of characters within the component (i.e. *.co will not match t.co).
Asterisks that wrap the domain seem to query the index....differently? (i.e. *.example.* does not match www.example.net, www.example.com, or www.example.org). Although, this test could be a search-quality issue (unrelated to asterisks), because you could be indexing redirects by their target url.

In order to test, I had to go through the full process of recreating the lens as opposed to simply editing it as it appeared that the lens would take a minute to recognize my changes. This correlates with the symptom produced in the original report. In retrospect, it probably would've been better to create a proper environment and script the creation of the lens in order to simplify said testing.

I also had some issues trying to come up with domains that kagi might've indexed which also met the constraints that I was trying to test...sort of like an EICAR (used by AntiVirus), but for search. The example.org, example.net, example.com provides multiple TLDs which kagi appears to have indexed. Single-letter domains, though, (usually used by url-shorteners) don't seem to be used enough to have been indexed. Hence, I was only able to use t.co (since people tend to link that domain). I also did not bother testing homographs or puny-code encoded domains as it seems that those are presently indexed by kagi in their encoded form.

Anyways, hopefully this helps provide more clarity on the issues being encountered within lenses.

z64

In order to test, I had to go through the full process of recreating the lens as opposed to simply editing it as it appeared that the lens would take a minute to recognize my changes

Can you go into a little more detail on the exact steps? Mainly, after editing the lens, did you try new queries or the same one? If you repeated the same search you made moments ago there is a client-side cache (~10mins) that may be interfering with this kind of test. We could look into fixing that of course, but want to understand where the issue might be.

arizvisa

z64 Can you go into a little more detail on the exact steps? Mainly, after editing the lens, did you try new queries or the same one?

Yes. For reproducability the fields were set the same. The queries that I made were the ones that were listed. Namely, searching "blog.kagi.com" for the word "search".

z64 If you repeated the same search you made moments ago there is a client-side cache (_10mins) that may be interfering with this kind of test.

Okay, that's good to know. Thanks for clarifying. I'll create a separate browser profile for each individual test.

z64

@arizvisa With our rollout yesterday, we made several tweaks to how lenses work from the sources we use to how we use them. Please give them another shot and let us know what you think.

arizvisa

z64 Awesome. I'll re-test it then.

As per the discussion, I've been using blog.kagi.com/* as the included site, giving it a keyword of "!test", and querying with "search". I'll also test with blog.kagi.com as the included site, with the same fields set. Both will be created as "new lenses" (that result in a new ID that hasn't been edited before). Are there any issues that you foresee with this methodology?

Also, I'll also test with a new browser profile to ensure that client-side caching should hopefully not have an effect.

arizvisa

Awesome. Since your guys' recent update, it seems that the symptoms that I encountered are now gone and the expected results are returned immediately after creating the new lens.

Thanks for fixing it, y'all. 🙂

arizvisa

Also, just for the sake of being verbose. It works the same regardless of the "editing" a lens, or creating a new one. I also tested in my original browser profile, and it returns >0 results. This includes both "blog.kagi.com/*" and "blog.kagi.com".

I haven't specifically experimented with using the asterisk to match against the domain.. but since originally this report was about distinguishing whether I was encountering a caching symptom or a results symptom, I'll create a separate report if I encounter any specific issues with matching.

Thanks!