HTML source code full of whitespace

tuesday

The HTML that Kagi provides is (currently) full of unnecessary newlines and spaces. By my count, on the kagi.com home page, the code includes nearly 9000 extra line breaks and 32000 extra whitespace characters.

This issue is present on Kagi Search and a few other pages such as Summarizer.

While not critical, unnecessary space should be removed, as it looks odd and it makes the HTML file about 10% larger than it needs to be.

tuesday

To be clear, the numbers I quoted are based on the regex \n\s*\n, which only looks at instances where there are more than two newlines in a row. I'm not suggesting that the source code should be "minimized" to make it small but difficult to read.

jacobwinters

It makes me happy that users are using view source on our pages :D

I've noticed that too and it's been making me sad. We've been migrating to a new templating engine that isn't as good at managing whitespace as our old one. It has a few switches that we could turn on, but every time I've experimented with them it's created bugs in the app. Thus far we've tabled the project as not worth the effort and hoped that gzip would make up for it.

The indentation in rendered templates is weird because a page is made by gluing together a dozen source files that each start all the way to the left, and when you enter an {% if %} or a {% for %} you get an extra layer of indentation in the output even though you can't see the control flow code, only the whitespace around it. Getting rid of all the indentation is the best we can easily do. If we want DOM trees that look nice, I'm afraid we'll have to use element inspector.

I'm going to try having our template loader run some regex on our source files, which is a terrible hack, but at least contained to a couple of lines:

.gsub(/^[ \t]+/m, "")                         # don't send indentation over the network
.gsub(/^({%(?:[^%\n]|%(?!}))*%})\n+/m, "\\1") # eat \n+ after a line that's all one block

Page	Before, raw kB	zipped	After, raw kB	zipped
Empty SERP	153	36.0	134	34.6
Home page (more parts migrated)	353	43.9	197	39.4

Modest savings on the wire for SERP, but nonzero. On net I think the output is more readable. Doesn't appear to have caused any bugs.

jacobwinters

(I'm assuming that whitespace is the only thing that changed on those pages with the last deploy. I think that's approximately correct. Not giving numbers for a SERP with results because there have been more changes there.)