Build your own Hugo website search engine

5 septembre 2018 · 4 min read

The goal is to build a static site search :

  • without installing any JS module on your computer
  • without indexing stop words, i.e. words like “and, or, with, the…”
  • indexing words once by page (no duplicates)
  • your next created pages will be added automatically, without any additional work.

How does it work ?

  • HUGO will create a JSON file at the root of the site that will be used as a database (title, url, content).
  • When a user visits one page, this file is downloaded to its browser.
  • When words are written to the text box, some Javascript code will search for the words in the variables content and get the correponding URLs.
hugo-search-engine

Install

Creating the JSON file

Type the following command :

hugo new search.md

The file content/search.md is created. Modify it as this :

---
title: "Search"
date: 2018-09-04T15:32:54+02:00
layout: "search"
outputs: ["json"]
noindex: true
---

The new native front matter variables used here are layout to tell Hugo which template to use, and outputs to generate a JSON file. noindex is a user defined variable that we use to exclude this JSON file from beeing added to the search engine.

Create the template layouts/_default/search.json :

{{ $stopwords := (slice "again" "and" "in" "is" "no" "or" "this" "well" "yes") }}
{ "results": [
{{ $pages := (where .Site.RegularPages ".Params.noindex" "ne" true) }}
{{ $lastindex := (sub (len $pages) 1) }}
{{ range $index, $page := $pages }}
    {{ $scratch := newScratch }}
    {{- $content := (.Content | plainify | lower) -}}
    {{- $content := (replaceRE "\"" "" $content) -}}
    {{- $content := (replaceRE "“" "" $content) -}}
    {{- $content := (replaceRE "”" "" $content) -}}
    {{- $content := (replaceRE "\n+" " " $content) -}}
    {{- $content := (replaceRE "\r+" " " $content) -}}
    {{- $content := (replaceRE "\t+" " " $content) -}}
    {{- $content := (replaceRE " +" " " $content) }}
    {{- $content := (trim $content " ") }}
    {{- $words := (split $content " ") -}}
    {
    "url": "{{ .Permalink }}",
    "title": "{{ .Title }}",
    "content": "{{- range $words -}}
    {{- $word := (.) -}}
    {{- if and (not (in ($scratch.Get "seenwords") $word)) (ne (substr $word 0 1) $word) (not (in $stopwords $word)) -}}
        {{- print $word " " -}}
    {{- end -}}
    {{ $scratch.Add "seenwords" (slice $word) }}
    {{- end -}}"
    }
    {{- $scratch.Delete "seenwords" -}}
    {{- if ne $lastindex $index }},{{ end }}
{{ end }}
]}

This template gets all pages from your HUGO site (RegularPages) which are not to exclude from the search (variable noindex seen before). The content is splitted to words. A word is added if not already done and if it is not listed in the stop words array. So if you want to exclude useless words from indexation, add them to the list at the top of the page.

If you type the hugo command, the file http://localhost:1313/search/index.json is created :

{ "results": [
    {
    "url": "http://example.com/article3/",
    "title": "Article3",
    "content": "word3a word3b word3c"
    },
    {
    "url": "http://example.com/article2/",
    "title": "Article2",
    "content": "word2a word2b word2c"
    },
    {
    "url": "http://example.com/article1/",
    "title": "Article1",
    "content": "word1a word1b word1c"
    }
]}

Indexing pages

By default, all regular pages are added automatically to the search index. If you don’t want to add a page, just insert this in the front matter :

noindex: true

Default HTML template

In the main template where you want to add a search engine, add this CSS style for the input box, between tags <head></head> :

<style>
    .search {
        position: relative;
        display: inline-block;
        }
    .search-results {
        display: none;
        background-color: #fff;
        position: absolute;
        min-width: 100px;
        box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
        z-index: 1;
        }
    .search-results a {
        color: black;
        padding: 12px 16px;
        text-decoration: none;
        display: block;
        }
    .search-results a:hover {background-color: #ddd;}
</style>

Add the search box where you need it, between tags <body></body> :

For the search engine, we will use the libraries VueJS and axios via CDN. The first one is a usefull and famous JS framework and the second one will help to download the JSON index to the browser.

Just before the </body> tag, insert the following code. Some explanations are written in comments :

Et voilà !

References

Find other cases at https://gohugo.io/tools/search/.

#hugo #gohugo #search #engine #stopwords #vuejs #axios
SHARE