This document provides a high-level overview of how FlexSearch works with Web workers. You don't have to use FlexSearch with a Web worker, but creating search indexes and performing searchs are perfect use-cases for Web worker-based non-blocking asynchronous processing. A separate thread does the search index processing while the UI stays responsive.

Web workers offload work to a background thread

Web workers are scripts that runs on a thread separate from the main UI thread. Web workers run in the background allowing the main UI to keep the UI highly responsive.

Web worker considerations:

• Because they are non-blocking, Web workers perform things like sorting or image processing without making the page unresponsive.
• Web workers live in their own scope and connect directly manipulate the DOM.
• The main script communicates with Web worker with a two-way messaging system which essentially provides a fire-and-forget, pub/sub-like messaging system.
• Web workers are implicitly asynchronous. Unlike standard asynchrony, which often pauses to keep the UI responsive, Web workers provide true parallelism running on their own thread (and potentially on a different CPU core).
• Once you master their simple messaging, Web workers are very easy to use. Despite their interesting runtime behavior, they remain plain ol' JavaScript/TypeScript.
• Don't confuse Web workers with service workers. The former is tied to a specific browser tab; if you close the browser tab the worker dies. The latter sits between your app, the browser, the network to enable services such as offline support (WWPAs), caching, and push notifications.

FlexSearch Web worker sequence diagram

The chart below shows the sequence of operations to create and search a FlexSearch index. Processes performed in the worker are done on a thread other than than the main UI thread. The index is created in the background, in memory.

Because FlexSearch is 100% client-based, the search index is recreated on each page load. While that seems iffy, it's not. FlexSearch indexed 5000 markdown documents is less that 2/3 of a second. Read more about FlexSearch indexing and performance.

FlexSearch has a new feature called persistent searches which pushes FlexSearch towards being a self-hosted Algolia-like alternative. Read more about it here.

The sequence diagram's (slightly) abstracted code

Coding note: I vastly prefer to use TypeScript in my projects. This code was developed originally for use with this Eleventy site, so I held my nose and wrote JavaScript.

To keep things very clear, flex-search-messages.js defines the Web worker messages that FlexSearch needs. These message constants are used in both the Web worker and the standard client side JS.

flex-search-messages.js

export const CREATE_INDEX = "create_index";
export const INDEX_READY = "index_ready";
export const SEARCH_INDEX = "search_index";
export const SEARCH_RESULTS = "search_results";

The example code presented here is abstracted a bit to keep things simple. This explanation focuses on the worker and its messaging. TODO-addlink. FlexSearch's Document object provides the indexing and searching logic. The worker code below isn't syntactically a class, it behaves a little like one. It is instanced and it uses self to post messages to itself.

FlexSearch's Document index object provides the ability to have granular search control (ie, search and filter by document properties). There is also an Index object that a little simpler flat file index. Read more about FlexSearch's index types here.

FlexSearch's search results need a little massaging before passing them to your UI. That detail is hand-waved alway below with ... transform searchResults. TODO-addlink

search-worker.js — this code corresponds to the sequence diagram's "worker" column.

import { Document } from "/scripts/flexsearch.bundle.module.min.js";
import * as FlexSearch from "./flex-search-messages.js";

let index = null;

self.addEventListener("message", async (event) => {
    const { type, data } = event.data;
    
    if (type === FlexSearch.CREATE_INDEX) {    // Received CREATE_INDEX message.
        index = new Document({...})

        self.postMessage({ type: FlexSearch.INDEX_READY }); // Sent INDEX_READY message.
    
    } else if (type === FlexSearch.SEARCH_INDEX) {  // Received SEARCH_INDEX message.        
        const searchResults = index.search(data.payload);

        ... transform searchResults 
        
        self.postMessage({ type: FlexSearch.SEARCH_RESULTS, 
                           data: searchResults }); // Sent SEARCH_RESULTS message.
    }
});

search.js is included in every page. It runs on the UI thread and sends and receives messages to the worker. While abridged (and with a few details omitted for now), this code, in conjunction with the search-worker.js code above, is the abstacted code that corresponds to the sequence diagram above.

search.js — this code corresponds to the sequence diagram's "client" column.

import * as FlexSearch from "./flex-search-messages.js";

const worker = new Worker("/scripts/search-worker.js", { type: "module" });

// UI input element for value to search.
const input = document.getElementById("search-input");

let indexReady = false;

worker.addEventListener("message", (event) => {
    const { type, searchResults } = event.data;

    if (type === FlexSearch.INDEX_READY) {  
        indexReady = true;
    } else if (type === FlexSearch.SEARCH_RESULTS) {  // Received SEARCH_RESULTS message.
        // Json array of searchResults. 
        renderResults(searchResults); // Iterates over results and renders to the page. 
    }
});

input.addEventListener("input", () => {
    // Fetch value to query from UI input element. 
    const query = input.value.trim();

    if (!query || !indexReady) return;

    const payload = {query};

    worker.postMessage({ type: FlexSearch.SEARCH_INDEX, // Send SEARCH_INDEX message.
                         data: { payload } });
})