This document provides a high-level overview of how FlexSearch works with Web workers. You don't have to use FlexSearch with a Web worker, but creating search indexes and performing searchs are perfect use-cases for Web worker-based non-blocking asynchronous processing. A separate thread does the search index processing while the UI stays responsive.

What is FlexSearch

FlexSearch is a superb, browser-based search engine. It supports a non-blocking asynchronous processing model using Web workers and works great with Svelte/Vue/React and static site builders like Eleventy and Gatsby. It provides the search engine for the blog you're reading.

The notion of an effective browser-based search engine is a little eye-brow raising. Your truly have to see FlexSearch in action to believe what it is—it is highly performant. But don't take my word for it, get this [Git repo](https://github.com/rogerpence/test-flexsearch-performance) and run the example on your box. On my PC, FlexSearch indexed 5000 documents in about 700ms. For many sites, FlexSearch is a worthy alternative to Algolia, ElasticSearch, and TypeSense.

This document describes the current search implementation in src/scripts/search-worker.js.

Web workers offload work to a background thread

Web workers are scripts that runs on a thread separate from the main UI thread. Web workers run in the background allowing the main UI to keep the UI highly responsive.

Web worker considerations:

• Because they are non-blocking, Web workers perform things like sorting or image processing without making the page unresponsive.

• Web workers live in their own scope and connect directly manipulate the DOM.

• The main script communicates with Web worker with a two-way messaging system which essentially provides a fire-and-forget, pub/sub-like messaging system.

• Web workers are implicitly asynchronous. Unlike standard asynchrony, which often pauses to keep the UI responsive, Web workers provide true parallelism running on their own thread (and potentially on a different CPU core).

• Once you master their simple messaging, Web workers are very easy to use. Despite their interesting runtime behavior, they remain plain ol' JavaScript/TypeScript.

• Don't confuse Web workers with service workers. The former is tied to a specific tab; if you close the browser tab the worker dies. The latter sits between your app, the browser, the network to enable services such as offline support (WWPAs), caching, and push notifications.

Non-blocking: Because they run on a separate thread, heavy computations (like image processing or large data sorting) can happen without freezing the user interface or making the page unresponsive. Isolated Environment: Workers do not have access to the DOM (the HTML/CSS). They live in their own scope and cannot directly manipulate the page. Communication: They communicate with the main script using a messaging system (postMessage to send data and an onmessage event listener to receive it)

The problem this revision fixes

The FlexSearch docs discuss a tag property that names a property that can later be used to constrain the search to that property.

As I understand it, the tag property in the FlexSearch index defintion below identifies the folder property as search constraint.

index = new Document({
    document: {
        id: "id",
        store: ["id", "title", "url", "tags", "folder", "description"],
        //store: true,
        index: ["title", "url", "content", "tags", "description"],
        tag: "folder",
    },
    tokenize: "forward",
});

At runtime, if the search options argument includes a tag property, that property should apply the constraint to the search results--searching only index objects where the folder property is post.

const rawSearchResult = index.search(searchTerm, {
    tag: { folder: 'post' },
    enrich: true,
    limit: 20,
});

If it worked, this feature makes for a very clean and direct way to filter search results (within the search itself). Alas, I'm either doing something or this feature doesn't work, because it worked only intermittently for me.

The revised search function

The revised search function is at the end of this article.

This revision's swindle is that it searches unconditionally across all search objects and then filters the search results to include only those optional values specified.

Beware that the FlexSearch search limit should probably be a little higher than it needs to be if optional search values are used. Consider a search limit of 20. Whatever search results removed by the after-search filtering come of out that 20, so there may be way less than 20, even zero, results to show.

The other downside is that this version is just butt ugly. Its use of flatMap, reduce, and filter are challenging to quickly grasp.

The revised function supports three behaviors:

1. Search across indexed content with FlexSearch Document.
2. Optionally constrain matches to one indexed field (title, content, or tags).
3. Optionally constrain results to a folder (post or link).

It also deduplicates results by document id and adds metadata (sourceFields) indicating which indexed field(s) matched.

Inputs

search(payload) expects:

• payload.query: search text.
• payload.property_constraint: one of all, title, content, tags.
• payload.folder_constraint: one of all, post, posts, link, links.

Helper Functions

normalizeFolderConstraint(folder)

Normalizes folder values from UI controls:

• all (or empty) -> null (no folder constraint)
• posts -> post
• links -> link
• otherwise lowercased value

groupByDocId(results)

Takes FlexSearch enriched field-grouped results and returns a map of:

• key: doc.id
• value: array of field names that matched (title, content, tags, etc.)

This is later written to each result as sourceFields.

Search Flow

1. Normalize constraints

The function extracts and normalizes constraints:

• folder -> normalizedFolderConstraint
• property -> normalizedPropertyConstraint

2. Run one FlexSearch query

It runs:

index.search(searchTerm, { enrich: true, limit: 20 })

This returns grouped results by indexed field.

3. Apply property constraint (field scope)

If property_constraint is one of allowed fields (title, content, tags), it keeps only the matching field group from the FlexSearch response.

If property is all (or invalid), it keeps all field groups.

4. Build source-field map

groupByDocId(scopedSearchResult) builds a lookup of matching fields per doc id.

5. Flatten + dedupe

The function flattens all matches into one list and deduplicates by id.

For each unique doc:

• adds sourceFields (from the lookup)
• pushes the enriched doc into final results

6. Apply folder constraint

Finally, it filters deduped docs by folder when requested:

• no folder constraint -> all docs pass
• folder constraint (post or link) -> only docs with matching doc.folder

Why Folder Filtering Is Post-Search

The current implementation intentionally applies folder filtering in JavaScript after enrichment/dedupe.

Reason: in this runtime, FlexSearch tag filtering did not return expected matches for this dataset/configuration. The post-filter path is deterministic and currently correct.

Output Shape

Returns an array of enriched doc objects from store:

• id
• title
• url
• tags
• folder
• sourceFields (added by this function)

Example Behavior (query = "json")

With current data:

• folder all + property all -> 6 hits
• folder post + property all -> 5 hits
• folder link + property all -> 1 hit

And property constraints narrow within those folder results (for example, tags only).

The revised search function

function search(payload) {
    const searchTerm = payload.query;
    const constrainedProperty = payload.property_constraint;
    const constrainedFolder = payload.folder_constraint;
    const normalizedFolderConstraint =
        normalizeFolderConstraint(constrainedFolder);
    const normalizedPropertyConstraint = constrainedProperty
        ?.trim()
        .toLowerCase();

    const searchArgs = {
        enrich: true,
        limit: 20,
    };

    const rawSearchResult = index.search(searchTerm, searchArgs);
    const scopedSearchResult =
        normalizedPropertyConstraint &&
        normalizedPropertyConstraint !== "all" &&
        availableSearchConstraintFields.includes(normalizedPropertyConstraint)
            ? rawSearchResult.filter(
                  ({ field }) => field === normalizedPropertyConstraint,
              )
            : rawSearchResult;

    const uniqueIdCombos = groupByDocId(scopedSearchResult);
    console.log(JSON.stringify(uniqueIdCombos, null, 4));

    const searchResult = scopedSearchResult
        .flatMap((item) => item.result)
        .reduce((acc, current) => {
            if (!acc.find((item) => item.id === current.id)) {
                current.doc["sourceFields"] = uniqueIdCombos[current.id];
                acc.push(current.doc);
            }
            return acc;
        }, [])
        .filter((doc) => {
            if (!normalizedFolderConstraint) return true;
            return doc.folder === normalizedFolderConstraint;
        });

    console.log("searchResult", JSON.stringify(searchResult, null, 4));

    return searchResult;
}

Deciphering this code:

const searchResult = scopedSearchResult
    .flatMap((item) => item.result)
    .reduce((acc, current) => {
        if (!acc.find((item) => item.id === current.id)) {
            current.doc["sourceFields"] = uniqueIdCombos[current.id];
            acc.push(current.doc);
        }
        return acc;
    }, [])
    .filter((doc) => {
        if (!normalizedFolderConstraint) return true;
        return doc.folder === normalizedFolderConstraint;
    });

flatmap

Applied against the search objects, this code

const test = scopedSearchResult.flatMap((item) => item.result);

produces this:

[
    {
        "id": 1000,
        "doc": {
            "id": 1000,
            "title": "https://github.com/wesbos/JSON-Alexander",
            "url": "https://github.com/wesbos/JSON-Alexander",
            "tags": "links,json,web-dev",
            "folder": "link",
            "description": "A really good JSON viewer browser Extension."
        }
    },
    ...
]

flatmap and reduce

Applied against the search objects, this code

const test = scopedSearchResult
    .flatMap((item) => item.result)
    .reduce((acc, current) => {
        if (!acc.find((item) => item.id === current.id)) {
            current.doc["sourceFields"] = uniqueIdCombos[current.id];
            acc.push(current.doc);
        }
        return acc;
    }, []);

produces this:

[
    {
        "id": 1000,
        "title": "https://github.com/wesbos/JSON-Alexander",
        "url": "https://github.com/wesbos/JSON-Alexander",
        "tags": "links,json,web-dev",
        "folder": "link",
        "description": "A really good JSON viewer browser extension.",
        "sourceFields": [
            "title",
            "url",
            "content",
            "tags",
            "description"
        ]
    },
    ...
]

See note on sourceFields below.

flatMap, reduce, and filter

When normalizedFolderConstraint contains a folder name, the filter method limits the results to only search objects where the folder property is equal to the normalizedFolderConstraint variable.

const searchResult = scopedSearchResult
    .flatMap((item) => item.result)
    .reduce((acc, current) => {
        if (!acc.find((item) => item.id === current.id)) {
            current.doc["sourceFields"] = uniqueIdCombos[current.id];
            acc.push(current.doc);
        }
        return acc;
    }, [])
    .filter((doc) => {
        if (!normalizedFolderConstraint) return true;
        return doc.folder === normalizedFolderConstraint;
    });

Identifying where the search term was found

The search results includes a sourceFields property that shows the properties of the search object in which the search term was found. The original intent of this was be to be able to show the user where the search term was found. I'm not sure that's of use, but it remains for now.

This code:

function groupByDocId(results) {
    return results.reduce((acc, { field, result }) => {
        for (const { doc } of result) {
            (acc[doc.id] ??= []).push(field);
        }
        return acc;
    }, {});
}

...

const uniqueIdCombos = groupByDocId(scopedSearchResult);

produces this object which provides, by search object id, the properties of the search object in which the search term was found.

{
    "2": [
        "content"
    ],
    "4": [
        "content"
    ],
    "5": [
        "content"
    ],
    "10": [
        "content"
    ],
    "12": [
        "tags"
    ],
    "16": [
        "content"
    ],
    "1000": [
        "title",
        "url",
        "content",
        "tags",
        "description"
    ]
}

The full original worker

This original doesn't include the code to search specific properties, but it does include the abiility to constrain the search by the property specified with the tag option (which I couldn't make work!)

import { Document } from "/scripts/flexsearch.bundle.module.min.js";

const availableSearchConstraintFields = ["title", "content", "tags"];

let index = null;

self.addEventListener("message", async (event) => {
    const { type, data } = event.data;

    if (type === "init") {
        /**
         * The 'store' property defines the search result output properties.
         * The 'index' property defines the properties that are indexed.
         */
        index = new Document({
            document: {
                id: "id",
                store: ["id", "title", "url", "tags", "folder"],
                //store: true,
                index: ["title", "url", "content", "tags"],
                tag: "folder",
            },
            tokenize: "forward",
        });

        /**
         * Search result properties are fetched from the index object.
         */
        for (const post of data.posts) {
            index.add({
                id: post.id,
                title: post.title,
                content: post.content,
                tags: (post.tags || []).join(","),
                url: post.url,
                folder: post.folder,
            });
        }

        self.postMessage({ type: "ready" });
    } else if (type === "search") {
        const results = search(data.payload);
        self.postMessage({ type: "results", data: results });
    }
});

function groupByDocId(results) {
    return results.reduce((acc, { field, result }) => {
        for (const { doc } of result) {
            (acc[doc.id] ??= []).push(field);
        }
        return acc;
    }, {});
}

function search(payload) {
    const searchTerm = payload.query;
    const constrainedProperty = payload.property_constraint;
    const constrainedFolder = payload.folder_constraint;

    const searchArgs = {
        enrich: true,
        limit: 20,
    };

    if (constrainedFolder !== "all") {
        searchArgs["tag"] = { folder: constrainedFolder };
    }

    const rawSearchResult = index.search(searchTerm, searchArgs);

    const uniqueIdCombos = groupByDocId(rawSearchResult);
    console.log(JSON.stringify(uniqueIdCombos, null, 4));

    const searchResult = rawSearchResult
        .flatMap((item) => item.result)
        .reduce((acc, current) => {
            if (!acc.find((item) => item.id === current.id)) {
                current.doc["sourceFields"] = uniqueIdCombos[current.id];
                acc.push(current.doc);
            }
            return acc;
        }, []);

    return searchResult;
}