Skip to content

search: accommodate for fast typers #2031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jul 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@
/node_modules/
/test-results/
/playwright-report/
/tests/git-scm.spec.js-snapshots/*-darwin.png
/tests/git-scm.spec.js-snapshots/*-win32.png
71 changes: 71 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,20 @@ $ git sparse-checkout set layouts content static assets hugo.yml data script
$ git reset --hard
```

> [!NOTE]
> On Windows, if you cannot use [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/) for some reason, you will be unable to build the site as-is. The reason is that some URLs of the git-scm.com site contain question marks, for historical reasons. These question marks are obviously encoded as `%3F` in the URLs, but the way Hugo works, they are literal question marks in the filenames of the corresponding files. Such filenames are accommodated easily by Linux filesystems, but on Windows, filenames containing question marks are forbidden. For that reason, to build the site on Windows, the following command needs to be run (assuming a Bash, as the rest of this document):
>
> ```sh
> for file in $(find -name \*.html -exec grep -l '^url: .*?' {} \;)
> do
> git update-index --assume-unchanged "$file" &&
> sed -i '/^url: /s/?//g' "$file" ||
> break
> done
> ```
>
> This edits the affected files' `url` front-matter attributes to avoid writing those files containing question marks. Obviously, the result does not support the backwards-compatible URLs that contain URL-encoded question marks.

> [!NOTE]
> If you _already_ have a full clone and wish to accelerate development by focusing only on a small subset of the pages, you may want to run the `git sparse-checkout set [...]` command mentioned above.

Expand Down Expand Up @@ -93,12 +107,66 @@ $ HUGO_UGLYURLS=false hugo
$ npx -y pagefind --site public --serve
```

If you want to make sure that the same Pagefind version is used as when the site is deployed, use this command:

```console
$ npx -y pagefind@$(sed -n 's/^ *pagefind_version: //p' <./hugo.yml) --site public
```

Note that running Pagefind will make the process about 7 times slower, and the site will not be re-rendered and live-reloaded in the browser when you change files in `content/` (unlike with `hugo serve -w`).

## Running the test suite

Believe it or not, https://git-scm.com/ has its own test suite. It uses [Playwright](https://playwright.dev/) to perform a couple of tests that verify that the site "looks right". These tests live in `tests/` and are configured via `playwright.config.js`.

> [!NOTE]
> Building the site, letting Pagefind generate the search index, and then running the test suite can be quite time consuming. To accelerate the development cycle, it is _highly_ recommended to use a sparse checkout instead of a full clone. The minimal sparse checkout required to run the test suite can be configured like this:
>
> ```console
> $ MSYS_NO_PATHCONV=1 git config set --worktree core.sparseCheckoutCone false
> $ git config set --worktree core.sparseCheckout true
> $ git sparse-checkout set \
> /README.md \
> /assets/ \
> /content/404.html \
> /content/_index.html \
> /content/about/small-and-fast.html \
> /content/downloads/guis/ \
> /content/search/ \
> /data/ \
> /external/book/content/book/_index.html \
> /external/book/content/book/az/v2/Başlanğıc-Git-Nədir.html \
> /external/book/content/book/en/v2/Getting-Started-About-Version-Control.html \
> /external/book/content/book/en/v2/_index.html \
> /external/book/content/book/fr/v2/Démarrage-rapide-À-propos-de-la-gestion-de-version.html \
> /external/book/data/ \
> /external/docs/content/docs/git-add/fr.html \
> /external/docs/content/docs/git-clone.html \
> /external/docs/content/docs/git-commit.html \
> /external/docs/content/docs/git-commit/fr.html \
> /external/docs/content/docs/git-config.html \
> /external/docs/content/docs/git-config/fr.html \
> /external/docs/content/docs/git-remote/fr.html \
> /external/docs/content/docs/gitrevisions.html \
> /external/docs/content/docs/gitrevisions/fr.html \
> /external/docs/data/ \
> /hugo.yml \
> /layouts/ \
> /playwright.config.js \
> /script/ \
> /static/ \
> /tests/git-scm.spec.js
> ```
>
> On Windows, unless you're doing all this in WSL, do not forget to run the commands mentioned earlier to edit the `url` front-matter attributes that contain question marks!
>
> The site can then be built quickly via these commands:
>
> ```console
> $ HUGO_MEMORYLIMIT=1 time hugo &&
> npx -y pagefind@$(sed -n 's/^ *pagefind_version: //p' <./hugo.yml) --site public
> ```

To run these tests in your local setup, you need a working node.js installation. After that, you need to install Playwright:

```console
Expand All @@ -121,6 +189,9 @@ $ PLAYWRIGHT_TEST_URL='http://localhost:5000/' npx playwright test --project=fir

For more fine-grained testing, you can pass `-g <regex>` to run only the matching test cases.

> [!NOTE]
> When running the test suite on platforms other than Linux, the first run will "fail" in the `dark mode` test case. That is expected! This test case relies on previously-generated screenshots that are stored in `tests/git-scm.spec.js-snapshots/`, and for bandwidth reasons only the Linux ones are committed in the Git repository (because they are required to run the PR/CI builds). The first run will store those screenshots so that subsequent runs of this test case will succeed, though.

## Update manual pages

First, install the Ruby prerequisites:
Expand Down
18 changes: 12 additions & 6 deletions assets/js/application.js
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,8 @@ var Search = {

if(term != Search.currentSearch) {
Search.currentSearch = term;
const language = document.querySelector("html")?.getAttribute("lang");
const allResultsURL = `${baseURLPrefix}search/results?search=${term}${language && `&language=${language}`}`;
$("#search-results").html(`
<header> Search Results </header>
<table>
Expand All @@ -253,7 +255,9 @@ var Search = {
<td class="matches">
<ul>
<li>
<a class="highlight" id="show-results-label">Searching for <span id="search-term">&nbsp;</span>...</a>
<a class="highlight" id="show-results-label" href="${allResultsURL}">
Searching for <span id="search-term">&nbsp;</span>...
</a>
</li>
</ul>
</td>
Expand Down Expand Up @@ -284,16 +288,12 @@ var Search = {
$("#search-term").text(term);
this.initializeSearchIndex(async () => {
const results = await Search.pagefind.debouncedSearch(term);
if (results === null) return;
if (results.results.length === 0) {
if (results === null || results.results.length === 0) {
$("#show-results-label").text("No matching pages found.");
return;
}

const language = document.querySelector("html")?.getAttribute("lang");
$("#show-results-label")
.text("Show all results...")
.attr('href', `${baseURLPrefix}search/results?search=${term}${language && `&language=${language}`}`);

const loadButton = $("#load-more-results");
loadButton.text(`Loading ${
Expand Down Expand Up @@ -371,6 +371,12 @@ var Search = {
selectResultOption: function() {
var link = $('#search-results a')[Search.selectedIndex];
var url = $(link).attr('href');
if (!url) {
const term = $('#search-text').val();
if (!term) return;
const language = document.querySelector("html")?.getAttribute("lang");
url = `${baseURLPrefix}search/results?search=${term}${language && `&language=${language}`}`;
}
window.location.href = url;
selectedIndex = 0;
},
Expand Down
18 changes: 16 additions & 2 deletions tests/git-scm.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ test.afterEach(async ({ page }, testInfo) => {
}
})

if (process.platform === 'win32') test.setTimeout(60_000) // give it some time

test('generator is Hugo', async ({page}) => {
await page.goto(url)
await expect(page.locator('meta[name="generator"]')).toHaveAttribute('content', /^Hugo /)
Expand Down Expand Up @@ -103,9 +105,17 @@ test.describe('Linux', () => {
test('search', async ({ page }) => {
await page.goto(url)

// Search for "commit"
// Full search for "commit"
const searchBox = page.getByPlaceholder('Type / to search entire site…')
await searchBox.fill('commit')
await searchBox.press('Enter')

await expect(page).toHaveURL(/\/search\/results\?search=commit/)

await page.goto(url)

// Search for "commit"
await searchBox.fill('commit')
await searchBox.press('Shift')

// Expect the div to show up
Expand Down Expand Up @@ -250,7 +260,11 @@ test('book', async ({ page }) => {

// Navigate to a page whose URL contains a question mark
await page.goto(`${url}book/az/v2/Başlanğıc-Git-Nədir?`)
await expect(page).toHaveURL(/Ba%C5%9Flan%C4%9F%C4%B1c-Git-N%C9%99dir%3F/)
if (process.platform === 'win32' && process.env.PLAYWRIGHT_TEST_URL?.startsWith('http://localhost')) {
await expect(page).toHaveURL(/Ba%C5%9Flan%C4%9F%C4%B1c-Git-N%C9%99dir\?/)
} else {
await expect(page).toHaveURL(/Ba%C5%9Flan%C4%9F%C4%B1c-Git-N%C9%99dir%3F/)
}
await expect(page.getByRole('document')).toHaveText(/Snapshot’lar, Fərqlər Yox/)

// the repository URL now points to the Azerbaijani translation
Expand Down
Loading