Skip to content

SOLR-18348: Move building suggester to async process outside of commit lifecycle. - #4742

Draft
epugh wants to merge 4 commits into
apache:branch_9xfrom
epugh:SOLR-18348
Draft

SOLR-18348: Move building suggester to async process outside of commit lifecycle.#4742
epugh wants to merge 4 commits into
apache:branch_9xfrom
epugh:SOLR-18348

Conversation

@epugh

@epugh epugh commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

https://issues.apache.org/jira/browse/SOLR-18348

Description

See the original JIRA for the problem. Basically, commits take longer when you have the suggester get built.

Solution

Messing around so far...

Tests

Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide
  • I have added a changelog entry for my change

epugh and others added 4 commits August 15, 2026 11:36
A suggester configured with buildOnCommit=true rebuilds synchronously
inside the newSearcherListener callback, which SolrCore's single-threaded
searcherExecutor runs. DirectUpdateHandler2.commit() blocks the committing
thread on that executor's future with no timeout, so a slow (e.g.
I/O-bound) rebuild blocks the commit - and the client waiting on it -
indefinitely instead of just slowing down. In production this surfaced as
client-side request timeouts requiring manual intervention, not just
degraded latency.

Add an opt-in buildOnCommitAsync suggester parameter (default false, so
existing buildOnCommit behavior and its "suggestions are fresh immediately
after commit" guarantee are unchanged unless a suggester asks for the new
behavior). When enabled, the rebuild runs on a dedicated executor instead
of inline, with a coalescing guard so a burst of commits can't queue up an
unbounded backlog of stale builds, and reader ref-counting so the searcher
being built from can't be closed out from under the async build.

Also:
- Track the index version each suggester was last built from and expose
  it (plus the current index version) in the suggest response, so a
  caller can tell whether a response reflects the current index or a
  stale one still catching up.
- Fix SolrSuggester.getSuggestions() to catch IllegalStateException from
  Lookup implementations (e.g. AnalyzingInfixSuggester) that throw rather
  than return no results when queried before any build has ever
  completed - newly reachable now that a query can land in that window
  with buildOnCommitAsync=true.
- Expose buildOnCommit status (in progress / last duration) via metrics.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ted suggest

finishStage() only ever propagated the "suggest" subsection from each
shard's response, silently dropping the per-suggester
builtFromIndexVersion/currentIndexVersion info added for the single-core
case. In a real SolrCloud deployment that meant a distributed suggest
query gave no signal at all about whether any shard answered from a
suggester still catching up (e.g. buildOnCommitAsync's rebuild still
running).

Raw index versions aren't comparable across separate cores, so rather
than merge numbers, add a single "stale" boolean per suggester (based on
builtFromIndexVersion < currentIndexVersion) at process() time, and have
finishStage() OR that flag across every shard into one top-level
suggesterStale field: true if any shard answered from a stale suggester.

Verified with a lightweight test that drives finishStage() directly
against fabricated shard responses (mirroring the existing MockShardRequest
pattern) rather than standing up a full multi-shard cluster.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@epugh

epugh commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Two scenarios, since the response shape differs — full detail on a single replica, just the yes/no flag on a distributed query.

Direct query to one replica (distrib=false, or a standalone core) — full detail, per suggester:

GET /solr/mycollection_shard1_replica_n1/suggest?suggest=true&suggest.dictionary=slowSuggester&suggest.q=elec&suggest.count=5&distrib=false&wt=json
{
  "responseHeader": { "status": 0, "QTime": 3 },
  "suggest": {
    "slowSuggester": {
      "elec": {
        "numFound": 2,
        "suggestions": [
          { "term": "electronics", "weight": 100, "payload": "" },
          { "term": "electric fan", "weight": 80, "payload": "" }
        ]
      }
    }
  },
  "suggesterIndexVersions": {
    "slowSuggester": {
      "builtFromIndexVersion": 41,
      "currentIndexVersion": 43,
      "stale": true
    }
  }
}

Normal distributed SolrCloud query (fans out to all shards, gets merged) — just the aggregate flag, no per-shard numbers:

GET /solr/mycollection/suggest?suggest=true&suggest.dictionary=slowSuggester&suggest.q=elec&suggest.count=5&wt=json
{
  "responseHeader": { "status": 0, "QTime": 12 },
  "suggest": {
    "slowSuggester": {
      "elec": {
        "numFound": 2,
        "suggestions": [
          { "term": "electronics", "weight": 100, "payload": "" },
          { "term": "electric fan", "weight": 80, "payload": "" }
        ]
      }
    }
  },
  "suggesterStale": true
}

suggesterStale: true just means at least one of the shards that answered this query was serving from a suggester that hadn't caught up yet — it doesn't say which shard or by how much. To pin that down you'd fall back to the first example (hit a specific replica directly with distrib=false) or check /admin/metrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant