feat: use GOPACKAGESDRIVER to load templ documents lazily

[prathshenoy]

Implement lazy loading and unloading of templ documents based on package dependency information provided by a custom GOPACKAGESDRIVER, ensuring correct open/close order using topological traversal.

The custom GOPACKAGESDRIVER must populate the OtherFiles field with all direct and transitive .templ file dependencies for each Package. It must also include the Templ LSP generated _templ.go file in the GoFiles and CompiledGoFiles fields to satisfy Gopls.

A custom GOPACKAGESDRIVER may not be necessary if #1128 can be resolved.

Closes #1123.

[joerdav]

Thanks folks!

I'm looking to test this out.

Are you using a bespoke GOPACKAGESDRIVER? Or the Bazel driver, I think it's bespoke because the Bazel driver won't return templ files in OtherFiles from my research.

[prathshenoy]

We have a bespoke GOPACKAGESDRIVER internally which we used for testing. As a part of #1128, we can probably create PRs to fix that in both go list and the Bazel driver.

[prathshenoy]

We wrote a short wrapper around the Bazel driver to allow you to test these changes. It essentially invokes the existing Bazel GOPACKAGESDRIVER, captures its JSON output, and injects any Templ files found in the same directories as the GoFiles into the OtherFiles field. You can use that to replace GOPACKAGESDRIVER and see how the updated response behaves.

Please let us know if you have any questions. We appreciate you taking the time to review this!

[joerdav]

Oh, that's ideal. I'll take a look at this at some point this week.

[joerdav]

All the work done so far does look good, and I'm confident that you folks have done some thorough testing!

I'm just struggling to get it working in my setup. For transparency I'm not a frequent user of bazel so am having to modify an existing app to work with bazel, and also get my setup working with it. I'll keep working at it since I'm confident it's my environment that is causing it to not work.

[prathshenoy]

Thank you for your time! Would you be willing to share the project you're working with? If you are, we can also double-check on our end.

[joerdav]

I've just worked up a modified version of an example app I use to test new features of templ with that uses bazel:
https://github.com/joerdav/shopping-list/tree/bazel

I'm using neovim and have set the following on the templ lsp:
Set GOPACKAGESDRIVER=shim
Set -no-preload

I have logged the result of opening a templ file the hovering a symbol.

{"time":"2025-04-24T15:18:39.432588+01:00","level":"INFO","msg":"client -> server: DidOpen","uri":"file:///Users/josephdavidson/src/joerdav/shopping-list/app/shopsweb/shops.templ"}
{"time":"2025-04-24T15:18:39.433585+01:00","level":"INFO","msg":"client <- server: WorkDoneProgressCreate"}
{"time":"2025-04-24T15:18:39.451363+01:00","level":"INFO","msg":"client <- server: Progress"}
{"time":"2025-04-24T15:18:39.451448+01:00","level":"INFO","msg":"client <- server: Configuration"}
{"time":"2025-04-24T15:18:39.826599+01:00","level":"INFO","msg":"client <- server: LogMessage","message":"2025/04/24 15:18:39 Created View (#1)\n\tdirectory=/Users/josephdavidson/src/joerdav/shopping-list\n\tview_type=\"GoPackagesDriver\"\n\troot_dir=\"file:///Users/josephdavidson/src/joerdav/shopping-list\"\n\tgo_version=\"go version go1.24.0 darwin/arm64\"\n\tbuild_flags=[]\n\tenv={GOOS:darwin GOARCH:arm64 GOCACHE:/Users/josephdavidson/Library/Caches/go-build GOMODCACHE:/Users/josephdavidson/.asdf/installs/golang/1.24.0/packages/pkg/mod GOPATH:/Users/josephdavidson/.asdf/installs/golang/1.24.0/packages GOFLAGS: GO111MODULE: GOTOOLCHAIN:auto GoVersion:24 GoVersionOutput:go version go1.24.0 darwin/arm64\n ExplicitGOWORK: EffectiveGOPACKAGESDRIVER:shim}\n\tenv_overlay=[]\n"}
{"time":"2025-04-24T15:18:42.808684+01:00","level":"INFO","msg":"client -> server: DidOpen end"}
{"time":"2025-04-24T15:18:42.931156+01:00","level":"INFO","msg":"client <- server: LogMessage","message":"2025/04/24 15:18:42 go/packages.Load #1\n\tview_id=\"1\"\n\tsnapshot=0\n\tdirectory=/Users/josephdavidson/src/joerdav/shopping-list\n\tquery=[./... builtin]\n\tpackages=354\n\tduration=3.104588625s\n"}
{"time":"2025-04-24T15:18:43.026204+01:00","level":"INFO","msg":"client <- server: Progress"}
{"time":"2025-04-24T15:18:43.026346+01:00","level":"INFO","msg":"client <- server: RegisterCapability"}
{"time":"2025-04-24T15:18:44.506865+01:00","level":"INFO","msg":"client <- server: LogMessage","message":"2025/04/24 15:18:44 telemetry prompt failed: malformed prompt result \"- 0 1725014763 528\"\n"}
{"time":"2025-04-24T15:19:10.240654+01:00","level":"INFO","msg":"client <- server: LogMessage","message":"2025/04/24 15:19:10 background imports cache refresh starting\n"}
{"time":"2025-04-24T15:19:12.735473+01:00","level":"INFO","msg":"client <- server: LogMessage","message":"2025/04/24 15:19:12 background refresh finished after 2.907261s\n"}
{"time":"2025-04-24T15:19:14.89047+01:00","level":"INFO","msg":"client -> server: Hover"}
{"time":"2025-04-24T15:19:14.890722+01:00","level":"WARN","msg":"completion: sourcemap not found in cache, it could be that didOpen was not called","uri":"file:///Users/josephdavidson/src/joerdav/shopping-list/app/shopsweb/shops.templ"}
{"time":"2025-04-24T15:19:14.890793+01:00","level":"INFO","msg":"client -> server: Hover end"}

Interestingly if I open a Go file in the same package it then seemst to load the templ file in properly, and the hover then works!

[joerdav]

https://github.com/a-h/templ/pull/1124/files#diff-bfa42da58e5fd0758e75fb459f76bb899c9f82f07381b8a17ab959c5195bccd1R748

I think maybe either before this point we need to convert to using the Go URI, or do it within the lazy loader?

Because I think what is happening is the shops.templ file is being sent to the GOPACKAGESDRIVER which in turn returns nothing?

Does your internal GOPACKAGESDRIVER implementation handle templ files as input, is that maybe the difference?

[prathshenoy]

You're absolutely right — thank you for catching that! I was trying to distill some of our internal code down to a minimal viable product, and I must’ve removed a bit too much in the process. I'm not sure how it got through when I was testing it out on my end, but that was clearly an oversight on my part.

Our internal GOPACKAGESDRIVER does handle Templ files correctly as input.

I've now updated the shim to rewrite file=*.templ queries to their corresponding file=*_templ.go equivalents. I believe this logic is best kept in the shim rather than in the Templ LSP itself, since Bazel doesn't always emit the *_templ.go files in the same directory as the original .templ file. For example, in our internal Bazel setup, those generated files live in the Bazel output cache.

[joerdav]

Success! That works a treat.

Thanks for the help on that one, I managed to learn a bit about bazel in the process.

@a-h Just wanted to give you an opportunity to look at this one before I merge. It's quite a niche feature so I thought not documenting it in the main website might be alright for now. I also would've suggested some LSP e2e tests, but I think the complexity there would be high, we would probably need to create a custom GOPACKAGESDRIVER.

[joerdav]

@prathshenoy @mnoah1 @JamyDev

I was wondering if it would be acceptable for us to create a "Used by" section in our README and include Ubers logo on there please? We're proud to have built something picked up by such a notable oranization like Uber.

Also am curious if you have any more detail you are allowed to share on what it's used for at Uber if you are comfortable sharing to quench my curiosity :D ?

[prathshenoy]

We really appreciate all the work y'all have put into Templ — it's been a great tool for us.

We’re currently working with our Open Source Program Office to understand what we’re able to share publicly about our use of the project. We’d be happy to provide more context if possible — just want to ensure we’re aligned internally first. I’ll follow up once we have more clarity.

@a-h In the meantime, are there any next steps you need from us for this PR?

[joerdav]

Thanks! Yeah fully understand that, thanks for looking into it for us.

The plan is to cut a release and then merge this in along with a few other PRs, so will be merged in the next few days :)

[prathshenoy]

I was wondering if it would be acceptable for us to create a "Used by" section in our README and include Ubers logo on there please? We're proud to have built something picked up by such a notable oranization like Uber.

Also am curious if you have any more detail you are allowed to share on what it's used for at Uber if you are comfortable sharing to quench my curiosity :D ?

@joerdav @a-h

Thank you for your patience! I followed up internally about your request.

Regarding the use of Templ at Uber, the company utilizes a range of technologies to support both web server and client-side functionality, including but not limited to Go and server-side rendered HTML. Templ is one of several technologies incorporated within this broader tech stack, and should not be interpreted as our central or exclusive solution for these purposes.

That said, Uber would prefer to be listed as both a notable contributor and a user. The company has made some key improvements to support large monorepos and would appreciate being included in a "Notable Contributors" or similar section if you're planning to add one.

On behalf of the team at Uber, we really appreciate the collaboration and all the great work you’ve done on this project. We’re looking forward to contributing more going forward!