x/tools/go/packages: Load might return duplicated errors, when export data requested

[mateusz834]

Reproducer:

package main

import (
	"log"

	"golang.org/x/tools/go/packages"
)

func main() {
	pkgs, err := packages.Load(&packages.Config{
		Mode: packages.LoadSyntax,
	}, "module/pkg")
	if err != nil {
		log.Fatal(err)
	}
	packages.PrintErrors(pkgs)
}

module/pkg package contains a file, that has a syntax error.

package pkg

func a() {
	var a
}

$ go run .
-: # module/pkg
pkg/invalid.go:4:7: syntax error: unexpected newline, expected type
/home/mateusz/tmp/module/pkg/invalid.go:4:7: expected type, found newline
/home/mateusz/tmp/module/pkg/invalid.go:4:6: declared and not used: a

This happens in two cases, when NeedDeps is not specified or NeedExportFile is set. In such cases the -export=true is passed to go list. That in turn causes it to return an error, which ends up being stored in Package.Errors with Kind == ListError. go/packages additionally parses the files, that go list returned, and it append them to Package.Errors, thus we end up with duplicated errors.

CC @adonovan @findleyr @matloob

[mateusz834]

This is what go/package executes:

$ GOROOT= GOPATH=/home/mateusz/go GO111MODULE= GOPROXY= PWD=/home/mateusz/tmp/module go list -e -json=Name,ImportPath,Error,Dir,GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,SwigFiles,SwigCXXFiles,SysoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap,Module -compiled=true -test=false -export=true -deps=true -find=false -pgo=off -- module/pkg
{
        "Dir": "/home/mateusz/tmp/module/pkg",
        "ImportPath": "module/pkg",
        "Name": "pkg",
        "Module": {
                "Path": "module",
                "Main": true,
                "Dir": "/home/mateusz/tmp/module",
                "GoMod": "/home/mateusz/tmp/module/go.mod",
                "GoVersion": "1.24.2"
        },
        "GoFiles": [
                "invalid.go"
        ],
        "CompiledGoFiles": [
                "invalid.go"
        ],
        "Error": {
                "ImportStack": [],
                "Pos": "",
                "Err": "# module/pkg\npkg/invalid.go:4:7: syntax error: unexpected newline, expected type\n"
        }
}

[gabyhelp]

Related Issues

• x/tools/go/packages: packages.Load: `internal error: go list gives conflicting information for package` #47065
• x/tools/go/packages: cgo error in one package causes entire Load to fail #31462 (closed)
• x/tools/go/packages: "conflicting information" error when importing internal or main package #36188 (closed)
• x/tools/go/loader: when loading >1 initial package in ImportBinary mode, some dependencies are loaded twice #9955 (closed)
• x/tools/go/packages: Load in NeedName|NeedFiles mode can produce incorrect results for certain modules #31894 (closed)
• x/tools/go/packages: preserve information about original errors #30379
• x/tools/go/packages: Unexpected package creation during export data loading #45218
• x/tools/go/packages: NeedExportFile causes error about invalid go.mod file to be dropped #67638

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[mateusz834]

I wonder whether the solution to this is just to clear errors from go list for packages that go/packages parses directly i.e. when ExportFile is not used to load the package. But then we could possibly hide some real error. I think that that such case could happen with go:embed, as this errors are only returned there (through go list). So it seems like go list should notify that this is a parse/type-check error?

[mateusz834]

So it seems like go list should notify that this is a parse/type-check error?

We might also detect that case as:

err.Pos == "" && strings.HasPrefix(err.Msg, fmt.Sprintf("# %v\n", pkg.ImportPath))

[gopherbot]

Change https://go.dev/cl/682855 mentions this issue: go/packages: remove duplicated syntax/type-check errors from go list