proposal: cmd/go: make it easier to fork modules

[josharian]

Proposal Details

Forking a module is painful, because of intra-module import paths.

If I fork github.com/upstream/foo as github.com/josharian/foo, and I still want things to just work, I must not only change the go.mod module name, but I must also change every single import of github.com/upstream/foo/PACKAGE/PATH to be github.com/josharian/foo/PACKAGE/PATH.

This is a drag. Today, I need a two or three line change in a module. It should be trivial, a thing one just does, but instead will generate over 10x that in unrelated changes.

I have a simple proposal to ease this situation: in go.mod, as a special case, allow for replace directives to do a version-unqualified replacement to the current module.

An example will make this clearer.

-- go.mod --
module github.com/josharian/foo
-- p/p.go
package p
-- x.go --
package x

import _ "github.com/upstream/foo/p"

This does not build today, and illustrates the problem: I must change the import to github.com/josharian/foo/p for it to compile.

I propose that adding this line to go.mod should work:

replace github.com/upstream/foo => github.com/josharian/foo

This says: While compiling this module, when you see github.com/upstream/foo, get it from github.com/josharian/foo. And github.com/josharian/foo is right at hand.

If this worked, then forking a module would be exactly two changes, a module change and a replace directive.

Right now doing this yields this error message:

go.mod:5: replacement module without version must be directory path (rooted or starting with . or ..)

So I believe this should be fully backwards compatible.

[seankhliao]

I don't think this should be allowed, it makes the situation much more confusing as each package can now be referred to by multiple different names.

This can also be abused to create a mess of unimportable dependencies:
you "fork" a module named "mod" (or some other short name because you want convenience when working on the local module), some other module imports your module, but also wants to do the same thing. "mod" now refers to that some other module, rather than the original module.

[josharian]

you "fork" a module named "mod"

I would argue that this is the problem here. If it hurts, don't do it.

The goal here is to make what should be a trivial task--fork, fix a minor bug, use it. That's currently so painful I live with bugs in dependencies that I could fix in a heartbeat. Or I vendor code, adding unnecessary weight. Or I end up with non-upstreamable, unmaintainable forks.

each package can now be referred to by multiple different names

I'd also be down with a new rename directive that outlaws the original import path. (Or just add those semantics to this special case?)

[mateusz834]

Personally, I don’t mind rewriting the entire repository (including go.mod and all imports), as it’s easy to automate using the go/* packages. However, what’s really annoying is that when you merge upstream changes, you often end up with numerous merge conflicts just in the imports (I’m planning to write a tool that can automatically resolve these merge conflicts at some point). This is why i could see this really helpful.

[gabyhelp]

Related Issues

• proposal: cmd/go: `go get` automatically add `replace` in go.mod #39185
• proposal: module system: improve replace directive compatibility with forked v2 modules #70316 (closed)
• affected/package: cmd/go allow renaming a module without replacing all import paths #59766 (closed)
• proposal: cmd/go: permit replace in go.mod to act as path alias #35382 (closed)
• proposal: cmd/go: allow replacing a subdirectory within a package #30886 (closed)
• proposal: go.mod's replace in .go import section, not in go.mod #37743 (closed)
• force location of package in go.mod #27253 (closed)
• cmd/go: easier go.mod updates when packages are in the same repository #50698 (closed)
• proposal: cmd/go: support `local` directive in go.mod #51779 (closed)

Related Code Changes

• cmd/go: use replacement, not replaced, module paths in binary

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ldemailly]

If we do improve on this situation (contributing a fix and using it while it gets merged) which I hope we do, specially given the sometime glacial pace at which PRs get merged:

We should also address go installability.

[jimmyfrasche]

I think in general it would be good to just not repeat the module path in local imports. Maybe say that an import that starts with a / is local to the current module. Externally you'd need import "github.com/mod/v6/foo" but internally you'd just need import "/foo". Forking a module that used locally rooted paths like that would just need an update to go.mod. And, forking aside, it's less to write and when reading it's immediately obvious whether a package is in the same module.

[seankhliao]

relative import paths were rejected before #20883 , I believe that includes module relative paths.

[josharian]

Most of that discussion pre-dates modules. And the emoji voting appears to me to have been overwhelmingly in favor of there being a problem, in particular with forks. Ian’s comment closing it doesn’t address that issue at all.

I’m not too particular about the mechanism…just that there be one.

[ldemailly]

I guess ./ would be more correct than / as root (unless thinking of / as a path in a chroot of the module)

But mostly the how is indeed less important than the why: minimal changes to code to be able to fork, patch, submit upstream and yet use the changed code. (including go install)