[SR-7982] Swift build forces recompilation with Makefile

[swift-ci]

Previous ID	SR-7982
Radar	rdar://problem/37440217
Original Reporter	andreasw (JIRA User)
Type	Bug

Environment

Ubuntu 18.04, Swift 4.2 dev.

Additional Detail from JIRA

Votes	0
Component/s	Compiler, Package Manager
Labels	Bug, Driver, StarterBug
Assignee	None
Priority	Medium

md5: b246f9ef9446cfcc35be81e5733a0227

Issue Description:

I have a simple Makefile for building:

build:

    swift build

After typing "make" the expected happens, the swift file is compiled.

After a second "make" the expected happens again, nothing is compiled.

Opening vim and typing ":make" I expected that nothing happens but the file is recompiled.

After a second ":make" in vim this time the expected happens again and nothing is compiled.

But if you close vim and type "make" again, the file has to be compiled again.

[aciidgh]

This happens because we add color diagnostics flag depending on whether the terminal is tty or not. We need to remove that flag from the command signature.

[swift-ci]

Comment by Andreas Wendleder (JIRA)

Would it be acceptable to remove all occurrences of Color in

swift-package-manager/Sources/Basic/TerminalController.swift?

[swift-ci]

Comment by Andreas Wendleder (JIRA)

This is still not resolved with Xcode 10.2 and Swift 5.0.

[keith]

@aciidb0mb3r any idea why swiftc rebuilds in this case? Shouldn't it be up to swiftc to no-op even if swiftpm / llbuild ask for it to build again?

[aciidgh]

@belkadan might know about that.

[belkadan]

The -color-diagnostics flag isn't a normal part of Swift's interface; it's an unstable frontend option. I guess the right fix would be to promote it to a proper option so that it can be marked DoesNotAffectIncrementalBuild and SwiftPM can stop using -Xfrontend to pass it. (I assume it's doing that because it actually captures the output before printing it, and so swiftc's tty autodetection fails?)

[keith]

I've submitted swiftlang/swift#24144 and #2099 for this

[keith]

The swiftc change is merged and the swiftpm change is ready but waiting so swiftpm still works with Xcode for the time being

[belkadan]

Looks like this is all merged through. Is it ready to be marked Resolved?

[keith]

#2103 Still needs to be merged

[swift-ci]

Comment by Andreas Wendleder (JIRA)

Doesn't work in 5.1. 🙁

[swift-ci]

Comment by Andreas Wendleder (JIRA)

Doesn't work in 5.2.

[keith]

andreasw (JIRA User) how are you validating if this is fixed or not? It looks to me at this point like it is, I'm testing by using `swift build -v` from a terminal, and then from a makefile. There is still a problem (which we can repurpose this bug for, or create a new one) where any change in environment variables seems to cause swiftpm to re-process the Package.swift files from all dependencies, which does still take some extra time, but it doesn't necessarily re-build anything in that case (unless the env var change actually has an affect on the output). That issue might also not be a trivial solve since Package.swift files can contain arbitrary logic around this.

[swift-ci]

Comment by Andreas Wendleder (JIRA)

Exactly as specified in the bug report; since executing make from within vim via ":m" is not a terminal everything is recompiled.

1. Type "make" on the command line - everything is compiled.

2. Type "make" again on command line - nothing is compiled since everything is already compiled.

3. Open vim and type ":m" which invokes make but not with terminal output - everything is compiled again.

[keith]

When you say "everything is recompiled" how are you determining this part? Here's an example of the behavior I see now with Swift 5.2:

1. Have a makefile with `swift build -v`
2. Run `make foo` from the command line twice, note the second time no commands are run
3. Run `make foo | tee /dev/null` from the command line, note Package.swift files are re-computed, and if you pass `-Xswiftc -driver-show-incremental` you see the swiftmodule file is being rebuilt, but no source files
4. Run `:make foo` in vim after step 3 and you'll see no commands are run
5. Repeat step 2 and then 4 and you'll see commands run each time.

The fact that it's running any commands at all is 2 fold:

1. Re my comment above, any change in env vars will cause Package.swift files to be recomputed, likely these change between make, shell, and vim (unless you call `swift build` with `env -i` )
2. This code:

        if BuildParameters.isTTY {
            args += ["-color-diagnostics"]
        }

Adds the `-color-diagnostics` are only in the shell without pipe case, but since my original swiftc changes, this doesn't cause a full-recompile.

So tl;dr AFAICT things aren't being recompiled here (besides the module being re-merged) but there is more noise in the logs, and arguably more things happening, than there should be