|
| 1 | +<!--{ |
| 2 | + "title": "About the go command" |
| 3 | +}--> |
| 4 | + |
| 5 | +<p>The Go distribution includes a command, named |
| 6 | +"<code><a href="/cmd/go/">go</a></code>", that |
| 7 | +automates the downloading, building, installation, and testing of Go packages |
| 8 | +and commands. This document talks about why we wrote a new command, what it |
| 9 | +is, what it's not, and how to use it.</p> |
| 10 | + |
| 11 | +<h2>Motivation</h2> |
| 12 | + |
| 13 | +<p>You might have seen early Go talks in which Rob Pike jokes that the idea |
| 14 | +for Go arose while waiting for a large Google server to compile. That |
| 15 | +really was the motivation for Go: to build a language that worked well |
| 16 | +for building the large software that Google writes and runs. It was |
| 17 | +clear from the start that such a language must provide a way to |
| 18 | +express dependencies between code libraries clearly, hence the package |
| 19 | +grouping and the explicit import blocks. It was also clear from the |
| 20 | +start that you might want arbitrary syntax for describing the code |
| 21 | +being imported; this is why import paths are string literals.</p> |
| 22 | + |
| 23 | +<p>An explicit goal for Go from the beginning was to be able to build Go |
| 24 | +code using only the information found in the source itself, not |
| 25 | +needing to write a makefile or one of the many modern replacements for |
| 26 | +makefiles. If Go needed a configuration file to explain how to build |
| 27 | +your program, then Go would have failed.</p> |
| 28 | + |
| 29 | +<p>At first, there was no Go compiler, and the initial development |
| 30 | +focused on building one and then building libraries for it. For |
| 31 | +expedience, we postponed the automation of building Go code by using |
| 32 | +make and writing makefiles. When compiling a single package involved |
| 33 | +multiple invocations of the Go compiler, we even used a program to |
| 34 | +write the makefiles for us. You can find it if you dig through the |
| 35 | +repository history.</p> |
| 36 | + |
| 37 | +<p>The purpose of the new go command is our return to this ideal, that Go |
| 38 | +programs should compile without configuration or additional effort on |
| 39 | +the part of the developer beyond writing the necessary import |
| 40 | +statements.</p> |
| 41 | + |
| 42 | +<h2>Configuration versus convention</h2> |
| 43 | + |
| 44 | +<p>The way to achieve the simplicity of a configuration-free system is to |
| 45 | +establish conventions. The system works only to the extent that those conventions |
| 46 | +are followed. When we first launched Go, many people published packages that |
| 47 | +had to be installed in certain places, under certain names, using certain build |
| 48 | +tools, in order to be used. That's understandable: that's the way it works in |
| 49 | +most other languages. Over the last few years we consistently reminded people |
| 50 | +about the <code>goinstall</code> command |
| 51 | +(now replaced by <a href="/cmd/go/#hdr-Download_and_install_packages_and_dependencies"><code>go get</code></a>) |
| 52 | +and its conventions: first, that the import path is derived in a known way from |
| 53 | +the URL of the source code; second, that the place to store the sources in |
| 54 | +the local file system is derived in a known way from the import path; third, |
| 55 | +that each directory in a source tree corresponds to a single package; and |
| 56 | +fourth, that the package is built using only information in the source code. |
| 57 | +Today, the vast majority of packages follow these conventions. |
| 58 | +The Go ecosystem is simpler and more powerful as a result.</p> |
| 59 | + |
| 60 | +<p>We received many requests to allow a makefile in a package directory to |
| 61 | +provide just a little extra configuration beyond what's in the source code. |
| 62 | +But that would have introduced new rules. Because we did not accede to such |
| 63 | +requests, we were able to write the go command and eliminate our use of make |
| 64 | +or any other build system.</p> |
| 65 | + |
| 66 | +<p>It is important to understand that the go command is not a general |
| 67 | +build tool. It cannot be configured and it does not attempt to build |
| 68 | +anything but Go packages. These are important simplifying |
| 69 | +assumptions: they simplify not only the implementation but also, more |
| 70 | +important, the use of the tool itself.</p> |
| 71 | + |
| 72 | +<h2>Go's conventions</h2> |
| 73 | + |
| 74 | +<p>The <code>go</code> command requires that code adheres to a few key, |
| 75 | +well-established conventions.</p> |
| 76 | + |
| 77 | +<p>First, the import path is derived in a known way from the URL of the |
| 78 | +source code. For Bitbucket, GitHub, Google Code, and Launchpad, the |
| 79 | +root directory of the repository is identified by the repository's |
| 80 | +main URL, without the <code>http://</code> prefix. Subdirectories are named by |
| 81 | +adding to that path. |
| 82 | +For example, the Go example programs are obtained by running</p> |
| 83 | + |
| 84 | +<pre> |
| 85 | +git clone https://github.com/golang/example |
| 86 | +</pre> |
| 87 | + |
| 88 | +<p>and thus the import path for the root directory of that repository is |
| 89 | +"<code>github.com/golang/example</code>". |
| 90 | +The <a href="https://godoc.org/github.com/golang/example/stringutil">stringutil</a> |
| 91 | +package is stored in a subdirectory, so its import path is |
| 92 | +"<code>github.com/golang/example/stringutil</code>".</p> |
| 93 | + |
| 94 | +<p>These paths are on the long side, but in exchange we get an |
| 95 | +automatically managed name space for import paths and the ability for |
| 96 | +a tool like the go command to look at an unfamiliar import path and |
| 97 | +deduce where to obtain the source code.</p> |
| 98 | + |
| 99 | +<p>Second, the place to store sources in the local file system is derived |
| 100 | +in a known way from the import path, specifically |
| 101 | +<code>$GOPATH/src/<import-path></code>. |
| 102 | +If unset, <code>$GOPATH</code> defaults to a subdirectory |
| 103 | +named <code>go</code> in the user's home directory. |
| 104 | +If <code>$GOPATH</code> is set to a list of paths, the go command tries |
| 105 | +<code><dir>/src/<import-path></code> for each of the directories in |
| 106 | +that list. |
| 107 | +</p> |
| 108 | + |
| 109 | +<p>Each of those trees contains, by convention, a top-level directory named |
| 110 | +"<code>bin</code>", for holding compiled executables, and a top-level directory |
| 111 | +named "<code>pkg</code>", for holding compiled packages that can be imported, |
| 112 | +and the "<code>src</code>" directory, for holding package source files. |
| 113 | +Imposing this structure lets us keep each of these directory trees |
| 114 | +self-contained: the compiled form and the sources are always near each |
| 115 | +other.</p> |
| 116 | + |
| 117 | +<p>These naming conventions also let us work in the reverse direction, |
| 118 | +from a directory name to its import path. This mapping is important |
| 119 | +for many of the go command's subcommands, as we'll see below.</p> |
| 120 | + |
| 121 | +<p>Third, each directory in a source tree corresponds to a single |
| 122 | +package. By restricting a directory to a single package, we don't have |
| 123 | +to create hybrid import paths that specify first the directory and |
| 124 | +then the package within that directory. Also, most file management |
| 125 | +tools and UIs work on directories as fundamental units. Tying the |
| 126 | +fundamental Go unit—the package—to file system structure means |
| 127 | +that file system tools become Go package tools. Copying, moving, or |
| 128 | +deleting a package corresponds to copying, moving, or deleting a |
| 129 | +directory.</p> |
| 130 | + |
| 131 | +<p>Fourth, each package is built using only the information present in |
| 132 | +the source files. This makes it much more likely that the tool will |
| 133 | +be able to adapt to changing build environments and conditions. For |
| 134 | +example, if we allowed extra configuration such as compiler flags or |
| 135 | +command line recipes, then that configuration would need to be updated |
| 136 | +each time the build tools changed; it would also be inherently tied |
| 137 | +to the use of a specific toolchain.</p> |
| 138 | + |
| 139 | +<h2>Getting started with the go command</h2> |
| 140 | + |
| 141 | +<p>Finally, a quick tour of how to use the go command. |
| 142 | +As mentioned above, the default <code>$GOPATH</code> on Unix is <code>$HOME/go</code>. |
| 143 | +We'll store our programs there. |
| 144 | +To use a different location, you can set <code>$GOPATH</code>; |
| 145 | +see <a href="/doc/code.html">How to Write Go Code</a> for details. |
| 146 | + |
| 147 | +<p>We first add some source code. Suppose we want to use |
| 148 | +the indexing library from the codesearch project along with a left-leaning |
| 149 | +red-black tree. We can install both with the "<code>go get</code>" |
| 150 | +subcommand:</p> |
| 151 | + |
| 152 | +<pre> |
| 153 | +$ go get github.com/google/codesearch/index |
| 154 | +$ go get github.com/petar/GoLLRB/llrb |
| 155 | +$ |
| 156 | +</pre> |
| 157 | + |
| 158 | +<p>Both of these projects are now downloaded and installed into <code>$HOME/go</code>, |
| 159 | +which contains the two directories |
| 160 | +<code>src/github.com/google/codesearch/index/</code> and |
| 161 | +<code>src/github.com/petar/GoLLRB/llrb/</code>, along with the compiled |
| 162 | +packages (in <code>pkg/</code>) for those libraries and their dependencies.</p> |
| 163 | + |
| 164 | +<p>Because we used version control systems (Mercurial and Git) to check |
| 165 | +out the sources, the source tree also contains the other files in the |
| 166 | +corresponding repositories, such as related packages. The "<code>go list</code>" |
| 167 | +subcommand lists the import paths corresponding to its arguments, and |
| 168 | +the pattern "<code>./...</code>" means start in the current directory |
| 169 | +("<code>./</code>") and find all packages below that directory |
| 170 | +("<code>...</code>"):</p> |
| 171 | + |
| 172 | +<pre> |
| 173 | +$ cd $HOME/go/src |
| 174 | +$ go list ./... |
| 175 | +github.com/google/codesearch/cmd/cgrep |
| 176 | +github.com/google/codesearch/cmd/cindex |
| 177 | +github.com/google/codesearch/cmd/csearch |
| 178 | +github.com/google/codesearch/index |
| 179 | +github.com/google/codesearch/regexp |
| 180 | +github.com/google/codesearch/sparse |
| 181 | +github.com/petar/GoLLRB/example |
| 182 | +github.com/petar/GoLLRB/llrb |
| 183 | +$ |
| 184 | +</pre> |
| 185 | + |
| 186 | +<p>We can also test those packages:</p> |
| 187 | + |
| 188 | +<pre> |
| 189 | +$ go test ./... |
| 190 | +? github.com/google/codesearch/cmd/cgrep [no test files] |
| 191 | +? github.com/google/codesearch/cmd/cindex [no test files] |
| 192 | +? github.com/google/codesearch/cmd/csearch [no test files] |
| 193 | +ok github.com/google/codesearch/index 0.203s |
| 194 | +ok github.com/google/codesearch/regexp 0.017s |
| 195 | +? github.com/google/codesearch/sparse [no test files] |
| 196 | +? github.com/petar/GoLLRB/example [no test files] |
| 197 | +ok github.com/petar/GoLLRB/llrb 0.231s |
| 198 | +$ |
| 199 | +</pre> |
| 200 | + |
| 201 | +<p>If a go subcommand is invoked with no paths listed, it operates on the |
| 202 | +current directory:</p> |
| 203 | + |
| 204 | +<pre> |
| 205 | +$ cd github.com/google/codesearch/regexp |
| 206 | +$ go list |
| 207 | +github.com/google/codesearch/regexp |
| 208 | +$ go test -v |
| 209 | +=== RUN TestNstateEnc |
| 210 | +--- PASS: TestNstateEnc (0.00s) |
| 211 | +=== RUN TestMatch |
| 212 | +--- PASS: TestMatch (0.00s) |
| 213 | +=== RUN TestGrep |
| 214 | +--- PASS: TestGrep (0.00s) |
| 215 | +PASS |
| 216 | +ok github.com/google/codesearch/regexp 0.018s |
| 217 | +$ go install |
| 218 | +$ |
| 219 | +</pre> |
| 220 | + |
| 221 | +<p>That "<code>go install</code>" subcommand installs the latest copy of the |
| 222 | +package into the pkg directory. Because the go command can analyze the |
| 223 | +dependency graph, "<code>go install</code>" also installs any packages that |
| 224 | +this package imports but that are out of date, recursively.</p> |
| 225 | + |
| 226 | +<p>Notice that "<code>go install</code>" was able to determine the name of the |
| 227 | +import path for the package in the current directory, because of the convention |
| 228 | +for directory naming. It would be a little more convenient if we could pick |
| 229 | +the name of the directory where we kept source code, and we probably wouldn't |
| 230 | +pick such a long name, but that ability would require additional configuration |
| 231 | +and complexity in the tool. Typing an extra directory name or two is a small |
| 232 | +price to pay for the increased simplicity and power.</p> |
| 233 | + |
| 234 | +<h2>Limitations</h2> |
| 235 | + |
| 236 | +<p>As mentioned above, the go command is not a general-purpose build |
| 237 | +tool. |
| 238 | +In particular, it does not have any facility for generating Go |
| 239 | +source files <em>during</em> a build, although it does provide |
| 240 | +<a href="/cmd/go/#hdr-Generate_Go_files_by_processing_source"><code>go</code> |
| 241 | +<code>generate</code></a>, |
| 242 | +which can automate the creation of Go files <em>before</em> the build. |
| 243 | +For more advanced build setups, you may need to write a |
| 244 | +makefile (or a configuration file for the build tool of your choice) |
| 245 | +to run whatever tool creates the Go files and then check those generated source files |
| 246 | +into your repository. This is more work for you, the package author, |
| 247 | +but it is significantly less work for your users, who can use |
| 248 | +"<code>go get</code>" without needing to obtain and build |
| 249 | +any additional tools.</p> |
| 250 | + |
| 251 | +<h2>More information</h2> |
| 252 | + |
| 253 | +<p>For more information, read <a href="/doc/code.html">How to Write Go Code</a> |
| 254 | +and see the <a href="/cmd/go/">go command documentation</a>.</p> |
0 commit comments