You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
That's great for simple GET requests, but suppose we want to *send* some data in the body of a POST request, for example. Here's how that works:
111
+
112
+
```go
113
+
script.Echo(data).Post(URL).Stdout()
114
+
```
115
+
116
+
If we need to customise the HTTP behaviour in some way, such as using our own HTTP client, we can do that:
117
+
118
+
```go
119
+
script.NewPipe().WithHTTPClient(&http.Client{
120
+
Timeout: 10 * time.Second,
121
+
}).Get("https://example.com").Stdout()
122
+
```
123
+
124
+
Or maybe we need to set some custom header on the request. No problem. We can just create the request in the usual way, and set it up however we want. Then we pass it to `Do`, which will actually perform the request:
The HTTP server could return some non-okay response, though; for example, “404 Not Found”. So what happens then?
133
+
134
+
In general, when any pipe stage (such as `Do`) encounters an error, it produces no output to subsequent stages. And `script` treats HTTP response status codes outside the range 200-299 as errors. So the answer for the previous example is that we just won't *see* any output from this program if the server returns an error response.
135
+
136
+
Instead, the pipe “remembers” any error that occurs, and we can retrieve it later by calling its `Error` method, or by using a *sink* method such as `String`, which returns an `error` value along with the result.
137
+
138
+
`Stdout` also returns an error, plus the number of bytes successfully written (which we don't care about for this particular case). So we can check that error, which is always a good idea in Go:
139
+
140
+
```go
141
+
_, err:= script.Do(req).Stdout()
142
+
if err != nil {
143
+
log.Fatal(err)
144
+
}
105
145
```
106
146
107
-
Suppose we want to execute some external program instead of doing the work ourselves. We can do that too:
147
+
If, as is common, the data we get from an HTTP request is in JSON format, we can use [JQ](https://stedolan.github.io/jq/) queries to interrogate it:
We can also run external programs and get their output:
108
154
109
155
```go
110
156
script.Exec("ping 127.0.0.1").Stdout()
111
157
```
112
158
113
-
But maybe we don't know the arguments yet; we might get them from the user, for example. We'd like to be able to run the external command repeatedly, each time passing it the next line of input. No worries:
159
+
Note that `Exec` runs the command concurrently: it doesn't wait for the command to complete before returning any output. That's good, because this `ping` command will run forever (or until we get bored).
160
+
161
+
Instead, when we read from the pipe using `Stdout`, we see each line of output as it's produced:
162
+
163
+
```
164
+
PING 127.0.0.1 (127.0.0.1): 56 data bytes
165
+
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.056 ms
166
+
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.054 ms
167
+
...
168
+
```
169
+
170
+
In the `ping` example, we knew the exact arguments we wanted to send the command, and we just needed to run it once. But what if we don't know the arguments yet? We might get them from the user, for example.
171
+
172
+
We might like to be able to run the external command repeatedly, each time passing it the next line of data from the pipe as an argument. No worries:
If there isn't a built-in operation that does what we want, we can just write our own:
178
+
That `{{.}}` is standard Go template syntax; it'll substitute each line of data from the pipe into the command line before it's executed. You can write as fancy a Go template expression as you want here (but this simple example probably covers most use cases).
179
+
180
+
If there isn't a built-in operation that does what we want, we can just write our own, using `Filter`:
120
181
121
182
```go
122
183
script.Echo("hello world").Filter(func (r io.Reader, w io.Writer) error {
Notice that the "hello world" appeared before the "filtered n bytes". Filters run concurrently, so the pipeline can start producing output before the input has been fully read.
193
+
The `func` we supply to `Filter` takes just two parameters: a reader to read from, and a writer to write to. The reader reads the previous stages of the pipe, as you might expect, and anything written to the writer goes to the *next* stage of the pipe.
194
+
195
+
If our `func` returns some error, then, just as with the `Do` example, the pipe's error status is set, and subsequent stages become a no-op.
196
+
197
+
Filters run concurrently, so the pipeline can start producing output before the input has been fully read, as it did in the `ping` example. In fact, most built-in pipe methods, including `Exec`, are implemented *using*`Filter`.
133
198
134
199
If we want to scan input line by line, we could do that with a `Filter` function that creates a `bufio.Scanner` on its input, but we don't need to:
135
200
@@ -193,12 +258,15 @@ These are functions that create a pipe with a given contents:
| [`Slice`](https://pkg.go.dev/github.com/bitfield/script#Slice) | slice elements, one per line
203
271
| [`Stdin`](https://pkg.go.dev/github.com/bitfield/script#Stdin) | standard input
204
272
@@ -212,6 +280,7 @@ Filters are methods on an existing pipe that also return a pipe, allowing you to
212
280
|[`Column`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Column)| Nth column of input |
213
281
|[`Concat`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Concat)| contents of multiple files |
214
282
|[`Dirname`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Dirname)| removes filename from each line, leaving only leading path components |
283
+
|[`Do`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Do)| response to supplied HTTP request |
215
284
|[`Echo`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Echo)| all input replaced by given string |
216
285
|[`Exec`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Exec)| filtered through external command |
217
286
|[`ExecForEach`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ExecForEach)| execute given command template for each line of input |
@@ -220,18 +289,20 @@ Filters are methods on an existing pipe that also return a pipe, allowing you to
220
289
|[`FilterScan`](https://pkg.go.dev/github.com/bitfield/script#Pipe.FilterScan)| user-supplied function filtering each line to a writer |
221
290
|[`First`](https://pkg.go.dev/github.com/bitfield/script#Pipe.First)| first N lines of input |
222
291
|[`Freq`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Freq)| frequency count of unique input lines, most frequent first |
292
+
|[`Get`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Get)| response to HTTP GET on supplied URL |
223
293
|[`Join`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Join)| replace all newlines with spaces |
224
294
|[`JQ`](https://pkg.go.dev/github.com/bitfield/script#Pipe.JQ)| result of `jq` query |
225
295
|[`Last`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Last)| last N lines of input|
226
296
|[`Match`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Match)| lines matching given string |
227
297
|[`MatchRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.MatchRegexp)| lines matching given regexp |
298
+
|[`Post`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Post)| response to HTTP POST on supplied URL |
228
299
|[`Reject`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Reject)| lines not matching given string |
229
300
|[`RejectRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.RejectRegexp)| lines not matching given regexp |
230
301
|[`Replace`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Replace)| matching text replaced with given string |
231
302
|[`ReplaceRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ReplaceRegexp)| matching text replaced with given string |
232
303
|[`SHA256Sums`](https://pkg.go.dev/github.com/bitfield/script#Pipe.SHA256Sums)| SHA-256 hashes of each listed file |
233
304
234
-
Note that filters run concurrently, rather than producing nothing until each stage has fully read its input. This is convenient for executing long-running comands, for example. If you do need to wait for the pipeline to complete, call `Wait`.
305
+
Note that filters run concurrently, rather than producing nothing until each stage has fully read its input. This is convenient for executing long-running comands, for example. If you do need to wait for the pipeline to complete, call [`Wait`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Wait).
235
306
236
307
## Sinks
237
308
@@ -254,6 +325,7 @@ Sinks are methods that return some data from a pipe, ending the pipeline and ext
0 commit comments