Added multithreaded support

It's now possible to fire up multiple event loops in separate
goroutines. All that's needed is to set the `events.NumLoops` options
before calling `Serve`.

There are a few breaking API changes.

- The events pass an evio.Conn param that represents the unique
incoming socket connection.
- Prewrite and Postwrite events have been removed.
- Wake and Dial functions have been removed.
- The Transform utility has been removed.

The older version has been tagged as `v0.1.0` for vendoring purposes.

Author: tidwall

README.md

@@ -17,23 +17,20 @@ This project is not intended to be a general purpose replacement for the standar
 
 You would not want to use this framework if you need to handle long-running requests (milliseconds or more). For example, a web api that needs to connect to a mongo database, authenticate, and respond; just use the Go net/http package instead.
 
-There are many popular event loop based applications in the wild such as Nginx, Haproxy, Redis, and Memcached. All of these are single-threaded and very fast and written in C.
-
-The reason I wrote this framework is so I can build certain network services that perform like the C apps above, but I also want to continue to work in Go.
-
+There are many popular event loop based applications in the wild such as Nginx, Haproxy, Redis, and Memcached. All of these are very fast and written in C.
 
+The reason I wrote this framework is so that I can build certain networking services that perform like the C apps above, but I also want to continue to work in Go.
 
 ## Features
 
-- [Fast](#performance) single-threaded event loop
+- [Fast](#performance) single-threaded or [multithreaded](#multithreaded) event loop
+- Built-in [load balancing](#load-balancing) options
 - Simple API
 - Low memory usage
 - Supports tcp, [udp](#udp), and unix sockets
 - Allows [multiple network binding](#multiple-addresses) on the same event loop
 - Flexible [ticker](#ticker) event
 - Fallback for non-epoll/kqueue operating systems by simulating events with the [net](https://golang.org/pkg/net/) package
-- Ability to [wake up](#wake-up) connections from long running background operations
-- [Dial](#dial-out) an outbound connection and process/proxy on the event loop
 - [SO_REUSEPORT](#so_reuseport) socket option
 
 ## Getting Started
@@ -61,7 +58,7 @@ import "github.com/tidwall/evio"
 
 func main() {
 	var events evio.Events
-	events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
+	events.Data = func(c evio.Conn, in []byte) (out []byte, action evio.Action) {
 		out = in
 		return
 	}
@@ -89,8 +86,6 @@ The event type has a bunch of handy events:
 - `Closed` fires when a connection has closed.
 - `Detach` fires when a connection has been detached using the `Detach` return action.
 - `Data` fires when the server receives new data from a connection.
-- `Prewrite` fires prior to all write attempts from the server.
-- `Postwrite` fires immediately after every write attempt.
 - `Tick` fires immediately after the server starts and will fire again after a specified interval.
 
 ### Multiple addresses
@@ -114,127 +109,30 @@ events.Tick = func() (delay time.Duration, action Action){
 }
 ```
 
-### Wake up
-
-A connection can be woken up using the `Wake` function that is made available through the `Serving` event. This is useful for when you need to offload an operation to a background goroutine and then later notify the event loop that it's time to send some data.
-
-Example echo server that when encountering the line "exec" it waits 5 seconds before responding.
-
-```go
-var srv evio.Server
-var mu sync.Mutex
-var execs = make(map[int]int)
-
-events.Serving = func(srvin evio.Server) (action evio.Action) {
-	srv = srvin // hang on to the server control, which has the Wake function
-	return
-}
-events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
-	if in == nil {
-		// look for `in` param equal to `nil` following a wake call.
-		mu.Lock()
-		for execs[id] > 0 {
-			out = append(out, "exec\r\n"...)
-			execs[id]--
-		}
-		mu.Unlock()
-	} else if string(in) == "exec\r\n" {
-		go func(){
-			// do some long running operation
-			time.Sleep(time.Second*5)
-			mu.Lock()
-			execs[id]++
-			mu.Unlock()
-			srv.Wake(id)
-		}()
-	} else {
-		out = in
-	}
-	return
-}
-```
-
-### Dial out
-
-An outbound connection can be created by using the `Dial` function that is made available through the `Serving` event. Dialing a new connection will return a new connection ID and attach that connection to the event loop in the same manner as incoming connections. This operation is completely non-blocking including any DNS resolution.
-
-All new outbound connection attempts will immediately fire an `Opened` event and end with a `Closed` event. A failed connection will send the connection error through the `Closed` event.
-
-```go
-var srv evio.Server
-var mu sync.Mutex
-var execs = make(map[int]int)
-
-events.Serving = func(srvin evio.Server) (action evio.Action) {
-    srv = srvin // hang on to the server control, which has the Dial function
-    return
-}
-events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
-    if string(in) == "dial\r\n" {
-        id := srv.Dial("tcp://google.com:80")
-        // We now established an outbound connection to google.
-        // Treat it like you would incoming connection.
-    } else {
-        out = in
-    }
-    return
-}
-```
-
-### Data translations
-
-The `Translate` function wraps events and provides a `ReadWriter` that can be used to translate data off the wire from one format to another. This can be useful for transparently adding compression or encryption.
-
-For example, let's say we need TLS support:
-
-```go
-var events Events
-
-// ... fill the events with happy functions
-
-cer, err := tls.LoadX509KeyPair("certs/ssl-cert-snakeoil.pem", "certs/ssl-cert-snakeoil.key")
-if err != nil {
-	log.Fatal(err)
-}
-config := &tls.Config{Certificates: []tls.Certificate{cer}}
-
-// wrap the events with a TLS translator
-
-events = evio.Translate(events, nil, 
-	func(id int, rw io.ReadWriter) io.ReadWriter {
-		return tls.Server(evio.NopConn(rw), config)
-	},
-)
-
-log.Fatal(evio.Serve(events, "tcp://0.0.0.0:443"))
-```
+## UDP
 
-Here we wrapped the event with a TLS translator. The `evio.NopConn` function is used to converts the `ReadWriter` a `net.Conn` so the `tls.Server()` call will work.
+The `Serve` function can bind to UDP addresses. 
 
-There's a working TLS example at [examples/http-server/main.go](examples/http-server/main.go) that binds to port 8080 and 4443 using an developer SSL certificate. The 8080 connections will be insecure and the 4443 will be secure.
+- All incoming and outgoing packets are not buffered and sent individually.
+- The `Opened` and `Closed` events are not availble for UDP sockets, only the `Data` event.
 
-```sh
-$ cd examples/http-server
-$ go run main.go --tlscert example.pem
-2017/11/02 06:24:33 http server started on port 8080
-2017/11/02 06:24:33 https server started on port 4443
-```
+## Multithreaded
 
-```sh
-$ curl http://localhost:8080
-Hello World!
-$ curl -k https://localhost:4443
-Hello World!
-```
+The `events.NumLoops` options sets the number of loops to use for the server. 
+Setting this to a value greater than 1 will effectively make the server multithreaded for multi-core machines. 
+Which means you must take care with synchonizing memory between all event callbacks. 
+Setting to 0 or 1 will run the server single-threaded. 
+Setting to -1 will automatically assign this value equal to `runtime.NumProcs()`.
 
-## UDP
+## Load balancing
 
-The `Serve` function can bind to UDP addresses. 
+The `events.LoadBalance` options sets the load balancing method. 
+Load balancing is always a best effort to attempt to distribute the incoming connections between multiple loops.
+This option is only available when `events.NumLoops` is set.
 
-- The `Opened` event will fire when a UDP packet is received from a new remote address.
-- The `Closed` event will fire when the server is shutdown or the `Close` action is explicitly returned from an event.
-- The `Wake` and `Dial` operations are not available to UDP connections.
-- All incoming and outgoing packets are not buffered and sent individually.
+- `Random` requests that connections are randomly distributed.
+- `RoundRobin` requests that connections are distributed to a loop in a round-robin fashion.
+- `LeastConnections` assigns the next accepted connection to the loop with the least number of active connections.
 
 ## SO_REUSEPORT