08.1.md

#8.1 Sockets Some network application developers says that lower layer is all about programming of sockets, it's may not true in all points, but many applications are using sockets indeed. How you ever think about these questions, how browsers communicate with web servers when you are surfing on the internet? How MSN connects you and your friends? Many services like these are using sockets to transfer data, so sockets occupy an important position in network programming today, and we're going to use sockets in Go in this section.

##What is socket？ Socket is from Unix, and "everything is a file" is the basic philosophy of Unix, so everything can be operated with "open -> write/read -> close". Socket is one implementation of this philosophy, network socket is a special I/O, and socket is a kind of file descriptor. Socket has a function call for opening a socket like a file, it returns a int descriptor of socket, and it will be used in following operations like create connection, transfer data, etc.

Here are two types of sockets that are commonly used: stream socket(SOCK_STREAM) and datagram socket(SOCK_DGRAM). Stream socket is connection-oriented, like TCP; datagram socket does not have connection, like UDP.

##Socket communication Before we understand how sockets communicate each other, we need to figure out how to make sure that every socket is unique, otherwise communication is out of question. We can give every process a PID in local, but it's not able to work in network. Fortunately, TCP/IP helps us this solve this problem. IP address of network layer is unique in network of hosts, and "protocol + port" is unique of applications in hosts, then we can use this principle to make sockets be unique.

Figure 8.1 network protocol layers

Applications that are based on TCP/IP are using APIs of sockets for programming, and network becomes big part of our lives, that's why some people say that "everything is about socket".

##Socket basic knowledge We know that socket has two types which are TCP socket and UDP socket, TCP and UDP are protocols, and we also need IP address and port to have unique sockets.

###IPv4 Global internet uses TCP/IP as its protocol, where IP is the network layer and core part of TCP/IP. IPv4 means its version is 4, development to date has spent over 30 years.

The bit number of IPv4 address is 32, which means 2^32 devices are able to connect internet. Due to rapid develop of internet, IP addresses are almost out of stock in recent years.

Address format:127.0.0.1, 172.122.121.111.

###IPv6 IPv6 is the next version or next generation of internet, it's being made for solving problems of implementing IPv4. Its address has 128 bit long, so we don't need to worry about shortage of addresses, for example, you can have more than 1000 IP addresses for every square meter on the earth with IPv6. Other problems like peer to peer connection, service quality(QoS), security, multiple broadcast, etc are also be improved.

Address format: 2002:c0e8:82e7:0:0:0:c0e8:82e7.

###IP types in Go Package net in Go provides many types, functions and methods for network programming, the definition of IP as follows:

type IP []byte

Functions ParseIP(s string) IP is for converting IP format from IPv4 to IPv6:

package main
import (
	"net"
	"os"
	"fmt"
)
func main() {
	if len(os.Args) != 2 {
		fmt.Fprintf(os.Stderr, "Usage: %s ip-addr\n", os.Args[0])
		os.Exit(1)
	}
	name := os.Args[1]
	addr := net.ParseIP(name)
	if addr == nil {
		fmt.Println("Invalid address")
	} else {
		fmt.Println("The address is ", addr.String())
	}
	os.Exit(0)
}

It returns corresponding IP format for given IP address.

##TCP socket What we can do when we know how to visit a web service through a network port? As a client, we can send a request to appointed network port, and gets its feedback; as a server, we need to bind a service to appointed network port, wait for clients' requests and gives them feedback.

In package net, it has a type called TCPConn for this kind of clients and servers, this type has two key functions:

func (c *TCPConn) Write(b []byte) (n int, err os.Error)
func (c *TCPConn) Read(b []byte) (n int, err os.Error)

TCPConn can be used as either client or server for reading and writing data.

We also need a TCPAddr to represent TCP address information:

type TCPAddr struct {
	IP IP
	Port int
}

We use function ResolveTCPAddr to get a TCPAddr in Go:

func ResolveTCPAddr(net, addr string) (*TCPAddr, os.Error)

• Arguments of net can be one of "tcp4", "tcp6" or "tcp", where are TCP(IPv4-only), TCP(IPv6-only) or TCP(IPv4 or IPv6).
• addr can be domain name or IP address, like "www.google.com:80" or "127.0.0.1:22".

###TCP client Go uses function DialTCP in package net to create a TCP connection, and returns a TCPConn object; after connection created, server has a same type connection object for this connection, and exchange data with each other. In general, clients send requests to server through TCPConn and get servers respond information; servers read and parse clients requests, then return feedback. This connection will not be invalid until one side close it. The function of creating connection as follows:

func DialTCP(net string, laddr, raddr *TCPAddr) (c *TCPConn, err os.Error)

• Arguments of net can be one of "tcp4", "tcp6" or "tcp", where are TCP(IPv4-only), TCP(IPv6-only) or TCP(IPv4 or IPv6).
• laddr represents local address, set it to nil in most of cases.
• raddr represents remote address.

Let's write a simple example to simulate a client request to connect a web server based on HTTP. We need a simple HTTP request header:

"HEAD / HTTP/1.0\r\n\r\n"

Server respond information format may like follows:

HTTP/1.0 200 OK
ETag: "-9985996"
Last-Modified: Thu, 25 Mar 2010 17:51:10 GMT
Content-Length: 18074
Connection: close
Date: Sat, 28 Aug 2010 00:43:48 GMT
Server: lighttpd/1.4.23

Client code:

package main

import (
	"fmt"
	"io/ioutil"
	"net"
	"os"
)

func main() {
	if len(os.Args) != 2 {
		fmt.Fprintf(os.Stderr, "Usage: %s host:port ", os.Args[0])
		os.Exit(1)
	}
	service := os.Args[1]
	tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
	checkError(err)
	conn, err := net.DialTCP("tcp", nil, tcpAddr)
	checkError(err)
	_, err = conn.Write([]byte("HEAD / HTTP/1.0\r\n\r\n"))
	checkError(err)
	result, err := ioutil.ReadAll(conn)
	checkError(err)
	fmt.Println(string(result))
	os.Exit(0)
}
func checkError(err error) {
	if err != nil {
		fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
		os.Exit(1)
	}
}

In above example, we use user input as argument service and pass to net.ResolveTCPAddr to get a tcpAddr, then we pass tcpAddr to function DialTCP to create a TCP connection conn, then use conn to send request information. Finally, use ioutil.ReadAll to read all content from conn, which is server feedback.

###TCP server We have a TCP client now, and we also can use package net to write a TCP server. In server side, we need to bind service to specific inactive port, and listen to this port, so it's able to receive client requests.

func ListenTCP(net string, laddr *TCPAddr) (l *TCPListener, err os.Error)
func (l *TCPListener) Accept() (c Conn, err os.Error)

Arguments are the same as DialTCP, let's implement a time sync service, port is 7777:

package main

import (
	"fmt"
	"net"
	"os"
	"time"
)

func main() {
	service := ":7777"
	tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
	checkError(err)
	listener, err := net.ListenTCP("tcp", tcpAddr)
	checkError(err)
	for {
		conn, err := listener.Accept()
		if err != nil {
			continue
		}
		daytime := time.Now().String()
		conn.Write([]byte(daytime)) // don't care about return value
		conn.Close()                // we're finished with this client
	}
}
func checkError(err error) {
	if err != nil {
		fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
		os.Exit(1)
	}
}

After the service started, it is waiting for clients requests. When it gets client requests, Accept and gives feedback of current time information. It's worth noting that when error occurs in for loop, it continue running instead of exiting because record error log in server is better than crash, which makes service be stable.

The above code is not good enough because we didn't use goroutine to accept multiple request as same time. Let's make it better:

package main

import (
	"fmt"
	"net"
	"os"
	"time"
)

func main() {
	service := ":1200"
	tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
	checkError(err)
	listener, err := net.ListenTCP("tcp", tcpAddr)
	checkError(err)
	for {
		conn, err := listener.Accept()
		if err != nil {
			continue
		}
		go handleClient(conn)
	}
}

func handleClient(conn net.Conn) {
	defer conn.Close()
	daytime := time.Now().String()
	conn.Write([]byte(daytime)) // don't care about return value
	// we're finished with this client
}
func checkError(err error) {
	if err != nil {
		fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
		os.Exit(1)
	}
}

Through the separation of business process to the function handleClient, we implemented concurrency for our service. Simply add go keyword to implement concurrency, it's one of reasons that goroutine is simple and powerful.

Some people may ask: this server does not do anything meaningful, what if we need to send multiple requests for different time format in one connection, how can we do that?

package main

import (
	"fmt"
	"net"
	"os"
	"time"
	"strconv"
)

func main() {
	service := ":1200"
	tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
	checkError(err)
	listener, err := net.ListenTCP("tcp", tcpAddr)
	checkError(err)
	for {
		conn, err := listener.Accept()
		if err != nil {
			continue
		}
		go handleClient(conn)
	}
}

func handleClient(conn net.Conn) {
	conn.SetReadDeadline(time.Now().Add(2 * time.Minute)) // set 2 minutes timeout
	request := make([]byte, 128) // set maxium request length to 128KB to prevent flood attack
	defer conn.Close()  // close connection before exit
	for {
		read_len, err := conn.Read(request)

		if err != nil {
			fmt.Println(err)
			break
		}

		if read_len == 0 {
			break // connection already closed by client
		} else if string(request) == "timestamp" {
			daytime := strconv.FormatInt(time.Now().Unix(), 10)
			conn.Write([]byte(daytime))
		} else {
			daytime := time.Now().String()
			conn.Write([]byte(daytime)) 
		}

		request = make([]byte, 128) // clear last read content
	}
}

func checkError(err error) {
	if err != nil {
		fmt.Fprintf(os.Stderr, "Fatal error: %s", err.Error())
		os.Exit(1)
	}
}

In this example, we use conn.Read() to constantly read client requests, and we cannot close connection because client may have more requests. Due to timeout of conn.SetReadDeadline(), it closes automatically when client has not request sent in a period of time, so it jumps of code block of for loop. Notice that request need to create max size limitation in order to prevent flood attack; clean resource after processed every request because conn.Read() append new content instead of rewriting.

###Control TCP connections Control functions of TCP:

func DialTimeout(net, addr string, timeout time.Duration) (Conn, error)

Setting timeout of connections, it's suitable for clients and servers:

func (c *TCPConn) SetReadDeadline(t time.Time) error
func (c *TCPConn) SetWriteDeadline(t time.Time) error

Setting timeout of write/read of one connection:

func (c *TCPConn) SetKeepAlive(keepalive bool) os.Error

It's worth to consider whether keep long connection between client and server, long connection can reduce overhead of creating connections, it's good for applications that need to exchange data frequently.

More information please loop up official documentation of package net.

##UDP socket The only different between UDP socket and TCP socket is processing method for multiple requests in server side, it's because UDP does not have function like Accept. Other functions just replace TCP with UDP.

func ResolveUDPAddr(net, addr string) (*UDPAddr, os.Error)
func DialUDP(net string, laddr, raddr *UDPAddr) (c *UDPConn, err os.Error)
func ListenUDP(net string, laddr *UDPAddr) (c *UDPConn, err os.Error)
func (c *UDPConn) ReadFromUDP(b []byte) (n int, addr *UDPAddr, err os.Error
func (c *UDPConn) WriteToUDP(b []byte, addr *UDPAddr) (n int, err os.Error)

UDP client code sample:

package main

import (
	"fmt"
	"net"
	"os"
)

func main() {
	if len(os.Args) != 2 {
		fmt.Fprintf(os.Stderr, "Usage: %s host:port", os.Args[0])
		os.Exit(1)
	}
	service := os.Args[1]
	udpAddr, err := net.ResolveUDPAddr("udp4", service)
	checkError(err)
	conn, err := net.DialUDP("udp", nil, udpAddr)
	checkError(err)
	_, err = conn.Write([]byte("anything"))
	checkError(err)
	var buf [512]byte
	n, err := conn.Read(buf[0:])
	checkError(err)
	fmt.Println(string(buf[0:n]))
	os.Exit(0)
}
func checkError(err error) {
	if err != nil {
		fmt.Fprintf(os.Stderr, "Fatal error ", err.Error())
		os.Exit(1)
	}
}

UDP server code sample:

package main

import (
	"fmt"
	"net"
	"os"
	"time"
)

func main() {
	service := ":1200"
	udpAddr, err := net.ResolveUDPAddr("udp4", service)
	checkError(err)
	conn, err := net.ListenUDP("udp", udpAddr)
	checkError(err)
	for {
		handleClient(conn)
	}
}
func handleClient(conn *net.UDPConn) {
	var buf [512]byte
	_, addr, err := conn.ReadFromUDP(buf[0:])
	if err != nil {
		return
	}
	daytime := time.Now().String()
	conn.WriteToUDP([]byte(daytime), addr)
}
func checkError(err error) {
	if err != nil {
		fmt.Fprintf(os.Stderr, "Fatal error ", err.Error())
		os.Exit(1)
	}
}

##Summary Through description and programming of TCP and UDP sockets, we can see that Go has very good support for socket programming, and they are easy to use. Go also provides many functions for building high performance socket applications.

##Links

• Directory
• Previous section: Web services
• Next section: WebSocket