Rewrite (#41)

This PR does a very large refactor (aka rewrite) of the driver do
prepare for general availability.

## Design Considerations
* Any request to external systems must return as soon as the context
deadline is exceeded
* Any request to external systems that do not take a context (open/close
session) must have a timeout
* Packages that are not meant to be public to users must be in /internal
* No extension of the standard interface as context cancellation
fulfills the requirements; thus, concepts such as “operation” or
“cursor” are not needed.
* Contract testing instead of full integration testing (DUST). This
enables all tests to run locally and in Github actions. And they are
fast. This is done by creating a test thrift server using httptest
package.
* Implement RoundTripper interface and wrap TCLIServiceClient to have
access to HTTP response headers and status. This will enable retry.
* Fully declarative configuration with defaults instead of constants
spreaded around

## New features
* Support for operation cancellation and timeout by using context.
* Direct Results. If the resultsset is small and the query takes less
than 5 seconds, there will only be one request made, shortcutting 3 or 4
requests from the standard thrift protocol.
* Support for session parameters (maxRows, timeout, catalog, schema, and
any other session params in SET such as timezone, ansi_mode, etc).
* Fully configurable logging

## Regressions
* Parameterized queries are not supported anymore. Client side string
interpolation is prone to errors and Databricks will soon support it
from the server side anyway.
* DSN does not start with "databricks" anymore. So one can just do
`username:password@host:port/path/to/db?param=foo`

## Testing
There are 3 levels of testing:
* Unit: see internal/config/config_test.go
* Contract: see rows_test.go
* End to End with test server: see driver_e2e_test.go

## Bug fixes
* Many with regards to how to query, as synchronous queries were never
really supported in the server.
* Some data type supported conversions where fixed

## Examples

```go
package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"
	"os"
	"time"

	_ "github.com/databricks/databricks-sql-go"
)

func main() {
	// Opening a driver typically will not attempt to connect to the database.
	db, err := sql.Open("databricks", <DSN>)

	if err != nil {
		// This will not be a connection error, but a DSN parse error or
		// another initialization error.
		log.Fatal(err)
	}
	defer db.Close()

	ctx, cancel1 := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel1()
	var res int
	err := db.QueryRowContext(ctx, `SELECT id FROM RANGE(100000000) ORDER BY RANDOM() + 2 asc`).Scan(&res)
	if err != nil {
		fmt.Printf("err: %v\n", err1)
	} else {
		fmt.Printf("result: %d\n", res)
	}
}

```

```go
package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"
	"os"
	"strconv"

	dbsql "github.com/databricks/databricks-sql-go"
)

func main() {
	port, err := strconv.Atoi(os.Getenv("DATABRICKS_PORT"))
	if err != nil {
		log.Fatal(err.Error())
	}
	connector, err := dbsql.NewConnector(
		dbsql.WithServerHostname(os.Getenv("DATABRICKS_HOST")),
		dbsql.WithPort(port),
		dbsql.WithHTTPPath(os.Getenv("DATABRICKS_HTTPPATH")),
		dbsql.WithAccessToken(os.Getenv("DATABRICKS_ACCESSTOKEN")),
		dbsql.WithSessionParams(map[string]string{"timezone": "America/Sao_Paulo", "ansi_mode": "true"}),
	)
	if err != nil {
		// This will not be a connection error, but a DSN parse error or
		// another initialization error.
		log.Fatal(err)
	}
	db := sql.OpenDB(connector)
	defer db.Close()
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	res := &time.Time{}
	if qerr := db.QueryRowContext(ctx, `select now()`).Scan(res); qerr != nil {
		fmt.Printf("err: %v\n", qerr)
	} else {
		fmt.Println(res)
	}

}

```

## TODO
* Testing is still in a minimum, even though we added a lot more tests
than previously
* Retry. The interfaces are all there, we just need to write the logic
to retry now.

Signed-off-by: Andre Furlan <[email protected]>
Co-authored-by: Raymond Cypher <[email protected]>

Author: andrefurlan-db

.github/CODEOWNERS

@@ -2,4 +2,5 @@
 # the repo. Unless a later match takes precedence, these
 # users will be requested for review when someone opens a
 # pull request.
-*       @arikfr @susodapop @moderakh @yunbodeng-db
+*       @mattdeekay @susodapop @moderakh @yunbodeng-db @andrefurlan-db @rcypher-databricks
+

.gitignore

@@ -9,6 +9,7 @@
 
 # Test binary, built with `go test -c`
 *.test
+test-results/
 
 # Output of the go coverage tool, specifically when used with LiteIDE
 *.out
@@ -45,3 +46,21 @@ Icon
 Network Trash Folder
 Temporary Items
 .apdisk
+
+
+.env
+
+# Output of the production build
+bin/
+
+
+# Dependency directories
+vendor/
+
+*.log
+*.bak
+_tmp*
+
+.vscode/
+__debug_bin
+.DS_Store

.golangci.yml

@@ -0,0 +1,62 @@
+linters:
+  disable-all: true
+  enable:
+    - bodyclose
+    - deadcode
+    - depguard
+    - dogsled
+    # - dupl
+    - errcheck
+    # - exportloopref
+    # - funlen
+    # - gochecknoinits
+    # - goconst
+    # - gocritic
+    # - gocyclo
+    - gofmt
+    # - goimports
+    # - gomnd
+    - goprintffuncname
+    - gosec
+    - gosimple
+    - govet
+    - ineffassign
+    # - lll
+    # - misspell
+    - nakedret
+    # - noctx
+    - nolintlint
+    - staticcheck
+    - structcheck
+    # - stylecheck
+    - typecheck
+    # - unconvert
+    # - unparam
+    - unused
+    - varcheck
+    # - whitespace
+
+  # don't enable:
+  # - asciicheck
+  # - scopelint
+  # - gochecknoglobals
+  # - gocognit
+  # - godot
+  # - godox
+  # - goerr113
+  # - interfacer
+  # - maligned
+  # - nestif
+  # - prealloc
+  # - testpackage
+  # - revive
+  # - wsl
+
+linters-settings:
+  gosec:
+    exclude-generated: true
+    severity: "low"
+    confidence: "low"
+
+run:
+  timeout: 5m

Makefile

@@ -0,0 +1,77 @@
+BINARY         = vulcan
+PACKAGE        = github.com/datajoydev/$(BINARY)
+VER_PREFIX     = $(PACKAGE)/version
+PRODUCTION     ?= false
+DATE           = $(shell date "+%Y-%m-%d")
+GIT_BRANCH     = $(shell git rev-parse --abbrev-ref 2>/dev/null)
+GIT_COMMIT     = $(shell git rev-parse HEAD 2>/dev/null)
+GIT_TAG        = $(shell if [ -z "`git status --porcelain`" ]; then git describe --exact-match --tags HEAD 2>/dev/null; fi)
+VERSIONREL     = $(shell if [ -z "`git status --porcelain`" ]; then git rev-parse --short HEAD 2>/dev/null ; else echo "dirty"; fi)
+PKGS           = $(shell go list ./... | grep -v /vendor)
+LDFLAGS        = -X $(VER_PREFIX).Version=$(VERSIONREL) -X $(VER_PREFIX).Revision=$(GIT_COMMIT) -X $(VER_PREFIX).Branch=$(GIT_BRANCH) -X $(VER_PREFIX).BuildUser=$(shell whoami)  -X $(VER_PREFIX).BuildDate=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")
+GOBUILD_ARGS   =
+GO             = CGO_ENABLED=0 go
+PLATFORMS      = windows linux darwin
+os             = $(word 1, $@)
+
+TEST_RESULTS_DIR ?= ./test-results
+
+
+ifneq (${GIT_TAG},)
+override DOCKER_TAG = ${GIT_TAG}
+override VERSIONREL = ${GIT_TAG}
+endif
+
+ifeq (${PRODUCTION}, true)
+LDFLAGS += -w -s -extldflags "-static"
+GOBUILD_ARGS += -v
+endif
+
+.PHONY: help
+help:  ## Show this help.
+	@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {sub("\\\\n",sprintf("\n%22c"," "), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
+
+.PHONY: all
+all: gen fmt lint test coverage  ## format and test everything
+
+bin/golangci-lint: go.mod go.sum
+	curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ./bin v1.48.0
+
+bin/gotestsum: go.mod go.sum
+	@mkdir -p bin/
+	$(GO) build -o bin/gotestsum gotest.tools/gotestsum
+
+.PHONY: tools
+tools: bin/golangci-lint bin/gotestsum  ## Build the development tools
+
+.PHONY: fmt
+fmt:  ## Format the go code.
+	gofmt -w -s .
+
+.PHONY: lint
+lint: bin/golangci-lint  ## Lint the go code to ensure code sanity.
+	./bin/golangci-lint run
+
+.PHONY: test
+test: bin/gotestsum  ## Run the go unit tests.
+	@echo "INFO: Running all go unit tests."
+	CGO_ENABLED=0 ./bin/gotestsum --format pkgname-and-test-fails --junitfile $(TEST_RESULTS_DIR)/unit-tests.xml ./...
+
+.PHONY: coverage
+coverage: bin/gotestsum  ## Report the unit test code coverage.
+	@echo "INFO: Generating unit test coverage report."
+	CGO_ENABLED=0 ./bin/gotestsum --format pkgname-and-test-fails -- -coverprofile=$(TEST_RESULTS_DIR)/coverage.txt -covermode=atomic ./...
+	go tool cover -html=$(TEST_RESULTS_DIR)/coverage.txt -o $(TEST_RESULTS_DIR)/coverage.html
+
+.PHONY: sec
+sec: lint  ## Run the snyk vulnerability scans.
+	@echo "INFO: Running go vulnerability scans."
+	snyk test .
+
+.PHONY: build
+build: linux darwin  ## Build the multi-arch binaries
+
+.PHONY: $(PLATFORMS)
+$(PLATFORMS):
+	mkdir -p bin
+	GOOS=$(os) GOARCH=amd64 $(GO) build $(GOBUILD_ARGS) -ldflags '$(LDFLAGS)' -o bin/$(BINARY)-$(os)-amd64 .

NOTICE

@@ -48,22 +48,24 @@ You can set HTTP Timeout value by appending a `timeout` query parameter (in mill
 databricks://:[your token]@[Workspace hostname][Endpoint HTTP Path]?timeout=1000&maxRows=1000
 ```
 
-## Testing
-
+## Develop
+
+### Lint
+We use `golangci-lint` as the lint tool. If you use vs code, just add the following settings:
+``` json
+{
+    "go.lintTool": "golangci-lint",
+    "go.lintFlags": [
+        "--fast"
+    ]
+}
+```
 ### Unit Tests
 
 ```bash
 go test
 ```
 
-### e2e Tests
-
-To run tests against a SQL warehouse, you need to pass a DSN environment variable first:
-
-```
-$ DATABRICKS_DSN="databricks://:[email protected]/sql/1.0/endpoints/12345a1b2c3d456f" go test
-```
-
 ## Issues
 
 If you find any issues, feel free to create an issue or send a pull request directly.
@@ -74,4 +76,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md)
 
 ## License
 
-[Apache 2.0](https://github.com/databricks/databricks-sql-nodejs/blob/master/LICENSE)
+[Apache 2.0](https://github.com/databricks/databricks-sql-go/blob/main/LICENSE)