Move high-throughput sample to go application (hyperledger#364)

Signed-off-by: Nikhil Gupta <[email protected]>

high throughput app

Author: nikhil550

high-throughput/README.md

@@ -107,56 +107,86 @@ Change back into the `high-throughput` directory in `fabic-samples`. Start the n
 
 If successful, you will see messages of the Fabric test network being created and the chaincode being deployed, followed by the execution time of the script:
 ```
-Total setup execution time : 141 secs ...
+Total setup execution time : 81 secs ...
 ```
 
 The `high-throughput` chaincode is now ready to receive invocations.
 
 ### Invoke the chaincode
-All invocations are provided as scripts in `scripts` folder. You can use these scripts to create and remove assets that you put on the ledger.
+
+You can invoke the `high-througput` chaincode using a Go application in the `application-go` folder. The Go application will allow us to submit many transactions to the network concurrently. Navigate to the application:
+```
+cd application-go
+```
 
 #### Update
-The format for update is: `./scripts/update-invoke.sh name value operation` where `name` is the name of the variable to update, `value` is the value to
-add to the variable, and `operation` is either `+` or `-` depending on what type of operation you'd like to add to the variable. In the future,
-multiply/divide operations will be supported (or add them yourself to the chaincode as an exercise!)
+The format for update is: `go run app.go update name value operation` where `name` is the name of the variable to update, `value` is the value to add to the variable, and `operation` is either `+` or `-` depending on what type of operation you'd like to add to the variable.
 
-Example: `./scripts/update-invoke.sh myvar 100 +`
+Example: `go run app.go update myvar 100 +`
 
-#### Get
-The format for get is: `./get-invoke.sh name` where `name` is the name of the variable to get.
+#### Query
+You can query the value of a variable by running `go run app.go get name` where `name` is the name of the variable to get.
 
-Example: `./scripts/get-invoke.sh myvar`
+Example: `go run app.go get myvar`
 
 #### Prune
-Pruning takes all the deltas generated for a variable and combines them all into a single row, deleting all previous rows. This helps cleanup
-the ledger when many updates have been performed.
+Pruning takes all the deltas generated for a variable and combines them all into a single row, deleting all previous rows. This helps cleanup the ledger when many updates have been performed.
 
-The format for pruning is: `./scripts/prune-invoke.sh name` where `name` is the name of the variable to prune.
+The format for pruning is: `go run app.go prune name` where `name` is the name of the variable to prune.
 
-Example: `./scripts/prune-invoke.sh myvar`
+Example: `go run app.go prune myvar`
 
 #### Delete
-The format for delete is: `./delete-invoke.sh name` where `name` is the name of the variable to delete.
+The format for delete is: `go run app.go delete name` where `name` is the name of the variable to delete.
 
-Example: `./scripts/delete-invoke.sh myvar`
+Example: `go run app.go delete myvar`
 
 ### Test the Network
-Two scripts are provided to show the advantage of using this system when running many parallel transactions at once: `many-updates.sh` and
-`many-updates-traditional.sh`. The first script accepts the same arguments as `update-invoke.sh` but duplicates the invocation 1000 times
-and in parallel. The final value, therefore, should be the given update value * 1000. Run this script to confirm that your network is functioning
-properly. You can confirm this by checking your peer and orderer logs and verifying that no invocations are rejected due to improper versions.
 
-The second script, `many-updates-traditional.sh`, also sends 1000 transactions but using the traditional storage system. It'll update a single
-row in the ledger 1000 times, with a value incrementing by one each time (i.e. the first invocation sets it to 0 and the last to 1000). The
-expectation would be that the final value of the row is 999. However, the final value changes each time this script is run and you'll find
-errors in the peer and orderer logs.
+The application provides two methods that demonstrate the advantages of this system by submitting many concurrent transactions to the smart contract: `manyUpdates` and `manyUpdatesTraditional`. The first function accepts the same arguments as `update-invoke.sh` but runs the invocation 1000 times in parallel. The final value, therefore, should be the given update value * 1000.
 
-There are two other scripts, `get-traditional.sh`, which simply gets the value of a row in the traditional way, with no deltas, and `del-traditional.sh` will delete an asset in the traditional way.
+The second function, `manyUpdatesTraditional`, submits 1000 transactions that attempt to upddate the same key in the world state 1000 times.
+
+Run the following command to create and update `testvar1` a 1000 times:
+```
+go run app.go manyUpdates testvar1 100 +
+```
 
-Examples:
-`./scripts/many-updates.sh testvar 100 +` --> final value from `./scripts/get-invoke.sh testvar` should be 100000
+The application will query the variable after submitting the transaction. The result should be `100000`.
+
+We will now see what happens when you try to run 1000 concurrent updates using a traditional transaction. Run the following command to create a variable named `testvar2`:
+```
+go run app.go update testvar2 100 +
+```
+The variable will have a value of 100:
+```
+2020/10/27 18:01:45 Value of variable testvar2 :  100
+```
 
-`./scripts/many-updates-traditional.sh testvar` --> final value from `./scripts/get-traditional.sh testvar` is undefined
+Now lets try to update `testvar2` 1000 times in parallel:
+```
+go run app.go manyUpdatesTraditional testvar2 100 +
+```
+
+When the program ends, you may see that none of the updates succeeded.
+```
+2020/10/27 18:03:15 Final value of variable testvar2 :  100
+```
+
+The transactions failed because multiple transactions in each block updated the same key. Because of these transactions generated read/write conflicts, the transactions included in each block were rejected in the validation stage.
+
+You can can examine the peer logs to view the messages generated by the rejected blocks:
+
+
+`docker logs peer0.org1.example.com
+[...]
+2020-10-28 17:37:58.746 UTC [gossip.privdata] StoreBlock -> INFO 2190 [mychannel] Received block [407] from buffer
+2020-10-28 17:37:58.749 UTC [committer.txvalidator] Validate -> INFO 2191 [mychannel] Validated block [407] in 2ms
+2020-10-28 17:37:58.750 UTC [validation] validateAndPrepareBatch -> WARN 2192 Block [407] Transaction index [0] TxId [b6b14cf988b0d7d35d4e0d7a0d2ae0c9f5569bc10ec5010f03a28c22694b8ef6] marked as invalid by state validator. Reason code [MVCC_READ_CONFLICT]
+2020-10-28 17:37:58.750 UTC [validation] validateAndPrepareBatch -> WARN 2193 Block [407] Transaction index [1] TxId [9d7c4f6ff95a0f22e01d6ffeda261227752e78db43f2673ad4ea6f0fdace44d1] marked as invalid by state validator. Reason code [MVCC_READ_CONFLICT]
+2020-10-28 17:37:58.750 UTC [validation] validateAndPrepareBatch -> WARN 2194 Block [407] Transaction index [2] TxId [9cc228b61d8841208feb6160254aee098b1b3a903f645e62cfa12222e6f52e65] marked as invalid by state validator. Reason code [MVCC_READ_CONFLICT]
+2020-10-28 17:37:58.750 UTC [validation] validateAndPrepareBatch -> WARN 2195 Block [407] Transaction index [3] TxId [2ae78d363c30b5f3445f2b028ccac7cf821f1d5d5c256d8c17bd42f33178e2ed] marked as invalid by state validator. Reason code [MVCC_READ_CONFLICT]
+```
 
 ### Clean up
 

high-throughput/application-go/.gitignore

@@ -0,0 +1,4 @@
+wallet
+!wallet/.gitkeep
+
+keystore

high-throughput/application-go/app.go

@@ -0,0 +1,70 @@
+/*
+Copyright 2020 IBM All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package main
+
+import (
+	"log"
+	"os"
+
+	f "github.com/hyperledger/fabric-samples/high-throughput/application-go/functions"
+)
+
+func main() {
+
+	var function, variableName, change, sign string
+
+	if len(os.Args) <= 2 {
+		log.Println("Usage: function variableName")
+		log.Fatalf("functions: update manyUpdates manyUpdatesTraditional get prune delete")
+	} else if (os.Args[1] == "update" || os.Args[1] == "manyUpdates" || os.Args[1] == "manyUpdatesTraditional") && len(os.Args) < 5 {
+		log.Fatalf("error: provide value and operation")
+	} else if len(os.Args) == 3 {
+		function = os.Args[1]
+		variableName = os.Args[2]
+	} else if len(os.Args) == 5 {
+		function = os.Args[1]
+		variableName = os.Args[2]
+		change = os.Args[3]
+		sign = os.Args[4]
+	}
+
+	// Handle different functions
+	if function == "update" {
+		result, err := f.Update(function, variableName, change, sign)
+		if err != nil {
+			log.Fatalf("error: %v", err)
+		}
+		log.Println("Value of variable", string(variableName), ": ", string(result))
+
+	} else if function == "delete" || function == "prune" || function == "delstandard" {
+		result, err := f.DeletePrune(function, variableName)
+		if err != nil {
+			log.Fatalf("error: %v", err)
+		}
+		log.Println(string(result))
+	} else if function == "get" || function == "getstandard" {
+		result, err := f.Query(function, variableName)
+		if err != nil {
+			log.Fatalf("error: %v", err)
+		}
+		log.Println("Value of variable", string(variableName), ": ", string(result))
+	} else if function == "manyUpdates" {
+		log.Println("submitting 1000 concurrent updates...")
+		result, err := f.ManyUpdates("update", variableName, change, sign)
+		if err != nil {
+			log.Fatalf("error: %v", err)
+		}
+		log.Println("Final value of variable", string(variableName), ": ", string(result))
+	} else if function == "manyUpdatesTraditional" {
+		log.Println("submitting 1000 concurrent updates...")
+		result, err := f.ManyUpdates("putstandard", variableName, change, sign)
+		if err != nil {
+			log.Fatalf("error: %v", err)
+		}
+		log.Println("Final value of variable", string(variableName), ": ", string(result))
+	}
+}

high-throughput/application-go/functions/deletePrune.go

@@ -0,0 +1,69 @@
+/*
+Copyright 2020 IBM All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package functions
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+
+	"github.com/hyperledger/fabric-sdk-go/pkg/core/config"
+	"github.com/hyperledger/fabric-sdk-go/pkg/gateway"
+)
+
+// DeletePrune deletes or prunes a variable
+func DeletePrune(function, variableName string) ([]byte, error) {
+
+	err := os.Setenv("DISCOVERY_AS_LOCALHOST", "true")
+	if err != nil {
+		return nil, fmt.Errorf("error setting DISCOVERY_AS_LOCALHOST environemnt variable: %v", err)
+	}
+
+	wallet, err := gateway.NewFileSystemWallet("wallet")
+	if err != nil {
+		return nil, fmt.Errorf("failed to create wallet: %v", err)
+	}
+
+	if !wallet.Exists("appUser") {
+		err := populateWallet(wallet)
+		if err != nil {
+			return nil, fmt.Errorf("failed to populate wallet contents: %v", err)
+		}
+	}
+
+	ccpPath := filepath.Join(
+		"..",
+		"..",
+		"test-network",
+		"organizations",
+		"peerOrganizations",
+		"org1.example.com",
+		"connection-org1.yaml",
+	)
+
+	gw, err := gateway.Connect(
+		gateway.WithConfig(config.FromFile(filepath.Clean(ccpPath))),
+		gateway.WithIdentity(wallet, "appUser"),
+	)
+	if err != nil {
+		return nil, fmt.Errorf("failed to connect to gateway: %v", err)
+	}
+	defer gw.Close()
+
+	network, err := gw.GetNetwork("mychannel")
+	if err != nil {
+		return nil, fmt.Errorf("failed to get network: %v", err)
+	}
+
+	contract := network.GetContract("bigdatacc")
+
+	result, err := contract.SubmitTransaction(function, variableName)
+	if err != nil {
+		return result, fmt.Errorf("failed to Submit transaction: %v", err)
+	}
+	return result, err
+}

high-throughput/application-go/functions/manyUpdates.go

@@ -0,0 +1,86 @@
+/*
+Copyright 2020 IBM All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package functions
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"sync"
+
+	"github.com/hyperledger/fabric-sdk-go/pkg/core/config"
+	"github.com/hyperledger/fabric-sdk-go/pkg/gateway"
+)
+
+// ManyUpdates allows you to push many cuncurrent updates to a variable
+func ManyUpdates(function, variableName, change, sign string) ([]byte, error) {
+
+	err := os.Setenv("DISCOVERY_AS_LOCALHOST", "true")
+	if err != nil {
+		return nil, fmt.Errorf("error setting DISCOVERY_AS_LOCALHOST environemnt variable: %v", err)
+	}
+
+	wallet, err := gateway.NewFileSystemWallet("wallet")
+	if err != nil {
+		return nil, fmt.Errorf("failed to create wallet: %v", err)
+	}
+
+	if !wallet.Exists("appUser") {
+		err := populateWallet(wallet)
+		if err != nil {
+			return nil, fmt.Errorf("failed to populate wallet contents: %v", err)
+		}
+	}
+
+	ccpPath := filepath.Join(
+		"..",
+		"..",
+		"test-network",
+		"organizations",
+		"peerOrganizations",
+		"org1.example.com",
+		"connection-org1.yaml",
+	)
+
+	gw, err := gateway.Connect(
+		gateway.WithConfig(config.FromFile(filepath.Clean(ccpPath))),
+		gateway.WithIdentity(wallet, "appUser"),
+	)
+	if err != nil {
+		return nil, fmt.Errorf("failed to connect to gateway: %v", err)
+	}
+	defer gw.Close()
+
+	network, err := gw.GetNetwork("mychannel")
+	if err != nil {
+		return nil, fmt.Errorf("failed to get network: %v", err)
+	}
+
+	contract := network.GetContract("bigdatacc")
+
+	var wg sync.WaitGroup
+
+	for i := 0; i < 1000; i++ {
+		wg.Add(1)
+		go func() ([]byte, error) {
+			defer wg.Done()
+			result, err := contract.SubmitTransaction(function, variableName, change, sign)
+			if err != nil {
+				return result, fmt.Errorf("failed to evaluate transaction: %v", err)
+			}
+			return result, nil
+		}()
+	}
+
+	wg.Wait()
+
+	result, err := contract.EvaluateTransaction("get", variableName)
+	if err != nil {
+		return nil, fmt.Errorf("failed to evaluate transaction: %v", err)
+	}
+	return result, err
+}