Binding quickstart - Cron input PostgreSQL output (#669)

* Binding quickstart - Cron input PostgreSQL output
Signed-off-by: Nir Mashkowski <[email protected]>

* Updated to properly load from APP_PORT and DAPR_HTTP_PORT, or default value.

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adds Dockerfile and docker compose file to initialize orders table in postgress in an automated way.

Signed-off-by: Paul Yuknewicz <[email protected]>

* Moving database files to db folder and add README

Signed-off-by: Paul Yuknewicz <[email protected]>

* readme updates

Signed-off-by: Paul Yuknewicz <[email protected]>

* Cleaning up bindings file names

Signed-off-by: Paul Yuknewicz <[email protected]>

* Fixing crash caused by moving orders.json.  keeping in root folder.

Signed-off-by: Paul Yuknewicz <[email protected]>

* Tuning names and flake8 lint pass

Signed-off-by: Paul Yuknewicz <[email protected]>

* Complete code and lint review of Python examples

Signed-off-by: Paul Yuknewicz <[email protected]>

* updating comments with `dapr run ..` that works

Signed-off-by: Paul Yuknewicz <[email protected]>

* Lowercase sqldb binding; fixing request formatting and error handling

Signed-off-by: Paul Yuknewicz <[email protected]>

* removed grpc dependency and use flask for routes

Signed-off-by: Paul Yuknewicz <[email protected]>

* Fixed Axios issues with incorrect playload

Signed-off-by: Paul Yuknewicz <[email protected]>

* fixed env vars and var groups

Signed-off-by: Paul Yuknewicz <[email protected]>

* Fixed forced exit, and comments

Signed-off-by: Paul Yuknewicz <[email protected]>

* Go changes for styling and cont. event handling

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adding READMEs and MMD tests for Python

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adding C# README files and MMD tests

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adding Go README files and MMD tests

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adding Javascript README files and MMD

Signed-off-by: Paul Yuknewicz <[email protected]>

* Renamed `app` files in each language to be consistent with idiom.  e.g. program.cs for C#, index.js for Javascript, and app.python for Python.  Per review feedback from @amulyavarote

Signed-off-by: Paul Yuknewicz <[email protected]>

* updating server port to be string

Signed-off-by: Paul Yuknewicz <[email protected]>

* Changing folder structure for consistency. Nesting language source files in `./batch` for http and sdk variations. Fixing up code and readmes to match.

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adding makefiles for validate (MMD test) and docker (inert)

Signed-off-by: Paul Yuknewicz <[email protected]>

* Adjusting MMD with escape chars and sleeps

Signed-off-by: Paul Yuknewicz <[email protected]>

* Fixed make validate (match, timeouts, sleeps) for C# and Go

Signed-off-by: Paul Yuknewicz <[email protected]>

* Updated MMD with precision fixes.  All passing!

Signed-off-by: Paul Yuknewicz <[email protected]>

* change for java maven builds to build clean

Signed-off-by: Paul Yuknewicz <[email protected]>

* WIP on Java

Signed-off-by: Paul Yuknewicz <[email protected]>

* Java SDK is done!

Signed-off-by: Paul Yuknewicz <[email protected]>

* Java signed off!

Signed-off-by: Paul Yuknewicz <[email protected]>

* pretty formatted POM and java so better with merges in future

Signed-off-by: Paul Yuknewicz <[email protected]>

* Update bindings/javascript/sdk/README.md

Accepted

Co-authored-by: greenie-msft <[email protected]>

Co-authored-by: Paul Yuknewicz <[email protected]>
Co-authored-by: Mark Fussell <[email protected]>
Co-authored-by: greenie-msft <[email protected]>

bindings/components/binding-cron.yaml

@@ -0,0 +1,11 @@
+apiVersion: dapr.io/v1alpha1
+kind: Component
+metadata:
+  name: cron
+  namespace: quickstarts
+spec:
+  type: bindings.cron
+  version: v1
+  metadata:
+  - name: schedule
+    value: "@every 10s" # valid cron schedule

bindings/components/binding-postgres.yaml

@@ -0,0 +1,11 @@
+apiVersion: dapr.io/v1alpha1
+kind: Component
+metadata:
+  name: sqldb
+  namespace: quickstarts
+spec:
+  type: bindings.postgres
+  version: v1
+  metadata:
+  - name: url # Required
+    value: "user=postgres password=docker host=localhost port=5432 dbname=orders pool_min_conns=1 pool_max_conns=10"

bindings/csharp/http/README.md

@@ -0,0 +1,67 @@
+# Dapr Bindings (HTTP)
+
+In this quickstart, you'll create a microservice to demonstrate Dapr's bindings API to work with external systems as inputs and outputs. The service listens to input binding events from a system CRON and then outputs the contents of local data to a PostreSql output binding. 
+
+Visit [this](https://docs.dapr.io/developing-applications/building-blocks/bindings/) link for more information about Dapr and Bindings.
+
+> **Note:** This example leverages only HTTP REST.  If you are looking for the example using the Dapr SDK [click here](../sdk).
+
+This quickstart includes one service:
+
+- .NET/C# service `batch`
+
+### Run and initialize PostgreSQL container
+
+1. Open a new terminal, change directories to `../../../db`, and run the container with [Docker Compose](https://docs.docker.com/compose/): 
+
+<!-- STEP
+name: Run and initialize PostgreSQL container
+expected_return_code:
+background: true
+sleep: 5
+timeout_seconds: 6
+-->
+
+```bash
+cd ../../db
+docker compose up
+```
+
+<!-- END_STEP -->
+
+### Run C# service with Dapr
+
+2. Open a new terminal window, change directories to `./batch` in the quickstart directory and run: 
+
+<!-- STEP
+name: Install C# dependencies
+-->
+
+```bash
+cd ./batch
+ls
+dotnet restore
+```
+
+<!-- END_STEP -->
+3. Run the C# service app with Dapr: 
+
+<!-- STEP
+name: Run csharp-quickstart-binding-http service
+working_dir: ./batch
+expected_stdout_lines:
+  - '== APP == insert into orders (orderid, customer, price) values (1, ''John Smith'', 100.32)'
+  - '== APP == insert into orders (orderid, customer, price) values (2, ''Jane Bond'', 15.4)'
+  - '== APP == insert into orders (orderid, customer, price) values (3, ''Tony James'', 35.56)'
+  - '== APP == Finished processing batch'
+expected_stderr_lines:
+output_match_mode: substring
+sleep: 11
+timeout_seconds: 30
+-->
+
+```bash
+dapr run --app-id csharp-quickstart-binding-http --app-port 7001 --components-path ../../../components -- dotnet run
+```
+
+<!-- END_STEP -->

bindings/csharp/http/batch/Debug/net6.0/appsettings.Development.json

@@ -0,0 +1,9 @@
+{
+    "Logging": {
+      "LogLevel": {
+        "Default": "Information",
+        "Microsoft": "Warning",
+        "Microsoft.Hosting.Lifetime": "Information"
+      }
+    }
+  }

bindings/csharp/http/batch/Debug/net6.0/appsettings.json

@@ -0,0 +1,10 @@
+{
+    "Logging": {
+      "LogLevel": {
+        "Default": "Information",
+        "Microsoft": "Warning",
+        "Microsoft.Hosting.Lifetime": "Information"
+      }
+    },
+    "AllowedHosts": "*"
+  }

bindings/csharp/http/batch/Debug/net6.0/batch.deps.json

@@ -0,0 +1,115 @@
+{
+  "runtimeTarget": {
+    "name": ".NETCoreApp,Version=v6.0",
+    "signature": ""
+  },
+  "compilationOptions": {},
+  "targets": {
+    ".NETCoreApp,Version=v6.0": {
+      "batch/1.0.0": {
+        "dependencies": {
+          "Swashbuckle.AspNetCore": "6.2.3"
+        },
+        "runtime": {
+          "batch.dll": {}
+        }
+      },
+      "Microsoft.Extensions.ApiDescription.Server/3.0.0": {},
+      "Microsoft.OpenApi/1.2.3": {
+        "runtime": {
+          "lib/netstandard2.0/Microsoft.OpenApi.dll": {
+            "assemblyVersion": "1.2.3.0",
+            "fileVersion": "1.2.3.0"
+          }
+        }
+      },
+      "Swashbuckle.AspNetCore/6.2.3": {
+        "dependencies": {
+          "Microsoft.Extensions.ApiDescription.Server": "3.0.0",
+          "Swashbuckle.AspNetCore.Swagger": "6.2.3",
+          "Swashbuckle.AspNetCore.SwaggerGen": "6.2.3",
+          "Swashbuckle.AspNetCore.SwaggerUI": "6.2.3"
+        }
+      },
+      "Swashbuckle.AspNetCore.Swagger/6.2.3": {
+        "dependencies": {
+          "Microsoft.OpenApi": "1.2.3"
+        },
+        "runtime": {
+          "lib/net6.0/Swashbuckle.AspNetCore.Swagger.dll": {
+            "assemblyVersion": "6.2.3.0",
+            "fileVersion": "6.2.3.0"
+          }
+        }
+      },
+      "Swashbuckle.AspNetCore.SwaggerGen/6.2.3": {
+        "dependencies": {
+          "Swashbuckle.AspNetCore.Swagger": "6.2.3"
+        },
+        "runtime": {
+          "lib/net6.0/Swashbuckle.AspNetCore.SwaggerGen.dll": {
+            "assemblyVersion": "6.2.3.0",
+            "fileVersion": "6.2.3.0"
+          }
+        }
+      },
+      "Swashbuckle.AspNetCore.SwaggerUI/6.2.3": {
+        "runtime": {
+          "lib/net6.0/Swashbuckle.AspNetCore.SwaggerUI.dll": {
+            "assemblyVersion": "6.2.3.0",
+            "fileVersion": "6.2.3.0"
+          }
+        }
+      }
+    }
+  },
+  "libraries": {
+    "batch/1.0.0": {
+      "type": "project",
+      "serviceable": false,
+      "sha512": ""
+    },
+    "Microsoft.Extensions.ApiDescription.Server/3.0.0": {
+      "type": "package",
+      "serviceable": true,
+      "sha512": "sha512-LH4OE/76F6sOCslif7+Xh3fS/wUUrE5ryeXAMcoCnuwOQGT5Smw0p57IgDh/pHgHaGz/e+AmEQb7pRgb++wt0w==",
+      "path": "microsoft.extensions.apidescription.server/3.0.0",
+      "hashPath": "microsoft.extensions.apidescription.server.3.0.0.nupkg.sha512"
+    },
+    "Microsoft.OpenApi/1.2.3": {
+      "type": "package",
+      "serviceable": true,
+      "sha512": "sha512-Nug3rO+7Kl5/SBAadzSMAVgqDlfGjJZ0GenQrLywJ84XGKO0uRqkunz5Wyl0SDwcR71bAATXvSdbdzPrYRYKGw==",
+      "path": "microsoft.openapi/1.2.3",
+      "hashPath": "microsoft.openapi.1.2.3.nupkg.sha512"
+    },
+    "Swashbuckle.AspNetCore/6.2.3": {
+      "type": "package",
+      "serviceable": true,
+      "sha512": "sha512-cnzQDn0Le+hInsw2SYwlOhOCPXpYi/szcvnyqZJ12v+QyrLBwAmWXBg6RIyHB18s/mLeywC+Rg2O9ndz0IUNYQ==",
+      "path": "swashbuckle.aspnetcore/6.2.3",
+      "hashPath": "swashbuckle.aspnetcore.6.2.3.nupkg.sha512"
+    },
+    "Swashbuckle.AspNetCore.Swagger/6.2.3": {
+      "type": "package",
+      "serviceable": true,
+      "sha512": "sha512-qOF7j1sL0bWm8g/qqHVPCvkO3JlVvUIB8WfC98kSh6BT5y5DAnBNctfac7XR5EZf+eD7/WasvANncTqwZYfmWQ==",
+      "path": "swashbuckle.aspnetcore.swagger/6.2.3",
+      "hashPath": "swashbuckle.aspnetcore.swagger.6.2.3.nupkg.sha512"
+    },
+    "Swashbuckle.AspNetCore.SwaggerGen/6.2.3": {
+      "type": "package",
+      "serviceable": true,
+      "sha512": "sha512-+Xq7WdMCCfcXlnbLJVFNgY8ITdP2TRYIlpbt6IKzDw5FwFxdi9lBfNDtcT+/wkKwX70iBBFmXldnnd02/VO72A==",
+      "path": "swashbuckle.aspnetcore.swaggergen/6.2.3",
+      "hashPath": "swashbuckle.aspnetcore.swaggergen.6.2.3.nupkg.sha512"
+    },
+    "Swashbuckle.AspNetCore.SwaggerUI/6.2.3": {
+      "type": "package",
+      "serviceable": true,
+      "sha512": "sha512-bCRI87uKJVb4G+KURWm8LQrL64St04dEFZcF6gIM67Zc0Sr/N47EO83ybLMYOvfNdO1DCv8xwPcrz9J/VEhQ5g==",
+      "path": "swashbuckle.aspnetcore.swaggerui/6.2.3",
+      "hashPath": "swashbuckle.aspnetcore.swaggerui.6.2.3.nupkg.sha512"
+    }
+  }
+}

bindings/csharp/http/batch/Debug/net6.0/batch.runtimeconfig.json

@@ -0,0 +1,19 @@
+{
+  "runtimeOptions": {
+    "tfm": "net6.0",
+    "frameworks": [
+      {
+        "name": "Microsoft.NETCore.App",
+        "version": "6.0.0"
+      },
+      {
+        "name": "Microsoft.AspNetCore.App",
+        "version": "6.0.0"
+      }
+    ],
+    "configProperties": {
+      "System.GC.Server": true,
+      "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization": false
+    }
+  }
+}

bindings/csharp/http/batch/Properties/launchSettings.json

@@ -0,0 +1,15 @@
+{
+    "$schema": "http://json.schemastore.org/launchsettings.json",
+    "profiles": {
+       "CheckoutService": {
+        "commandName": "Project",
+        "dotnetRunMessages": "true",
+        "launchBrowser": true,
+        "launchUrl": "swagger",
+        "applicationUrl": "http://localhost:7001",
+        "environmentVariables": {
+          "ASPNETCORE_ENVIRONMENT": "Development"
+        }
+      }
+    }
+  }

bindings/csharp/http/batch/appsettings.Development.json

@@ -0,0 +1,9 @@
+{
+    "Logging": {
+      "LogLevel": {
+        "Default": "Information",
+        "Microsoft": "Warning",
+        "Microsoft.Hosting.Lifetime": "Information"
+      }
+    }
+  }

bindings/csharp/http/batch/appsettings.json

@@ -0,0 +1,10 @@
+{
+    "Logging": {
+      "LogLevel": {
+        "Default": "Information",
+        "Microsoft": "Warning",
+        "Microsoft.Hosting.Lifetime": "Information"
+      }
+    },
+    "AllowedHosts": "*"
+  }

bindings/csharp/http/batch/batch.csproj

@@ -0,0 +1,14 @@
+<Project Sdk="Microsoft.NET.Sdk.Web">
+
+    <PropertyGroup>
+      <OutputType>Exe</OutputType>
+      <TargetFramework>net6.0</TargetFramework>
+      <Nullable>enable</Nullable>
+      <ImplicitUsings>enable</ImplicitUsings>
+    </PropertyGroup>
+
+    <ItemGroup>
+      <PackageReference Include="Swashbuckle.AspNetCore" Version="6.2.3" />
+    </ItemGroup>
+
+  </Project>

bindings/csharp/http/batch/program.cs

@@ -0,0 +1,74 @@
+/*
+Copyright 2021 The Dapr Authors
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+     http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+using System;
+using System.IO;
+using System.Text;
+using System.Text.Json;
+using System.Text.Json.Serialization;
+using Microsoft.AspNetCore.Mvc;
+
+
+//dapr run --app-id csharp-quickstart-binding-http --app-port 7001 --components-path ../../../components -- dotnet run
+
+var cronBindingName = "cron";
+var sqlBindingName = "sqldb";
+
+var baseURL = Environment.GetEnvironmentVariable("BASE_URL") ?? "http://localhost";
+var daprPort = Environment.GetEnvironmentVariable("DAPR_HTTP_PORT") ?? "3500";
+var daprUrl = $"{baseURL}:{daprPort}/v1.0/bindings/{sqlBindingName}";
+
+var builder = WebApplication.CreateBuilder(args);
+var app = builder.Build();
+
+if (app.Environment.IsDevelopment()) {app.UseDeveloperExceptionPage();}
+
+var httpClient = new HttpClient();
+httpClient.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
+
+// Triggered by Dapr input binding
+app.MapPost("/" + cronBindingName, async () => {
+     Console.WriteLine("Processing batch..");
+
+     string jsonFile = File.ReadAllText("../../../orders.json");
+     var ordersArray = JsonSerializer.Deserialize<Orders>(jsonFile);
+     foreach(Order ord in ordersArray?.orders ?? new Order[] {}){
+          var sqlText = $"insert into orders (orderid, customer, price) values ({ord.OrderId}, '{ord.Customer}', {ord.Price});";
+          var payload = new DaprPayload(sql: new DaprPostgresBindingMetadata(cmd: sqlText), operation: "exec");
+          var orderJson = JsonSerializer.Serialize<DaprPayload>(payload);
+          var content = new StringContent(orderJson, Encoding.UTF8, "application/json");
+
+          Console.WriteLine(sqlText);
+
+          // Insert order using Dapr output binding via HTTP Post
+          try {
+               var resp = await httpClient.PostAsync(daprUrl, content);
+               resp.EnsureSuccessStatusCode();
+          } 
+          catch (HttpRequestException e) {
+               Console.WriteLine(e.ToString());
+               throw e;
+          }
+
+     }
+
+     Console.WriteLine("Finished processing batch");
+
+     return Results.Ok();
+});
+
+await app.RunAsync();
+
+public record DaprPostgresBindingMetadata([property: JsonPropertyName("sql")] string cmd);
+public record DaprPayload([property: JsonPropertyName("metadata")] DaprPostgresBindingMetadata sql, [property: JsonPropertyName("operation")] string operation);
+public record Order([property: JsonPropertyName("orderid")] int OrderId, [property: JsonPropertyName("customer")] string Customer, [property: JsonPropertyName("price")] float Price);
+public record Orders([property: JsonPropertyName("orders")] Order[] orders);

bindings/csharp/http/makefile

@@ -0,0 +1,2 @@
+include ../../../docker.mk
+include ../../../validate.mk