Merge branch 'master' into mobile-push-ga

Author: pwseg

src/_includes/content/functions/logs.md

@@ -1,6 +1,6 @@
 If your function throws an error, execution halts immediately. Segment captures the event, any outgoing requests/responses, any logs the function might have printed, as well as the error itself.
 
-Segment then displays the captured error information in the [Event Delivery](/docs/connections/event-delivery/) page for your function. You can use this information to find and fix unexpected errors.
+Segment then displays the captured error information in the [Event Delivery](/docs/connections/event-delivery/) page for your destination. You can use this information to find and fix unexpected errors.
 
 You can throw [an error or a custom error](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error){:target="_blank"} and you can also add helpful context in logs using the [`console` API](https://developer.mozilla.org/en-US/docs/Web/API/console){:target="_blank"}. For example:
 

src/connections/functions/destination-functions.md

@@ -140,7 +140,7 @@ If your function fails, you can check the error details and logs in the **Output
 Batch handlers are an extension of destination functions. When you define an `onBatch` handler alongside the handler functions for single events (for example: `onTrack` or `onIdentity`), you're telling Segment that the destination function can accept and handle batches of events.
 
 > info ""
-> Batching is available for destination functions only.
+> Batching is available for destination and destination insert functions only.
 
 ### When to use batching
 
@@ -269,7 +269,7 @@ To test the batch handler:
 2. Add events as a JSON array, with one event per element.
 3. Click **Run** to preview the batch handler with the specified events.
 
-> note ""
+> info ""
 > The Sample Event option tests single events only. You must use Manual Mode to add more than one event so you can test batch handlers.
 
 The editor displays logs and request traces from the batch handler.
@@ -306,7 +306,7 @@ Standard [function error types](/docs/connections/functions/destination-function
 ]
 ```
 
-After receiving the response from the `onBatch` handler, Segment only retries **event_4** and **event_5**.
+For example, after receiving the responses above from the `onBatch` handler, Segment only retries **event_4** and **event_5**.
 
 | Error Type             | Result  |
 | ---------------------- | ------- |

src/connections/functions/index.md

@@ -35,9 +35,6 @@ Learn more about [destination functions](/docs/connections/functions/destination
 #### Destination insert functions
 Destination insert functions help you enrich your data with code before you send it to downstream destinations.  
 
-> info "Destination Insert Functions in Public Beta"
-> Destination Insert Functions is in Public Beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
-
 Use cases: 
 - Implement custom logic and enrich data with third party sources 
 - Transform outgoing data with advanced filtration and computation

src/connections/functions/insert-functions.md

@@ -1,6 +1,5 @@
 ---
 title: Destination Insert Functions
-beta: true
 
 ---
 
@@ -14,9 +13,6 @@ Use Destination Insert Functions to enrich, transform, or filter your data befor
 
 **Customize filtration for your destinations**: Create custom logic with nested if-else statements, regex, custom business rules, and more to filter event data.
 
-> info "Destination Insert Functions in Public Beta"
-> Destination Insert Functions is in Public Beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
-
 
 ## Create destination insert functions
 
@@ -237,21 +233,181 @@ To enable your insert function:
 
 To prevent your insert function from processing data, toggle Enable Function off.
 
-{% comment %}
-## Batching the insert function
-
-Insert functions support batching with the `onBatch` handler. 
+## Batching the destination insert function
 
 Batch handlers are an extension of insert functions. When you define an `onBatch` handler alongside the handler functions for single events (for example, `onTrack` or `onIdentity`), you're telling Segment that the insert function can accept and handle batches of events. 
 
-Note the following limitations for batching with insert functions:
-- The batch request and response size is limited to 6mb.
-- Max count begins with 100 and goes up to 1,000.
+> info ""
+> Batching is available for destination and destination insert functions only. 
+
+### When to use batching
+
+Consider creating a batch handler if:
+
+- **You have a high-throughput function and want to reduce cost.** When you define a batch handler, Segment invokes the function once per *batch*, rather than once per event. As long as the function’s execution time isn’t adversely affected, the reduction in invocations should lead to a reduction in cost.
+
+- **Your destination supports batching**. When your downstream destination supports sending data downstream in batches you can define a batch handler to avoid throttling. Batching for functions is independent of batch size supported by the destination. Segment automatically handles batch formation for destinations.
 
 > info ""
-> Batching is available for insert and destination functions only. Read more about batching [in the Destination Functions docs](/docs/connections/functions/destination-functions/#batching-the-destination-function). 
+> If a batched function receives too low a volume of events (under one event per second) to be worth batching, Segment may not invoke the batch handler.
+
+### Define the batch handler
+
+Segment collects the events over a short period of time and combines them into a batch. The system flushes them when the batch reaches a certain number of events, or when the batch has been waiting for a specified wait time.
+
+To create a batch handler, define an `onBatch` function within your destination insert function. You can also use the "Default Batch" template found in the Functions editor to get started quickly.
+
+```js
+async function onBatch(events, settings){
+  // handle the batch of events
+  return events
+}
+```
+
+> info ""
+> The `onBatch` handler is an optional extension. Destination insert functions must still contain single event handlers as a fallback, in cases where Segment doesn't receive enough events to execute the batch.
+
+The handler function receives an array of events. The events can be of any supported type and a single batch may contain more than one event type. Handler functions can also receive function settings. Here is an example of what a batch can look like:
+
+```json
+[
+    {
+      "type": "identify",
+      "userId": "019mr8mf4r",
+      "traits": {
+        "email": "[email protected]",
+        "name": "Jake Peterson",
+        "age": 26
+      }
+    },
+    {
+      "type": "track",
+      "userId": "019mr8mf4r",
+      "event": "Song Played",
+      "properties": {
+        "name": "Fallin for You",
+        "artist": "Dierks Bentley"
+      }
+    },
+    {
+      "type": "track",
+      "userId": "971mj8mk7p",
+      "event": "Song Played",
+      "properties": {
+        "name": "Get Right",
+        "artist": "Jennifer Lopez"
+      }
+    }
+]
+```
+
+### Configure the event types within a batch
+
+Segment batches together any event _of any type_ that it sees over a short period of time to increase batching efficiency and give you the flexibility to decide how batches are created. If you want to split batches by event type, you can implement this in your functions code by writing a handler.
+
+```js
+async function onBatch(events, settings) {
+  // group events by type
+  const eventsByType = {}
+  for (const event of events) {
+    if (!(event.type in eventsByType)) {
+      eventsByType[event.type] = []
+    }
+    eventsByType[event.type].push(event)
+  }
+
+  // concurrently process sub-batches of a specific event type
+  const promises = Object.entries(eventsByType).map(([type, events]) => {
+    switch (type) {
+    case 'track':
+      return onTrackBatch(events, settings)
+    case 'identify':
+      return onIdentifyBatch(events, settings)
+    // ...handle other event types here...
+    }
+  })
+  try {
+    const results = await Promise.all(promises);
+    const batchResult = [].concat(...results); // Combine arrays into a single array
+    return batchResult;
+  } catch (error) {
+    throw new RetryError(error.message);
+  }
+}
+
+async function onTrackBatch(events, settings) {
+  // handle a batch of track events
+  return events
+}
+
+async function onIdentifyBatch(events, settings) {
+  // handle a batch of identify events
+  return events
+}
+```
+
+### Configure your batch parameters
+ 
+By default, Functions waits up to 10 seconds to form a batch of 20 events. You can increase the number of events included in each batch (up to 400 events per batch) by contacting [Segment support](https://segment.com/help/contact/){:target="_blank"}. Segment recommends users who wish to include fewer than 20 events per batch use destination insert functions without the `onBatch` handler.
+
+### Test the batch handler
+
+The [Functions editing environment](/docs/connections/functions/environment/) supports testing batch handlers.
+
+To test the batch handler:
+1. In the right panel of the Functions editor, click **customize the event yourself** to enter Manual Mode.
+2. Add events as a JSON array, with one event per element.
+3. Click **Run** to preview the batch handler with the specified events.
+
+> info ""
+> The Sample Event option tests single events only. You must use Manual Mode to add more than one event so you can test batch handlers.
+
+The editor displays logs and request traces from the batch handler.
+
+The [Public API](/docs/api/public-api) Functions/Preview endpoint also supports testing batch handlers. The payload must be a batch of events as a JSON array.
+
+
+### Handling batching errors
+
+Standard [function error types](/docs/connections/functions/destination-functions/#destination-functions-error-types) apply to batch handlers. Segment attempts to retry the batch in the case of Timeout or Retry errors. For all other error types, Segment discards the batch. It's also possible to report a partial failure by returning status of each event in the batch. Segment retries only the failed events in a batch until those events are successful or until they result in a permanent error.
+
+```json
+[
+	{
+		"status": 200
+	},
+	{
+		"status": 400,
+		"errormessage": "Bad Request"
+	},
+	{
+		"status": 200
+	},
+	{
+		"status": 500,
+		"errormessage": "Error processing request"
+	},
+	{
+		"status": 500,
+		"errormessage": "Error processing request"
+	},
+	{
+		"status": 200
+	},
+]
+```
+
+For example, after receiving the responses above from the `onBatch` handler, Segment only retries **event_4** and **event_5**.
+
+| Error Type             | Result  |
+| ---------------------- | ------- |
+| Bad Request            | Discard |
+| Invalid Settings       | Discard |
+| Message Rejected       | Discard |
+| RetryError             | Retry   |
+| Timeout                | Retry   |
+| Unsupported Event Type | Discard |
 
-{% endcomment %}
 
 {% comment %}
 
@@ -321,11 +477,6 @@ No, destination insert functions are currently available as cloud-mode destinati
 
 No, an insert function can only be connected to one destination.
 
-##### How do I publish a destination to the public Segment catalog?
-
-If you are a partner, looking to publish your destination and distribute your app through Segment catalog, visit the [Developer Center](https://segment.com/partners/developer-center/){:target="_blank"} and check out the Segment [partner docs](/docs/partners/).
-
-
 {% comment %}
 
 ## Using Segment's Public API