Skip to content

Fundamentals

Jump to bottom
Charles d'Avernas edited this page Jul 31, 2023 · 5 revisions

Cloud Events

A CloudEvent is a standardized format for representing events in cloud environments. It serves as a common data structure that encapsulates information about an event, such as its type, source, data, and other metadata. The adoption of CloudEvents fosters interoperability among various cloud services and systems, enabling seamless event exchange and integration.

CloudStreams ingests and dispatches CloudEvents using the Structured Content Mode over the HTTP Protocol Binding.

To publish CloudEvents to CloudStreams, perform a valid CloudEvent HTTP POST request to a CloudStreams Gateway's ingestion endpoint, typically located at /api/gateway/v1/cloud-events/pub.

Sample: Sending a CloudEvent to CloudStreams 
curl -XPOST -H "Content-type: application/cloudevents+json" -d '{
    "id": "123",
    "source": "https://cloud-streams.io",
    "type": "io.cloud-streams",
    "specversion": "1.1",
    "datacontenttype": "application/json",
    "subject": "foobar",
    "data":{
        "foo": "bar"
    }
}' '/api/gateway/v1/cloud-events/pub'

Streams

A stream is an ordered, append-only serie of consumed CloudEvents.

Streams can be read forward or backward, starting from the start of the stream (0), from a specific offset or from the end of the stream (-1).

CloudStreams stores CloudEvents into a single stream, ensuring they are securely captured and recorded, while providing comprehensive auditing capabilities. Additionally, the use of partitions within the single stream enables efficient event reading and querying based on different attributes such as source, type, subject, correlationId, or causationId (see: Partitions). This design empowers users to access specific sets of events with ease, facilitating streamlined event processing while maintaining full visibility into the complete event flow for auditing purposes.

Partitions

Upon ingestion, CloudStreams store CloudEvents in a global stream, ensuring reliable event capture, before automatically partitioning them based on attributes like source, type, subject, and optionally, correlation ID and causation ID.

Each partition represents a stream of references containing CloudEvents with shared attribute values, streamlining handling and enabling both scalable and responsive event-driven architectures.

CloudStreams partitions ingested cloud events into up to 5 different partition types:

  • by-source: a dedicated stream that allows consumers to read all events with a particular source context attribute.
  • by-type: a dedicated stream that allows consumers to read all events with a particular type context attribute.
  • by-subject: a dedicated stream that allows consumers to read all events with a particular subject context attribute.
  • by-correlation-id: a dedicated stream that allows consumers to read all events with a particular $correlationId metadata value.
  • by-causation-id: a dedicated stream that allows consumers to read all events with a particular $causationId metadata value.

Partitions are identified by a type (ex: by-source) and by an id, which is the value of the attribute or metadata the CloudEvents have been seggregated by (ex: https://my-fake-event-source.com).

Subscriptions

To subscribe to cloud events ingested by CloudStreams, you need to create a new subscription resource using the API Server or the Dashboard.

The following represents the subscription resource used to subscribe to all cloud events ingested by CloudStreams, and to dispatch them, at a rate of 1 event every 4 seconds, to an hypothetical webhook.site consumer:

apiVersion: cloud-streams.io/v1
kind: Subscription
metadata:
  name: subscription-1
spec:
  subscriber:
    uri: https://webhook.site/...
    rateLimit: 0.25
Clone this wiki locally