Skip to content

Add enhanced options and documentation for streaming data generation #115

@ronanstokes-db

Description

@ronanstokes-db

Issue to track:

1 - changes to streaming documentation to add
a) Delta Live Tables integration information,
b) example of generation of sliding event time windows
c) example of generation of simple IOT data with timestamps
d) example of generation of late arriving data with the above

2 - add options to simplify generation of the above
The above behavior is supported in the current version, but it would be useful to include options to simplify the generation of streaming data sets

2a - add options for
- ageLimit - ignore messages older than n seconds. This helps when benchmarking with trigger once using a rate
stream when significant time has elapsed between runs (otherwise there can be a large message backlog)
3 - support rate-micro-batch source

Metadata

Metadata

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions