-
Notifications
You must be signed in to change notification settings - Fork 98
Open
Labels
connectorsIssues related to the adapters/connectors crateIssues related to the adapters/connectors crateuser-reportedReported by a user or customerReported by a user or customer
Description
It is currently impossible to guarantee lateness when reading from a multi-partition Kafka topic, since ingest from different partitions can be out of sync by unbounded time range. One solution is to artificially delay reading from a partition if it gets too far ahead of other partitions.
No concrete design yet. One complication is "idle partitions" that stop producing outputs for a long time (or forever).
Related to: #4854
Metadata
Metadata
Assignees
Labels
connectorsIssues related to the adapters/connectors crateIssues related to the adapters/connectors crateuser-reportedReported by a user or customerReported by a user or customer