Weik.io

Streams

Manage durable event streams with NATS JetStream

Streams provide durable, ordered storage for CloudEvents, powered by NATS JetStream. They ensure reliable event delivery and enable event replay capabilities.

What are Streams?

Streams are append-only logs that store CloudEvents persistently. Unlike traditional message queues where messages are deleted after consumption, streams retain CloudEvents for replay and historical analysis.

Key Features

  • Durability - CloudEvents are persisted to disk
  • Ordering - CloudEvents maintain their order
  • Replay - Consumers can replay CloudEvents from any point
  • Retention - Configurable retention policies
  • Multiple Consumers - Many consumers can read from the same stream

Stream Types

Event Source Streams

Automatically created when you configure an Event Source:

  • Stream name: eventsource-{name}-stream
  • Subject: eventsource.{name}
  • Stores CloudEvents from the source

Event Channel Streams

Automatically created when you configure an Event Channel:

  • Stream name: eventchannel-{name}-stream
  • Subject: eventchannel.{name}
  • Stores CloudEvents being sent to the channel

Stream Configuration

Configure stream behavior:

Retention Policy

  • Limits - Keep last N messages
  • Age - Keep messages for N hours/days
  • Size - Keep until stream reaches size limit
  • Interest - Keep until all consumers acknowledge

Storage

  • File - Persistent storage on disk
  • Memory - In-memory storage (faster, not persistent)

Replicas

  • Configure replication for high availability
  • Ensure data durability across nodes

Stream Management

The Streams page allows you to:

  • View all configured streams
  • Monitor stream statistics
  • Configure retention policies
  • Manage stream consumers

Stream Statistics

Monitor stream health:

  • Message Count - Total messages in stream
  • Bytes - Storage used by stream
  • First Sequence - Oldest message sequence number
  • Last Sequence - Newest message sequence number
  • Consumer Count - Number of active consumers

Consumers

Consumers read from streams:

  • Push Consumers - Server pushes messages to consumer
  • Pull Consumers - Consumer pulls messages from server
  • Durable - Consumer state is persisted
  • Ephemeral - Consumer state is temporary

Use Cases

Streams are ideal for:

  • Event Sourcing - Store all state changes as CloudEvents
  • Audit Logging - Maintain immutable audit trails
  • Event Replay - Reprocess CloudEvents for testing or recovery
  • Multi-Consumer - Multiple services processing same CloudEvents

Integration with Other Features

Streams integrate with:

  • Event Sources - Store captured CloudEvents
  • Event Channels - Queue CloudEvents for delivery
  • Event Subscriptions - Route CloudEvents between streams
  • Integration Flows - Trigger flows from stream events

Best Practices

Set Appropriate Retention

  • Balance storage costs with replay needs
  • Use limits or age-based retention
  • Monitor stream growth

Choose Storage Wisely

  • Use file storage for durability
  • Use memory storage for temporary, high-throughput streams

Monitor Stream Health

  • Track message count growth
  • Monitor consumer lag
  • Set up alerts for issues

What’s Next