Event Channel Configuration

Defines an event channel that routes events from sources to subscribers. Event channels provide the messaging infrastructure for event-driven integrations.

Schema properties#

YAML examples#

In-memory channel#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: orders-channel
spec:
  Type: memory

NATS channel#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: distributed-events
spec:
  Type: nats
  Parameters:
    Url: nats://nats.example.com:4222
    Subject: weikio.events
    MaxReconnects: "10"
    ReconnectWait: "2"

RabbitMQ channel#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: rabbitmq-events
spec:
  Type: rabbitmq
  Parameters:
    Host: rabbitmq.example.com
    Port: "5672"
    VirtualHost: /
    Username: weikio
    Password: ${rabbitmq-password}
    Exchange: weikio-events
    ExchangeType: topic
    Durable: "true"

Apache Kafka channel#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: kafka-events
spec:
  Type: kafka
  Parameters:
    BootstrapServers: kafka1.example.com:9092,kafka2.example.com:9092
    Topic: weikio-events
    GroupId: weikio-consumers
    AutoOffsetReset: earliest
    EnableAutoCommit: "true"
  Requirements:
    confluent-kafka: ">=2.0.0"

Azure Service Bus channel#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: azure-events
spec:
  Type: azure-servicebus
  Parameters:
    ConnectionString: ${azure-servicebus-connection}
    QueueName: weikio-events
    MaxDeliveryCount: "5"
    LockDuration: "30"

Channel with variable mapping#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: secure-channel
spec:
  Type: rabbitmq
  Parameters:
    Host: ${messaging-host}
    Port: ${messaging-port}
    Username: ${messaging-username}
    Password: ${messaging-password}
    Exchange: events
  Variables:
    - SourceColumn: event_type
      TargetVariable: EventType
    - SourceColumn: event_id
      TargetVariable: EventId

Multiple channels#

apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: local-events
spec:
  Type: memory
  Parameters:
    MaxQueueSize: "1000"
---
apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: distributed-events
spec:
  Type: nats
  Parameters:
    Url: nats://nats.example.com:4222
    Subject: weikio.events
---
apiVersion: weik.io/v1
kind: EventChannel
metadata:
  name: persistent-events
spec:
  Type: rabbitmq
  Parameters:
    Host: rabbitmq.example.com
    Port: "5672"
    Username: weikio
    Password: ${rabbitmq-password}
    Exchange: persistent-events
    Durable: "true"

Channel types#

Memory channel#

In-process channel for single-instance deployments:

Type: memory
Parameters:
  MaxQueueSize: "1000"
  DropOldestOnFull: "false"

Use cases: Development, testing, single-instance production

NATS channel#

Lightweight, high-performance distributed messaging:

Type: nats
Parameters:
  Url: nats://server:4222
  Subject: topic.name
  MaxReconnects: "10"
  ReconnectWait: "2"
  Timeout: "5"

Use cases: Microservices, distributed systems, high-throughput scenarios

RabbitMQ channel#

Full-featured message broker with persistence:

Type: rabbitmq
Parameters:
  Host: server
  Port: "5672"
  VirtualHost: /
  Username: user
  Password: pass
  Exchange: exchange-name
  ExchangeType: topic
  RoutingKey: routing.key
  Durable: "true"
  AutoDelete: "false"

Use cases: Enterprise messaging, guaranteed delivery, complex routing

Kafka channel#

Distributed streaming platform:

Type: kafka
Parameters:
  BootstrapServers: server1:9092,server2:9092
  Topic: topic-name
  GroupId: consumer-group
  AutoOffsetReset: earliest
  EnableAutoCommit: "true"
  SecurityProtocol: PLAINTEXT

Use cases: Event streaming, log aggregation, high-volume data pipelines

Azure Service Bus channel#

Azure cloud messaging service:

Type: azure-servicebus
Parameters:
  ConnectionString: connection-string
  QueueName: queue-name
  MaxDeliveryCount: "5"
  LockDuration: "30"
  RequiresSession: "false"

Use cases: Azure-based integrations, cloud-native applications

Usage notes#

EventChannel configurations define the messaging infrastructure that connects event sources to subscribers. Choose the channel type based on your deployment architecture and requirements.

Channel selection#

Consider these factors:

• Memory: Single instance, no network overhead, no persistence
• NATS: Lightweight, fast, at-most-once delivery by default
• RabbitMQ: Feature-rich, reliable, supports complex routing
• Kafka: High throughput, persistent, supports replay
• Azure Service Bus: Cloud-native, managed service, integrates with Azure

Common parameters#

Most channel types support:

• Connection details (host, port, URL)
• Authentication (username, password, connection strings)
• Topic/queue/subject names
• Durability and persistence settings
• Retry and reconnection settings

Variable mapping#

The Variables array maps event data to named variables:

Variables:
  - SourceColumn: field_name
    TargetVariable: VariableName

Variables are available in subscribing integration flows.

Requirements#

Some channel types require additional Python packages:

Requirements:
  confluent-kafka: ">=2.0.0"
  pika: ">=1.3.0"

Packages are installed when the channel is initialized.

Channel topology#

Design channel topology for your use case:

Fan-out pattern: One source, multiple subscribers

EventSource → EventChannel → Multiple EventSubscriptions

Point-to-point: Direct source to subscriber

EventSource → EventChannel → Single EventSubscription

Hierarchical: Multiple channels for different event types

EventSource → EventChannel (orders)
EventSource → EventChannel (customers)
EventSource → EventChannel (products)

Reliability and durability#

Configure channels for reliability:

Transient (Memory):

• Fast, no overhead
• Lost on restart
• Single instance only

Durable (RabbitMQ, Kafka):

• Survives restarts
• Guaranteed delivery
• Supports clustering

At-least-once delivery:

• Messages delivered, may duplicate
• Implement idempotency in subscribers

At-most-once delivery:

• Fire and forget
• No delivery guarantees

Performance tuning#

Optimize channel performance:

• Set appropriate queue/buffer sizes
• Configure batch sizes
• Adjust timeout values
• Monitor channel backlog
• Scale message brokers as needed

Monitoring#

Monitor channel health:

• Message throughput
• Queue depth
• Delivery failures
• Connection status
• Consumer lag (Kafka)

Security#

Secure channel connections:

• Use authentication (username/password, tokens)
• Enable TLS/SSL for network encryption
• Store credentials in Variables with IsSecret: true
• Restrict network access to message brokers
• Use VPCs or private networks in cloud environments

Best practices#

• Choose channel type based on requirements, not familiarity
• Use memory channels for development and testing
• Use distributed channels (NATS, RabbitMQ, Kafka) for production
• Configure appropriate retention policies
• Implement dead-letter queues for failed messages
• Monitor channel performance and health
• Plan for scalability from the start
• Document channel dependencies
• Test failover scenarios
• Use managed services when possible (Azure Service Bus)

Migration between channel types#

Channels are abstracted, allowing migration:

1. Deploy new channel configuration
2. Create subscriptions on new channel
3. Test with parallel processing
4. Switch sources to new channel
5. Remove old channel when validated

Troubleshooting#

Common issues:

• Connection failures: Verify network connectivity and credentials
• Message loss: Check durability settings and acknowledgment configuration
• High latency: Monitor queue depth and consumer processing time
• Duplicate messages: Implement idempotency in subscribers
• Backpressure: Scale consumers or increase processing capacity