Platform Architecture

Understand Weik.io's platform architecture and core components

Weik.io is a distributed system made up of six components. They handle everything from running your integration code to routing messages and serving the web dashboard. All components communicate with each other through NATS.

Core components

Backend

The Backend is the control center. It does not run the integrations itself; it coordinates the other components. It exposes REST APIs, manages state and configuration, and coordinates the worker agents. When you deploy a new integration, the Backend stores it and determines where it should run.

Agent

The Agent is the worker. It runs Apache Camel flows, handles file transfers with Rclone, and processes events. As it works, it reports its status back to the Backend.

You usually run multiple agents. This lets you distribute the workload, isolate production from development, or place agents geographically closer to the databases they need to reach. If one agent goes down, the others keep running.

UI

The admin interface is a web dashboard. It gives you a visual way to manage integrations, check metrics, debug errors, and configure your API gateway. It communicates exclusively with the Backend via REST APIs.

CLI

The command-line tool (weikio) supports automation. While the UI is suited to monitoring, use the CLI to deploy YAML files, check statuses, and integrate Weik.io into your CI/CD pipelines.

Example CLI commands:

# Deploy an integration
weikio config apply integration.yaml

# Check status
weikio status

# View logs
weikio logs integration-name

APIM (API Management)

The API Management gateway is built on Microsoft YARP (Yet Another Reverse Proxy). When you expose an integration to the outside world, it goes through APIM. It handles the typical gateway tasks: enforcing rate limits, checking authentication, and transforming requests before they reach your integrations.

NATS

NATS is the message broker for the platform. It passes state updates and events between all the Weik.io components. It also handles event streaming for event-driven architectures and provides persistence using JetStream, so messages are not lost if a component restarts.

Configuration

Weik.io uses declarative YAML configuration. You define what you want—your integration flows, event subscriptions, or API endpoints—in a YAML file, and the platform applies it.

apiVersion: weik.io/v1alpha1
kind: Integration
metadata:
  name: example-integration
  description: Example integration

Because configuration is plain text, you can commit it to Git and deploy it via the CLI.

Integration technologies

Weik.io builds on established open-source tools rather than implementing its own integration engines.

Apache Camel

All integrations run on Apache Camel. It provides over 300 pre-built connectors, so you do not have to write custom code to connect to Salesforce, Kafka, AWS S3, or a legacy SAP system. It handles the routing, data transformation, and retry logic.

Learn more about available components in the Apache Camel Components documentation.

Rclone

For moving files between locations, Weik.io uses Rclone. It supports dozens of cloud storage providers, handles bandwidth throttling, and verifies checksums so files are not corrupted in transit. It works equally well for copying a local folder or syncing a large AWS S3 bucket to Azure Blob Storage.

Architecture diagram

Here is how the components fit together:

┌─────────────────────────────────────────────────────────────┐
│                           Users                              │
│                     (Browser/CLI/API)                        │
└──────────────┬──────────────────────┬──────────────────────┘
               │                       │
               ▼                       ▼
        ┌─────────┐             ┌──────────┐
        │   UI    │             │   CLI    │
        │ (Admin) │             │  (Tool)  │
        └────┬────┘             └─────┬────┘
             │                        │
             └────────┬───────────────┘
                      │ REST API
                      ▼
              ┌──────────────┐
              │   Backend    │
              │ (Orchestrator)│
              └───┬──────┬───┘
                  │      │
        ┌─────────┘      └──────────┐
        │                           │
        ▼                           ▼
   ┌────────┐                  ┌────────┐
   │  NATS  │◄────────────────►│  APIM  │
   │ (Msg)  │                  │ (API)  │
   └───┬────┘                  └────────┘
       │
       │ Events/Messages
       │
       ▼
   ┌────────────────┐
   │  Agent(s)      │
   │                │
   │  ┌──────────┐  │
   │  │ Apache   │  │
   │  │ Camel    │  │
   │  └──────────┘  │
   │                │
   │  ┌──────────┐  │
   │  │ RCLONE   │  │
   │  └──────────┘  │
   └────────────────┘

How data flows

Integration execution

When you deploy an integration, the CLI sends the YAML to the Backend. The Backend saves it and broadcasts a notice over NATS. An available Agent picks up the assignment, starts the Apache Camel route, and begins processing data. As it runs, it continuously publishes its health and metrics back to NATS, which the UI reads to render your dashboards.

Event processing

If an event source publishes events to a NATS stream, the Agent processes them and sends the results to destination systems. Status updates flow back through NATS.

API request

If an external client makes an API request, it reaches APIM first. APIM checks the API key and rate limits, then forwards the request to the Agent running your integration. The response flows back through APIM, where transformations are applied before it returns to the client.

Deployment patterns

You can scale Weik.io based on your needs.

Single-node development

For development, you usually run everything on a single machine or inside a devcontainer.

┌────────────────────────────┐
│      Single Machine        │
│                            │
│  ┌──────┐  ┌────────────┐ │
│  │ NATS │  │  Backend   │ │
│  └──────┘  │  + Agent   │ │
│            │  + UI      │ │
│            └────────────┘ │
└────────────────────────────┘

Multi-agent production

In production, you typically separate the Backend and UI onto management servers, while deploying multiple Agents across different regions or networks so they sit close to the systems they integrate with.

┌──────────────┐      ┌──────────────┐
│   Backend    │      │   Agent 1    │
│   + UI       │      │  (Region A)  │
│   + NATS     │◄────►│              │
└──────────────┘      └──────────────┘
       ▲
       │              ┌──────────────┐
       └─────────────►│   Agent 2    │
                      │  (Region B)  │
                      └──────────────┘

High availability

For mission-critical setups, you run a 3-node NATS cluster, redundant Backends, and multiple Agents to eliminate single points of failure. To handle more integrations, add more Agents.

┌──────────────┐      ┌──────────────┐
│  Backend 1   │◄────►│  Backend 2   │
│  + UI        │      │  + UI        │
└──────┬───────┘      └──────┬───────┘
       │                     │
       └──────┬──────────────┘
              ▼
      ┌───────────────┐
      │ NATS Cluster  │
      │  (3 nodes)    │
      └───────┬───────┘
              │
       ┌──────┴──────┐
       ▼             ▼
  ┌─────────┐   ┌─────────┐
  │ Agent 1 │   │ Agent 2 │
  └─────────┘   └─────────┘

Resource requirements

Minimum requirements per component:

Component	CPU	Memory	Storage
Backend	2 cores	2GB	10GB
Agent	2 cores	2GB	20GB
NATS	1 core	1GB	10GB
UI	1 core	512MB	1GB

Production recommendations:

Backend: 4 cores, 4GB RAM
Agent: 4 cores, 4GB RAM (per agent)
NATS: 2 cores, 2GB RAM (3-node cluster)

Next steps

Quickstart - Build your first integration
Integrations Overview - Learn about creating integrations
APIM Overview - Explore API management features