Weik.io

Integration Flows Overview

Learn about creating and managing Integration Flows in Weik.io

Integration Flows use Apache Camel under the hood to connect systems, move data around, and automate workflows. If you need to glue two APIs together or watch a database for changes, this is where you do it.

What are Integration Flows?

At their core, Integration Flows are Apache Camel routes. They define how your systems talk to each other. A typical flow will:

  • Listen for something to happen (like an HTTP request, a new file, or a timer firing)
  • Do something with the payload (transform it, filter it, or enrich it with other data)
  • Send the result somewhere else

You can write these flows in any DSL Camel supports, but we strongly recommend sticking with YAML. It’s much easier to read and maintain than XML or Java, especially when you’re looking at it six months from now trying to remember how it works.

Key Concepts

Apache Camel

We use Apache Camel as our integration engine. We chose it because:

  • It has over 300 pre-built components. You probably only need three or four for your current project, but it’s nice to know the rest are there when you suddenly have to talk to an AS400 system.
  • It implements standard Enterprise Integration Patterns.
  • It’s incredibly stable. Camel has been running in production environments for over a decade.

Routes

A route is exactly what it sounds like: the path your data takes through an integration. An integration can contain multiple routes, each with its own:

  • Source - Where the data comes from (HTTP endpoint, file system, database)
  • Processing steps - What happens to the data in transit
  • Destination - Where the data ends up

Components

Components are the adapters Camel uses to talk to the outside world. Some of the most common ones include:

  • HTTP/REST APIs
  • Databases (SQL, NoSQL)
  • File systems (FTP, SFTP, cloud storage)
  • Message queues (NATS, Kafka, RabbitMQ)
  • Email (SMTP, IMAP)
  • Cloud services (AWS, Azure, GCP)

If you can think of a system, there’s probably a component for it. You can browse the complete list at Apache Camel Components.

Transformations

Data rarely arrives in the exact format your destination system wants. You’ll use transformations to:

  • Convert between JSON, XML, and CSV
  • Map fields from one schema to another
  • Run custom scripts for weird edge cases

Developer Workflow

The best way to build an integration is to develop and test it locally before pushing it to Weik.io. The basic loop looks like this:

  1. Develop locally - You can write YAML by hand using Camel JBang, or use visual VS Code extensions like Karavan or Kaoto if you prefer a graphical editor.
  2. Store configuration - Put your API keys, database credentials, and other environment-specific values in Weik.io variables so you aren’t hardcoding them.
  3. Reference variables - Point to those variables in your application.properties file (see Using Variables).
  4. Add logging - Drop the weikio-logger kamelet into your routes. You will thank yourself later when something breaks in production and you need to figure out why.
  5. Set requirements - Tell Weik.io where this should run (for example, location=onpremise or environment=linux).
  6. Push to Weik.io - Deploy it with weikio integration push.
  7. Start the flow - Run weikio integration flows start. Weik.io will handle getting it running on the right agents.

Common Integration Patterns

Most integrations we see fall into one of three buckets:

Event-Driven Integrations

These sit idle until something happens, like a webhook firing or a file dropping into a folder.

- from:
    uri: "file:/data/inbox"
    steps:
      - marshal:
          json: {}
      - to:
          uri: "https://api.example.com/data"

This integration watches a directory for new files, converts each file to JSON, and sends it to an API endpoint.

Scheduled Integrations

Execute on a schedule:

  • Periodic data synchronization
  • Batch processing
  • Regular data exports
- from:
    uri: "timer:sync?period=300000"
    steps:
      - to:
          uri: "sql:SELECT * FROM orders WHERE status = 'pending'"
      - marshal:
          json: {}
      - to:
          uri: "https://api.example.com/orders"

This integration runs every 5 minutes (300000ms), fetches pending orders from a database, converts them to JSON, and sends them to an API endpoint.

API-Based Integrations

Expose Integration Flows as REST APIs:

  • Accept HTTP requests
  • Process in real-time
  • Return synchronous responses
- from:
    uri: "platform-http:/api/orders"
    parameters:
      httpMethodRestrict: "POST"
    steps:
      - unmarshal:
          json: {}
      - to:
          uri: "sql:INSERT INTO orders (data) VALUES (:#${body})"
      - setBody:
          constant: '{"status": "created"}'

This integration exposes a REST API endpoint at /api/orders that accepts POST requests, parses JSON payloads, stores them in a database, and returns a success response.

Integration Patterns

Message Translation

Transform formats between systems (XML to JSON, CSV to XML).

Content Enrichment

Enhance data with lookups from databases or APIs.

Content-Based Routing

Route messages to different destinations based on content.

Aggregation

Combine multiple messages into a single message.

What’s Next