Weik.io

Integration Flows Overview

Learn about creating and managing Integration Flows in Weik.io

Integration Flows use Apache Camel to connect systems, move data, and automate workflows. Use them to connect APIs, watch a database for changes, or transfer data between systems.

What are Integration Flows?

Integration Flows are Apache Camel routes. They define how your systems communicate with each other. A typical flow will:

  • Listen for something to happen (like an HTTP request, a new file, or a timer firing)
  • Do something with the payload (transform it, filter it, or enrich it with other data)
  • Send the result somewhere else

You can write these flows in any DSL Camel supports, but YAML is recommended because it is easier to read and maintain than XML or Java.

Key concepts

Apache Camel

Weik.io uses Apache Camel as its integration engine because:

  • It provides over 300 pre-built components for connecting to different systems.
  • It implements standard Enterprise Integration Patterns.
  • It is stable and has been running in production environments for over a decade.

Routes

A route is the path your data takes through an integration. An integration can contain multiple routes, each with its own:

  • Source - Where the data comes from (HTTP endpoint, file system, database)
  • Processing steps - What happens to the data in transit
  • Destination - Where the data ends up

Components

Components are the adapters Camel uses to communicate with external systems. Some of the most common ones include:

  • HTTP/REST APIs
  • Databases (SQL, NoSQL)
  • File systems (FTP, SFTP, cloud storage)
  • Message queues (NATS, Kafka, RabbitMQ)
  • Email (SMTP, IMAP)
  • Cloud services (AWS, Azure, GCP)

You can browse the complete list at Apache Camel Components.

Transformations

Data rarely arrives in the exact format your destination system requires. Use transformations to:

  • Convert between JSON, XML, and CSV
  • Map fields from one schema to another
  • Run custom scripts for special cases

Developer workflow

Develop and test an integration locally before pushing it to Weik.io. The basic workflow is:

  1. Develop locally - Write YAML by hand using Camel JBang, or use visual VS Code extensions like Karavan or Kaoto if you prefer a graphical editor.
  2. Store configuration - Put your API keys, database credentials, and other environment-specific values in Weik.io variables instead of hardcoding them.
  3. Reference variables - Point to those variables in your application.properties file (see Using Variables).
  4. Add logging - Add the weikio-log kamelet to your routes to make troubleshooting easier in production.
  5. Set requirements - Tell Weik.io where the integration should run (for example, location=onpremise or environment=linux).
  6. Push to Weik.io - Deploy it with weikio integration push.
  7. Start the flow - Run weikio integration flows start. Weik.io handles running it on the right agents.

Common integration patterns

Most integrations fall into one of three categories:

Event-driven integrations

These run when an event occurs, such as a webhook firing or a file arriving in a folder.

- from:
    uri: "file:/data/inbox"
    steps:
      - marshal:
          json: {}
      - to:
          uri: "https://api.example.com/data"

This integration watches a directory for new files, converts each file to JSON, and sends it to an API endpoint.

Scheduled integrations

Execute on a schedule:

  • Periodic data synchronization
  • Batch processing
  • Regular data exports
- from:
    uri: "timer:sync?period=300000"
    steps:
      - to:
          uri: "sql:SELECT * FROM orders WHERE status = 'pending'"
      - marshal:
          json: {}
      - to:
          uri: "https://api.example.com/orders"

This integration runs every 5 minutes (300000ms), fetches pending orders from a database, converts them to JSON, and sends them to an API endpoint.

API-based integrations

Expose Integration Flows as REST APIs:

  • Accept HTTP requests
  • Process in real-time
  • Return synchronous responses
- from:
    uri: "platform-http:/api/orders"
    parameters:
      httpMethodRestrict: "POST"
    steps:
      - unmarshal:
          json: {}
      - to:
          uri: "sql:INSERT INTO orders (data) VALUES (:#${body})"
      - setBody:
          constant: '{"status": "created"}'

This integration exposes a REST API endpoint at /api/orders that accepts POST requests, parses JSON payloads, stores them in a database, and returns a success response.

Integration patterns

Message translation

Transform formats between systems (XML to JSON, CSV to XML).

Content enrichment

Enhance data with lookups from databases or APIs.

Content-based routing

Route messages to different destinations based on content.

Aggregation

Combine multiple messages into a single message.

Next steps