Weik.io

MFT Setup

Setting up CoreSystems and MFT definitions

Set up Managed File Transfer by creating CoreSystem resources for storage endpoints and MFT definitions for transfer operations.

Creating CoreSystems

CoreSystems define storage backends used as source and destination endpoints in file transfers. Each CoreSystem is configured with rclone-compatible parameters.

Azure Blob Storage

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: weikio_blob
  description: "Weikio Azure Blob"
spec:
  category: storage
  type: azureblob
  title: "Weikio Azure Blob"
  parameters:
    account: weikio
    key: "{{azure:fabric-azure-weikiostorage}}"

SMB File Share

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: company_smb
  description: Company SMB Share
spec:
  category: storage
  type: smb
  title: "Company SMB"
  parameters:
    host: fileserver.company.local
    user: integrations
    pass: '{{azure:smb-password}}'
    domain: company.local

SFTP Server

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: partner_sftp
  description: Partner SFTP Server
spec:
  category: storage
  type: sftp
  title: "Partner SFTP"
  parameters:
    host: sftp.partner.com
    user: integration_user
    pass: '{{azure:sftp-password}}'
    port: 22

AWS S3 Bucket

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: backup_s3
  description: AWS S3 Backup Bucket
spec:
  category: storage
  type: s3
  title: "Backup S3 Bucket"
  parameters:
    provider: AWS
    access_key_id: '{{azure:aws-access-key}}'
    secret_access_key: '{{azure:aws-secret-key}}'
    region: us-east-1

Using Weik.io Variables

CoreSystems can use Weik.io Variables for credential management:

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: partner_ftp
  description: Partner FTP Server
spec:
  category: storage
  type: ftp
  title: "Partner FTP"
  parameters:
    host: ftp.partner.com
    user: '{{sys:FTP_USERNAME}}'
    pass: '{{sys:FTP_PASSWORD}}'
    port: 21

Creating MFT Definitions

MFT definitions reference CoreSystems by name and define transfer operations.

Basic File Transfer

Transfer a file from SMB share to Azure Blob storage daily:

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: hr_data_to_blob
spec:
  source:
    name: company_smb
    path: integrata/data/employees.csv
  destination:
    name: weikio_blob
    path: company/
  command: copy
  schedule: 0 0 0 * * ?
  requirements:
    Location: OnPremise
  parameters:
    camel.vault.azure.tenantId: '{{env:AZURE_TENANT_ID}}'
    camel.vault.azure.clientId: '{{env:AZURE_CLIENT_ID}}'
    camel.vault.azure.clientSecret: '{{env:AZURE_CLIENT_SECRET}}'
    camel.vault.azure.vaultName: kv-integrations-prod
  additionalPackages: camel-azure-key-vault

Key elements:

  • source.name and destination.name reference CoreSystem resources
  • command specifies the transfer operation (copy, sync, move)
  • schedule uses cron expression for timing
  • requirements controls which agent executes the transfer
  • parameters configures Azure Key Vault for credential management

Filtered File Transfer

Transfer only CSV files from SFTP to S3:

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: reports_to_s3
spec:
  source:
    name: partner_sftp
    path: exports/reports/
  destination:
    name: backup_s3
    path: reports/
  command: copy
  filters: "*.csv"
  schedule: 0 0 * * * ?

When CoreSystems contain credential references, no additional vault configuration is needed in the MFT definition.

Security Best Practices

NEVER hardcode credentials in YAML definitions. Always use secure secret management for passwords, API keys, and access tokens.

Use Azure Key Vault to store and retrieve secrets. Reference secrets using the {{azure:secret-name}} syntax:

parameters:
  pass: '{{azure:smb-password}}'
  key: '{{azure:azure-storage-key}}'
  access_key_id: '{{azure:aws-access-key}}'

Configure Azure Key Vault access in your MFT definition:

parameters:
  camel.vault.azure.tenantId: '{{env:AZURE_TENANT_ID}}'
  camel.vault.azure.clientId: '{{env:AZURE_CLIENT_ID}}'
  camel.vault.azure.clientSecret: '{{env:AZURE_CLIENT_SECRET}}'
  camel.vault.azure.vaultName: kv-integrations-prod
additionalPackages: camel-azure-key-vault

Weik.io Variables (Platform-Native Secret Management)

Use Weik.io Variables for centralized secret management. Reference variables using the {{sys:VARIABLE_NAME}} syntax:

parameters:
  pass: '{{sys:SMB_PASSWORD}}'
  key: '{{sys:AZURE_STORAGE_KEY}}'
  access_key_id: '{{sys:AWS_ACCESS_KEY}}'

Create variables using the Weik.io CLI:

apiVersion: weik.io/v1alpha1
kind: Variable
metadata:
  name: SMB_PASSWORD
spec:
  value: "your-secure-password"
  isSecret: true

See Using Variables for complete documentation.

Additional Security Practices

Use Read-Only Credentials When Possible

  • Restrict source systems to read-only access
  • Grant write access only where required

Rotate Credentials Regularly

  • Update secrets in Key Vault or Variables
  • CoreSystems automatically use updated values

Monitor Access Patterns

  • Review transfer logs regularly
  • Set up alerts for unusual activity

Reusing CoreSystems

Multiple MFTs can reference the same CoreSystem, enabling consistent storage configuration across transfers.

# MFT 1: Daily reports
apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: daily_reports
spec:
  source:
    name: company_smb
    path: reports/daily/
  destination:
    name: weikio_blob
    path: reports/
  command: copy
  schedule: 0 0 1 * * ?
---
# MFT 2: Weekly summaries (uses same CoreSystems)
apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: weekly_summaries
spec:
  source:
    name: company_smb
    path: reports/weekly/
  destination:
    name: weikio_blob
    path: summaries/
  command: copy
  schedule: 0 0 9 * * MON

Both MFTs reference company_smb and weikio_blob. Update the CoreSystem definition once to change connection details for all MFTs using it.

What’s Next