Weik.io

MFT Setup

Setting up CoreSystems and MFT definitions

To run Managed File Transfer, you need two things: CoreSystem resources that tell Weik.io how to connect to your storage, and MFT definitions that tell it what to move and when.

Creating CoreSystems

CoreSystems act as your source and destination endpoints. They are reusable connection profiles. Because they use Rclone, the parameters follow standard Rclone conventions.

Azure Blob Storage

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: weikio_blob
  description: "Weikio Azure Blob"
spec:
  category: storage
  type: azureblob
  title: "Weikio Azure Blob"
  parameters:
    account: weikio
    key: "{{azure:fabric-azure-weikiostorage}}"

SMB file share

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: company_smb
  description: Company SMB Share
spec:
  category: storage
  type: smb
  title: "Company SMB"
  parameters:
    host: fileserver.company.local
    user: integrations
    pass: '{{azure:smb-password}}'
    domain: company.local

SFTP server

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: partner_sftp
  description: Partner SFTP Server
spec:
  category: storage
  type: sftp
  title: "Partner SFTP"
  parameters:
    host: sftp.partner.com
    user: integration_user
    pass: '{{azure:sftp-password}}'
    port: 22

AWS S3 bucket

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: backup_s3
  description: AWS S3 Backup Bucket
spec:
  category: storage
  type: s3
  title: "Backup S3 Bucket"
  parameters:
    provider: AWS
    access_key_id: '{{azure:aws-access-key}}'
    secret_access_key: '{{azure:aws-secret-key}}'
    region: us-east-1

Using Weik.io Variables

If you do not use an external vault such as Azure Key Vault, you can use built-in Weik.io Variables for passwords and keys:

apiVersion: weik.io/v1alpha1
kind: CoreSystem
metadata:
  name: partner_ftp
  description: Partner FTP Server
spec:
  category: storage
  type: ftp
  title: "Partner FTP"
  parameters:
    host: ftp.partner.com
    user: '{{sys:FTP_USERNAME}}'
    pass: '{{sys:FTP_PASSWORD}}'
    port: 21

Creating MFT definitions

After the CoreSystems exist, create MFT definitions to move the files. MFT definitions reference CoreSystems by name.

Basic file transfer

The following definition moves a file from an SMB share to Azure Blob storage every day at midnight:

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: hr_data_to_blob
spec:
  source:
    name: company_smb
    path: integrata/data/employees.csv
  destination:
    name: weikio_blob
    path: company/
  command: copy
  schedule: 0 0 0 * * ?
  requirements:
    Location: OnPremise
  parameters:
    camel.vault.azure.tenantId: '{{env:AZURE_TENANT_ID}}'
    camel.vault.azure.clientId: '{{env:AZURE_CLIENT_ID}}'
    camel.vault.azure.clientSecret: '{{env:AZURE_CLIENT_SECRET}}'
    camel.vault.azure.vaultName: kv-integrations-prod
  additionalPackages: camel-azure-key-vault

Key fields:

  • source.name and destination.name point back to the CoreSystems you created earlier.
  • command tells the system what to do (copy, sync, or move).
  • schedule dictates when it runs, using standard cron syntax.
  • requirements makes sure the job runs on an agent in the right location (for example, one that can reach the on-premise SMB share).
  • parameters configures Azure Key Vault so the job can retrieve the required secrets.

Filtered file transfer

To transfer only specific files, such as CSV files from a folder on an SFTP server, use a filter:

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: reports_to_s3
spec:
  source:
    name: partner_sftp
    path: exports/reports/
  destination:
    name: backup_s3
    path: reports/
  command: copy
  filters: "*.csv"
  schedule: 0 0 * * * ?

Note: If your CoreSystems use Weik.io Variables for their secrets, the vault configuration is not needed in the MFT definition.

Security practices

Do not hardcode passwords in YAML files. Use a secret manager for all credentials.

Azure Key Vault

Pull secrets directly from Azure Key Vault using the {{azure:secret-name}} syntax:

parameters:
  pass: '{{azure:smb-password}}'
  key: '{{azure:azure-storage-key}}'
  access_key_id: '{{azure:aws-access-key}}'

To enable the connection, add these vault details to the bottom of the MFT definition:

parameters:
  camel.vault.azure.tenantId: '{{env:AZURE_TENANT_ID}}'
  camel.vault.azure.clientId: '{{env:AZURE_CLIENT_ID}}'
  camel.vault.azure.clientSecret: '{{env:AZURE_CLIENT_SECRET}}'
  camel.vault.azure.vaultName: kv-integrations-prod
additionalPackages: camel-azure-key-vault

Weik.io Variables

Alternatively, use built-in Weik.io Variables, referenced via the {{sys:VARIABLE_NAME}} syntax:

parameters:
  pass: '{{sys:SMB_PASSWORD}}'
  key: '{{sys:AZURE_STORAGE_KEY}}'
  access_key_id: '{{sys:AWS_ACCESS_KEY}}'

You create the variable value using the CLI:

apiVersion: weik.io/v1alpha1
kind: Variable
metadata:
  name: SMB_PASSWORD
spec:
  value: "your-secure-password"
  isSecret: true

See Using Variables for details.

Additional security practices

  • Use read-only accounts where possible. A source system usually only needs read access.
  • Rotate credentials. If you update a password in Key Vault or a Weik.io Variable, the CoreSystems pick up the change automatically.
  • Watch the logs. Monitor for failed transfers or unusual patterns to catch issues early.

Reusing CoreSystems

Separating the storage configuration from the transfer job lets you reuse connections. If five different jobs need to reach the same SMB server, you define the server once.

# MFT 1: Daily reports
apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: daily_reports
spec:
  source:
    name: company_smb
    path: reports/daily/
  destination:
    name: weikio_blob
    path: reports/
  command: copy
  schedule: 0 0 1 * * ?
---
# MFT 2: Weekly summaries
apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: weekly_summaries
spec:
  source:
    name: company_smb
    path: reports/weekly/
  destination:
    name: weikio_blob
    path: summaries/
  command: copy
  schedule: 0 0 9 * * MON

Both MFTs use company_smb and weikio_blob. If the SMB password changes, you update the company_smb CoreSystem and both jobs keep running without edits.

Next steps