Weik.io

File Transfer Patterns

Common file transfer patterns and use cases

Most file transfers follow a few standard scenarios: backing up data, sending files to a partner, or keeping two folders in sync. Because Weik.io MFT uses Rclone, the behavior of a transfer depends on the command you choose.

Core commands

The command field in an MFT definition determines how files are handled at the destination.

copy

command: copy

copy is the safest option. It transfers new and modified files from the source to the destination. If a file already exists at the destination, it is overwritten when the source is newer, but files at the destination that are not in the source are never deleted.

Best for:

  • Standard backups
  • Log aggregation
  • Scenarios where data loss is not acceptable

sync

command: sync

Use sync when the destination folder must be an exact mirror of the source. If you delete a file from the source, the next sync deletes it from the destination as well.

Use sync with care. If a sync job points at an empty source folder, it deletes the entire contents of the destination folder.

Best for:

  • Mirroring a file share to the cloud
  • Website deployments

move

command: move

move copies files to the destination and, after verifying that the copy succeeded, deletes them from the source.

Best for:

  • Ingesting files into a processing pipeline, so the same file is not processed twice
  • Archiving old data to free space on the source

Filtering transferred files

To transfer only some of the files in a folder, use standard glob patterns in the filters field.

filters: "*.csv"              # CSV files only
filters: "*.{csv,json,xml}"   # Multiple specific types
filters: "report_*.csv"       # Files starting with "report_"
filters: "data_2025-*.csv"    # Files matching a specific year pattern

Example scenarios

The following examples show common ways to combine commands and filters.

Daily database backup

If a database writes a compressed SQL dump to an SMB share every night, use copy to transfer it to cloud storage. Filtering for *.sql.gz excludes unrelated files in the folder.

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: db_backup_to_cloud
spec:
  source:
    name: backup_server_smb
    path: backups/database/
  destination:
    name: backup_s3
    path: database/
  command: copy
  schedule: 0 0 2 * * ?
  filters: "*.sql.gz"

Partner file exchange

To send generated invoices to a partner’s SFTP server and remove them from the local folder afterwards, use move.

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: invoice_to_partner
spec:
  source:
    name: company_smb
    path: exports/invoices/
  destination:
    name: partner_sftp
    path: incoming/
  command: move
  schedule: 0 0 */2 * * ?
  filters: "invoice_*.xml"

Because the definition uses move, the source folder stays clean and the next run does not re-send old invoices.

Cloud-to-cloud migration

To migrate an archive from Azure Blob to AWS S3, set up a copy job that runs overnight.

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: azure_to_s3_migration
spec:
  source:
    name: weikio_blob
    path: archive/
  destination:
    name: backup_s3
    path: migration/
  command: copy
  schedule: 0 0 3 * * ?

Keeping a replica in sync

To mirror a local document share to the cloud every four hours, use sync. Note that if someone deletes a document locally, the sync job also deletes the cloud copy.

apiVersion: weik.io/v1alpha1
kind: MFT
metadata:
  name: document_sync
spec:
  source:
    name: company_smb
    path: shared/documents/
  destination:
    name: weikio_blob
    path: documents/
  command: sync
  schedule: 0 */4 * * * ?

Next steps