Shape what moves
MFT 3 · 3:41
The supplier's folder also has a price list, and a filter pattern keeps it out of the archive so that only invoice PDFs move. The operation field decides whether the originals stay on the supplier's server.
What it covers
- The
filtersfield, with the pattern*.pdffor the invoices - How each pattern becomes an rclone include rule, followed by one exclude rule
- Applying the change as version 0.1.1, which the deployment picks up
- A new run that copies only the PDF, in the logs and in the Transfers section
- The
operationfield and the difference between copy, move and sync retries,retriesSleepandflags, the fields the scaffold shows as comments
Commands in this video
Terminal
$ weikio apply -f ./finance
$ weikio invoke finance/supplier-invoices --endpoint direct:mft-run
$ weikio logs finance/supplier-invoices --env devtestTranscript
- 0:02Last time's transfer runs, and one file has moved.
- 0:05This video shapes what moves, with the transfer's own two fields.
- 0:13The transfer is deployed and running, and its folder also holds a price list.
- 0:18Only the invoices should be copied to the archive, so this video changes what moves.
- 0:24The change is a line in the document, applied the same way as any integration.
- 0:33The field is filters, a list of patterns,
- 0:37and the pattern keeps only the files whose names match.
- 0:41For this transfer the pattern is star dot pdf, so only the invoice PDFs match.
- 0:48The product renders each pattern for rclone as an include rule,
- 0:52and one closing exclude follows, so only the matching files move.
- 0:58The changed document is applied the same way as the publish,
- 1:01and it builds a new version.
- 1:04The new version is 0.1.1, built like the first.
- 1:08The deployment follows latest, so it takes the new version.
- 1:12The agent restarts on it, the same publish and run loop as any integration.
- 1:25Run now again, as in the last video, and this run copies only the PDFs.
- 1:32The reply answers at once:
- 1:33the run is queued and starts when any run in progress has ended.
- 1:43The rclone line for the new file is in the logs.
- 1:46Only invoice-1042 dot pdf is in the line, and the price list is not.
- 1:53It is in the bucket, beside the file the first run moved.
- 1:56The Transfers section keeps a row for every file a transfer moves:
- 2:01the time, the file, the action, the size.
- 2:05Here the newest row reads Copied for the PDF, and the CSV file has no row.
- 2:14The other field is operation, an rclone subcommand,
- 2:18checked at apply against the list the pinned rclone publishes.
- 2:22Copy copies the files to the destination and leaves the originals where they are.
- 2:27Move copies them and then removes them from the source,
- 2:31the choice when the files should leave the supplier's folder once taken.
- 2:35Sync makes the destination match the source,
- 2:38and it removes files there the source no longer has. This example keeps copy.
- 2:48Two more fields shape the run,
- 2:50and they are commented in the scaffold, one sentence each.
- 2:54Retries says how many times rclone tries the work again within one run,
- 2:59and retriesSleep how long it waits between tries.
- 3:03Left out, rclone takes its own three tries and no wait between them.
- 3:08Flags pass along anything else rclone takes,
- 3:12written in the document verbatim, rendered before the filters.
- 3:20The filter is in, and it holds the rest back.
- 3:23The new version is running on the agent, a JVM as for any integration.
- 3:28Only the PDFs moved, and the operation stays copy.
Next in this track
MFT 4 · 3:44 What happened: the files behind a run Watched