Moving separate Camel applications into Weik.io

Many teams that use Apache Camel run each integration as its own application: a Spring Boot or Quarkus service, or a Camel JBang script on a server, each with its own build, its own deployment and its own way of logging. This post moves one such integration into Weik.io, where it becomes a folder of Camel files with versions, environments, logs, metrics and traces.

The example is erp-invoice-export of a team called finance, which fetches pending invoices from an ERP every minute, maps them and publishes them.

The folder

An integration in Weik.io is a directory. The route files go in as they are written, in Camel's YAML, XML or Java DSL. Java classes that the routes call go into beans/, which the build compiles with javac. Kamelets of this integration go into kamelets/, and kamelets that several integrations share become project kamelets. Static settings stay in application.properties, and any other file the routes read, such as mapping.txt, arrives on the agent at the same path. A README.md in the folder shows on the integration's page in the admin UI, Mermaid diagrams included.

finance/
  project.yaml
  variables/
    erp-url.yaml
  production/
    variables/
      erp-url.yaml
  erp-invoice-export/
    integration.yaml
    routes/
      fetch-invoices.camel.yaml
      transform.camel.yaml
      publish.camel.yaml
    beans/
      InvoiceMapper.java
    kamelets/
      erp-paging-source.kamelet.yaml
    application.properties
    mapping.txt
    README.md

The team keeps the folder in Git with a project.yaml at the top, so no file below has to repeat the project.

The dependencies

In the separate application the dependencies were in a pom.xml. In Weik.io the build resolves the Camel components your routes use. integration.yaml lists only what the routes don't show: a component used through a bean, and third-party jars by their Maven coordinates. A jar that is in no repository goes anywhere in the folder, such as a libs/ folder. The same file sets the version and the Camel line.

name: erp-invoice-export
version: 1.0.0
camelVersion: "4.22.1"
requirements: { os: linux }
dependencies:
  - camel:jdbc                                        # a Camel component the routes don't reveal by URI
  - mvn:com.microsoft.sqlserver:mssql-jdbc:12.8.1.jre11
kamelets: [erp-paging-source]                         # shared kamelets of the project

The hub resolves the jars once per version and keeps them by their SHA-256, so two agents on one version run the same files. An agent fetches only the files that changed between two versions. An air-gapped hub resolves everything through its own Maven mirror.

The values

A separate application usually had a properties file per environment. In Weik.io the routes keep using Camel's own placeholders, and the values come from the platform:

- route:
    from:
      uri: timer:tick?period=60000
      steps:
        - setHeader: { name: Authorization, simple: "Bearer {{erp-api-key}}" }
        - to: "{{erp-url}}/api/invoices/pending"
        - to: direct:transform

A variable without an environment is the default, and a variable with one overrides it there. Each value is its own document, so a developer who may edit devtest's values never sees production's.

apiVersion: weik.io/v1
kind: variable
metadata: { name: erp-url, project: finance, environment: production }
spec: { value: https://erp.example.internal }

Secrets have the same shape, but you set them with the CLI so that their values never go into Git:

$ weikio secret set finance/erp-api-key --env devtest
$ printf '%s' "$ERP_PROD_KEY" | weikio secret set finance/erp-api-key --env production
Target: https://hub.example, project finance (context), environment production (--env)
secret finance/erp-api-key set for production (revision 4183)

Run it, publish it, deploy it

weikio dev run runs the folder on your machine with Camel JBang on the integration's Camel line and devtest's values, before it was ever published. When it works, weikio apply publishes an immutable version and weikio deploy runs it in devtest:

$ weikio dev run ./erp-invoice-export --env devtest
$ weikio apply -f ./finance/erp-invoice-export --bump minor
Target: https://hub.example, project finance (from finance/project.yaml)
Uploading 2/9 files, building...
integration finance/erp-invoice-export updated (version 1.1.0, built in 17.3 s)
integration.yaml updated to 1.1.0

Nothing new is running yet. To run it:
  weikio deploy -f ./finance/erp-invoice-export --env devtest

$ weikio deploy finance/erp-invoice-export
Target: https://hub.example, project finance (context), environment devtest (context)
Deploying finance/erp-invoice-export 1.1.0 (latest) to devtest...
deployment devtest/finance/erp-invoice-export updated (version 1.0.0 -> 1.1.0)
Agents: 1 of 1 running after 3.4 s

From devtest the tested version moves on with weikio promote finance/erp-invoice-export --from devtest --to production, without a new build. In a team that keeps its integrations in Git, the pipeline runs these commands, and a promotion is a one-line pull request.

What the integration gets

• Versions that can't change once published, with the exact jar set of each one in weikio describe version
• Environments, promotion without a rebuild and weikio rollback to the previous version
• Logs, metrics and traces through OpenTelemetry, each labelled with the integration, version, environment and agent
• Problems raised when the integration fails, restarts in a loop or stops handling messages
• Two or more running copies on matching agents, or one copy at a time with Camel's master: component and the lock kept by the hub
• Roles that grant access down to this one integration, and an audit log of every change

Book a technical call to go through your own Camel applications with the Weik.io team.