Allow `job_run` to trigger on every deployment

[aborsato]

I am planning to use job_run resource to trigger a job that complements the deployment - crate/recreate tables, views, etc. This needs to run every deployment, not only once as the current semantics.

My ask is to have an option I can set to change the behavior to always run. Something like:

resources:
  job_runs:
    original_run:
      job_id: ${resources.jobs.original.id}
      job_parameters:
        catalog: main
      run_once: false # <--- HERE

This will allow us to define post-deployment work that can do more than just seeding data - schema evolution, data migration, etc.

[radakam]

Hi @aborsato, thanks for the detailed write-up. This is exactly the use case we had in mind. This will be supported via lifecycle.triggers.on_bundle_deploy on job_runs:

job_runs:
  my_run:
    job_id: ${resources.jobs.my_job.id}
    job_parameters:
      catalog: main
    lifecycle:
      triggers:
        - on_bundle_deploy: true

With the trigger armed, the run fires on every bundle deploy. Without it, behavior is unchanged: the run only fires when its config changes. Removing the trigger does not recreate the existing run.

The change is in #6231, which we expect to release today.