Private job image registry support

[cloorc]

Tell us about the problem you're trying to solve

We are hosting our own private docker image registry (Harbor). We need specify private registry for all possible source/target images. However, I can only specify the imagePullSecrets but not able to change all images' registry from configration of workers.

Describe the solution you’d like

Can specify an additional private registry besides image pull secrets like this:

airbyte:
  worker:
    job:
      kube:
        main:
          container:
            image-registry: ${JOB_KUBE_MAIN_CONTAINER_IMAGE_REGISTRY:}
            image-pull-policy: ${JOB_KUBE_MAIN_CONTAINER_IMAGE_PULL_POLICY:IfNotPresent}
            image-pull-secret: ${JOB_KUBE_MAIN_CONTAINER_IMAGE_PULL_SECRET:}

Describe the alternative you’ve considered or used

We've just changed the ProcessFactory implementation to support this.

Additional context

None.

Are you willing to submit a PR?

I've updated the implementation of ProcessFactory to support this.

[harshithmullapudi]

Hey this implementation you are suggesting, can you help us make a PR and so that team can look into it?

[salima-airbyte]

@andyjih: need your thoughts on this.

[cloorc]

Hey this implementation you are suggesting, can you help us make a PR and so that team can look into it?

Yes, a pull request about this enhancement is already submitted as :

#16969

[andyjih]

@davinchia Is this something you might have some thoughts or context on?