Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot apply resources with defined namespaces in manifest #163

Open
jesusha123 opened this issue Dec 20, 2021 · 24 comments · Fixed by #276
Open

Cannot apply resources with defined namespaces in manifest #163

jesusha123 opened this issue Dec 20, 2021 · 24 comments · Fixed by #276
Labels
blue-green-bug-bash Issues found in Blue-Green strategy bug bash idle Inactive for 14 days

Comments

@jesusha123
Copy link

Repro steps:

Current behaviour:

  • Kubernetes deployment fails
Error: the namespace from the provided object "kube-system" does not match the namespace "cert-manager". You must pass '--namespace=kube-system' 
Error: the namespace from the provided object "ingress-nginx" does not match the namespace "default". You must pass '--namespace=ingress-nginx'

Expected behaviour:

  • k8s-deploy should not pass namespace=default parameter to kubectl if namespace is not defined in github action

The reasoning is, official instructions for ingress-nginx, cert-manager and others do not need a namespace parameter in kubectl, and if namespace=default is passed, the command fails. I have to split the manifests into multiple manifests per each namespace as a workaround.

@jesusha123 jesusha123 added the blue-green-bug-bash Issues found in Blue-Green strategy bug bash label Dec 20, 2021
@github-actions
Copy link

github-actions bot commented Jan 3, 2022

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Jan 3, 2022
@OliverMKing
Copy link
Collaborator

Hello! We will look into this. Thanks!

@OliverMKing OliverMKing removed the idle Inactive for 14 days label Mar 3, 2022
@github-actions
Copy link

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Mar 17, 2022
@davidgamero
Copy link
Collaborator

i was able to repro, and just made a PR

@github-actions github-actions bot removed the idle Inactive for 14 days label Jun 14, 2022
@davidgamero
Copy link
Collaborator

merged the fix. please reopen if issue persists

@hansmbakker
Copy link

hansmbakker commented Nov 28, 2022

@davidgamero I believe I'm encountering this issue with k8s-deploy@v4 but I cannot reopen this issue.

I'm trying to deploy cert-manager 1.10.1. cert-manager is normally installed using kubectl apply -f.
I put the manifest from https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml in my repo and ran the following action:

      - name: Deploy cert manager (pre-requisite for actions-runner-controller)
        uses: Azure/k8s-deploy@v4
        with:
          action: deploy
          strategy: basic
          manifests: |
            manifests/deployments/cert-manager-v1.10.1.yml
          force: true

Log:

Run Azure/k8s-deploy@v4
  with:
    action: deploy
    strategy: basic
    manifests: manifests/deployments/cert-manager-v1.10.1.yml
  
    force: true
    namespace: default
    pull-images: true
    route-method: service
    version-switch-buffer: 0
    traffic-split-method: pod
    percentage: 0
    token: ***
    annotate-namespace: true
    private-cluster: false
    skip-tls-verify: false
  env:
    AZURE_HTTP_USER_AGENT: 
    AZUREPS_HOST_ENVIRONMENT: 
    KUBECONFIG: /home/runner/work/_temp/kubeconfig_16696[2](https://github.com/BNGBank/github-self-hosted-runners/actions/runs/3563241016/jobs/5985804619#step:9:2)622506[4](https://github.com/BNGBank/github-self-hosted-runners/actions/runs/3563241016/jobs/5985804619#step:9:4)
    KUBE_CONFIG_PATH: /home/runner/work/_temp/kubeconfig_166962622[5](https://github.com/BNGBank/github-self-hosted-runners/actions/runs/3563241016/jobs/5985804619#step:9:5)0[6](https://github.com/BNGBank/github-self-hosted-runners/actions/runs/3563241016/jobs/5985804619#step:9:6)4
Deploying manifests
  /opt/hostedtoolcache/kubectl/1.25.4/x64/kubectl apply -f /tmp/cert-manager-v1.10.1.yml --force --namespace default
  namespace/cert-manager unchanged
  customresourcedefinition.apiextensions.k[8](https://github.com/BNGBank/github-self-hosted-runners/actions/runs/3563241016/jobs/5985804619#step:9:8)s.io/clusterissuers.cert-manager.io configured
  customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io configured
  customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io configured
  customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io configured
  customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io configured
  customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-view configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-edit configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests configured
  clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests configured
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews configured
  mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
  validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  Error: Error: undefined

@OliverMKing
Copy link
Collaborator

OliverMKing commented Nov 28, 2022

I put the manifest from https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml in my repo and ran the following action:

Try it with the namespace: cert-manager option.

@OliverMKing OliverMKing reopened this Nov 28, 2022
@hansmbakker
Copy link

Try it with the namespace: cert-manager option.

I guess that is no solution because the manifest does not only reference the cert-manager namespace but also the kube-system namespace, and with your suggestion it is only possible to use 1 namespace?

@jaiveerk
Copy link
Collaborator

jaiveerk commented Nov 28, 2022

As a workaround for now, I'd suggest using multiple k8s-deploy commands in your YAML like so:

      - name: Deploy to first namespace
        uses: Azure/k8s-deploy@v4
        with:
          action: deploy
          strategy: basic
          namespace: (FIRST_NAMESPACE)
          manifests: |
            path/to/manifests/for/first/ns
          force: true
      - name: Deploy to second namespace
        uses: Azure/k8s-deploy@v4
        with:
          action: deploy
          strategy: basic
          namespace: (SECOND_NAMESPACE)
          manifests: |
            path/to/manifests/for/second/ns
          force: true

We recently patched an issue where kubectl errors were failing silently, causing the action to pass even if a kubectl command necessary to execute the action had failed. As a result, users were seeing rollout failures even though their resources had successfully deployed because our action was checking the default namespace (if no namespace was provided, but if one was provided to the action it would check that one) for the deployed resources, but since the resources would be deployed to the namespace specified in the YAML, the rollout check would not be able to find the deployed resource, thereby causing the action to fail.

Our solution was to make sure that if no namespace was provided to the action, kubectl would deploy and perform a rollout check on the default namespace, or to do the same to whatever other namespace may be provided. Of course, this leads to the issue that you ran into, where the namespace being used by the action in the kubectl command it runs (default) is different from the namespace specified in your deployment YAML, and kubectl fails as a result.

We plan on working on a fix to support deployment to multiple namespaces within a single run by checking if deployment manifests specify a namespace, but I recommend using the above workaround until we release a new version with this functionality.

I hope this helps!

@hansmbakker
Copy link

hansmbakker commented Nov 28, 2022

That suggested workaround means I have to take apart the manifest that is provided by cert-manager.

I prefer to stick with a manual kubectl apply one this case as a workaround.

@OliverMKing
Copy link
Collaborator

@hansmbakker we will keep you updated on this. We are working on a fix.

@hansmbakker
Copy link

Thank you both and thank you @jaiveerk for the explanation!

@github-actions
Copy link

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Dec 12, 2022
@OliverMKing
Copy link
Collaborator

Not released yet but a fix is merged in

@OliverMKing OliverMKing reopened this Feb 6, 2023
@github-actions github-actions bot removed the idle Inactive for 14 days label Feb 6, 2023
@Tchekda
Copy link

Tchekda commented Feb 14, 2023

Not released yet but a fix is merged in

Not sure if I should open a new issue but I tried using azure/k8s-deploy@main in order to test out the fix your merged, but I got this error

Error: File not found: '/home/myorg/_work/_actions/azure/k8s-deploy/main/lib/index.js'

@github-actions
Copy link

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Feb 28, 2023
@vojtechvelkjop
Copy link

Hi @OliverMKing any ETA pls ?
Issues was reported last year but issue is still in a place.
I have same problem with prometheus stack.

@github-actions github-actions bot removed the idle Inactive for 14 days label May 17, 2023
@github-actions
Copy link

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label May 31, 2023
@davidgamero
Copy link
Collaborator

Not released yet but a fix is merged in

Not sure if I should open a new issue but I tried using azure/k8s-deploy@main in order to test out the fix your merged, but I got this error

Error: File not found: '/home/myorg/_work/_actions/azure/k8s-deploy/main/lib/index.js'

@Tchekda
our release process doesn't store the final action on the main branch, and instead tags release branches, so unfortunately you can't test it out by referencing our main branch.

@vojtechvelkjop
we are updating our release action due to a change in github action permissions which has blocked our normal release process, but we are fixing it now

@github-actions github-actions bot removed the idle Inactive for 14 days label May 31, 2023
@github-actions
Copy link

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Jun 14, 2023
@zigmund
Copy link

zigmund commented Oct 13, 2023

Tried latest 4.9 and 4.10 releases, but issue is still here:

the namespace from the provided object "monitoring" does not match the namespace "default". You must pass '--namespace=monitoring' to perform this operation

@github-actions github-actions bot removed the idle Inactive for 14 days label Oct 13, 2023
@github-actions
Copy link

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Oct 27, 2023
@motcke
Copy link

motcke commented Mar 6, 2024

This issue I believe should have been fixed in version v4.1.0.0 by this #276

I've tried to use it but still getting the same error as others

...
...
Run azure/[email protected]
  with:
    manifests: kubernetes/cert-manager-v1.14.3-k8s-deployment.yaml
  
    namespace: default
    pull-images: true
    strategy: basic
    route-method: service
    version-switch-buffer: 0
    traffic-split-method: pod
    percentage: 0
    action: deploy
    force: false
    token: ***
    annotate-resources: true
    annotate-namespace: true
    private-cluster: false
    skip-tls-verify: false
  env:
    ...
Deploying manifests
  /usr/bin/kubectl apply -f /tmp/cert-manager-v1.14.[3](https://github.com/.......#step:6:3)-k[8](https://github.com/.......deployment.yaml --namespace default
  namespace/cert-manager unchanged
  customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io unchanged
  customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io unchanged
  customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io unchanged
  customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io unchanged
  customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io unchanged
  customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-cluster-view unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-view unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-edit unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests unchanged
  clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests unchanged
  clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews configured
  mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
  validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  the namespace from the provided object "cert-manager" does not match the namespace "default". You must pass '--namespace=cert-manager' to perform this operation.
  Error: Error: undefined
...
...

@irealworlds
Copy link

Still encountering this issue in v5 when trying to deploy a secret to cert-manager namespace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blue-green-bug-bash Issues found in Blue-Green strategy bug bash idle Inactive for 14 days
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants