Kubeflow Pipelines Cross-Tenant RCE (CVE-2026-77596)
Kubeflow Pipelines before 2.18.0 skips authorization for shared pipeline uploads, letting a low-privilege tenant run code in another namespace. Fixed in 2.18.0.
TL;DR
- A low-privilege tenant can upload a shared pipeline, or a new version of an existing shared pipeline, without a Kubernetes SubjectAccessReview. The advisory was published on 2026-10-28.
- Kubeflow Pipelines before 2.18.0 is affected in multi-user mode with
REQUIRE_NAMESPACE_FOR_PIPELINESfalse, the default. An unpinned recurring run that follows the latest version then runs the injected pipeline in another tenant's namespace. - The CVE record rates it CVSS v3.1 9.6 Critical. No public proof of concept and no exploitation in the wild are reported, and the CVE is not in CISA KEV as of 2026-11-06.
- Upgrade to Kubeflow Pipelines 2.18.0 or later. If you cannot, require a namespace on pipeline operations and put an authenticating proxy in front of the API server; see "How do I fix it?".
Am I affected?
You are exposed only when all of these hold:
- The cluster runs Kubeflow Pipelines before 2.18.0. The advisory lists releases up to 2.17.3 as affected.
- The API server runs in multi-user mode. Single-user installations skip the authorization path entirely and are not affected by this flaw.
REQUIRE_NAMESPACE_FOR_PIPELINESis false. It defaults to false and is not set in the shipped manifests, so a default multi-user installation is exposed.- The API server is reachable by the attacker. When it is reachable directly, without an authenticating proxy, an unauthenticated caller can use the bypass. Behind an authenticating proxy, the caller needs a valid low-privilege tenant identity.
- For the code-execution step, the target namespace has an unpinned recurring run that follows the latest pipeline version. A pinned run, or a run that embeds its specification, is not substituted.
The API server is the ml-pipeline deployment in the kubeflow namespace. Its HTTP upload endpoints are POST /apis/v2beta1/pipelines/upload and POST /apis/v2beta1/pipelines/upload_version, with the older v1beta1 paths served by the same server.
| Branch | Affected versions | Fixed version |
|---|---|---|
| Kubeflow Pipelines | every release before 2.18.0 (advisory: up to 2.17.3) | 2.18.0 (released 2026-10-28) |
Check the image tag and the namespace setting:
kubectl -n kubeflow get deploy ml-pipeline -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{" "}{.image}{"\n"}{end}'
kubectl -n kubeflow get deploy ml-pipeline -o yaml | grep -i REQUIRE_NAMESPACE_FOR_PIPELINES
An image tag lower than 2.18.0, with no REQUIRE_NAMESPACE_FOR_PIPELINES entry or with it set to false, matches the exposed default. When the variable is absent, the server uses false.
How do I fix it?
- Upgrade to Kubeflow Pipelines 2.18.0 or later. The 2.18.0 release is the first release with the authorization fix. Apply the matching manifests for your installation method.
- If you cannot upgrade now, set
REQUIRE_NAMESPACE_FOR_PIPELINES=trueon the API server. The validation then rejects an empty namespace before the upload reaches the unguarded branch. This is a mitigation, not the fix: a caller can still name a namespace it does not own, and the check that follows is the one the patch repairs. - Put an authenticating proxy in front of the API server and do not expose it directly. The advisory says a directly reachable server lets an unauthenticated caller use the bypass. A proxy that authenticates every request removes that path.
- Audit shared pipelines and their versions for container images, commands and arguments you do not recognize, and review recurring runs that follow the latest version.
The vendor advisory names no code-level workaround. Upgrading is the only complete remediation.
What is the root cause?
The HTTP upload path returns success from its authorization helper whenever the target namespace is empty, so a shared-pipeline upload never reaches the SubjectAccessReview. The helper in backend/src/apiserver/server/pipeline_upload_server.go at 2.17.3:
func (s *PipelineUploadServer) canUploadVersionedPipeline(r *http.Request, pipelineID string, resourceAttributes *authorizationv1.ResourceAttributes) error {
if !common.IsMultiUserMode() {
// Skip authorization if not multi-user mode.
return nil
}
if len(pipelineID) > 0 {
namespace, err := s.resourceManager.FetchNamespaceFromPipelineId(pipelineID)
if err != nil {
return util.Wrap(err, "Failed to authorize with the Pipeline ID")
}
if len(resourceAttributes.Namespace) == 0 {
resourceAttributes.Namespace = namespace
}
}
if resourceAttributes.Namespace == "" {
return nil
}
resourceAttributes.Group = common.RbacPipelinesGroup
// ... IsAuthorized() runs only after this point
}
An empty namespace is how Kubeflow Pipelines addresses a shared, cluster-wide pipeline. It is also the default when a caller omits the namespace query parameter. The validation that could reject it runs only when IsNamespaceRequiredForPipelines() is true:
func ValidateNamespaceRequired(namespace string) error {
if common.IsMultiUserMode() &&
common.IsNamespaceRequiredForPipelines() &&
namespace == "" {
return util.NewInvalidInputError("Namespace is required for pipeline operations")
}
return nil
}
IsNamespaceRequiredForPipelines() reads REQUIRE_NAMESPACE_FOR_PIPELINES with a default of false. No manifest under manifests/ sets it. The gRPC path for the same write is canAccessPipeline() in pipeline_server.go. It does the opposite: it remaps an empty namespace to the system namespace for create, update and delete. A comment there says shared-pipeline writes must still be authorized. The HTTP helper is therefore an omission rather than an intended exception.
How is it exploited?
- The attacker reaches the API server. Without an authenticating proxy, no account is needed. Behind a proxy, a low-privilege tenant account is enough.
- The attacker sends a multipart POST to
/apis/v2beta1/pipelines/uploadwithout anamespaceparameter, or to/apis/v2beta1/pipelines/upload_versionwith the id of an existing shared pipeline. canUploadVersionedPipeline()sees an empty namespace and returnsnil. No SubjectAccessReview is sent.- The upload creates a new shared pipeline, or attaches a new version to an existing shared pipeline. The version carries the attacker's container image, command and arguments.
- In another tenant's namespace, an unpinned recurring run that follows the latest version picks up the injected specification.
- The run executes the attacker's container under that run's service account in the victim namespace. That account's Kubernetes permissions now apply to the attacker's code.
The request shape is a standard KFP multipart upload. No public proof-of-concept code is published; the advisory describes the endpoints and the missing check.
How does the attack flow?
An empty namespace skips the authorization call, and a shared pipeline version carries the attacker's container into another tenant's recurring run.
What can an attacker do?
An attacker can run an arbitrary container in another tenant's namespace, under the service account of that tenant's recurring run. Every condition is listed here in full:
- Kubeflow Pipelines before 2.18.0.
- Multi-user mode, with
REQUIRE_NAMESPACE_FOR_PIPELINESfalse or unset. - The API server reachable by the attacker, either directly or through an authenticating proxy that admits a low-privilege account.
- An existing shared pipeline for the version-injection path, or permission to create a new shared pipeline for the create path.
- In the victim namespace, an unpinned recurring run that follows the latest version.
The impact includes reading the namespace's secrets that the run's service account can read, modifying its resources, and using its network position. The exact impact depends on the service account's role bindings. A pinned run, or a run that embeds its own specification, is not substituted by this path. The advisory does not demonstrate an unauthenticated path when an authenticating proxy protects the server.
Why does the fix work?
The fix maps an empty namespace to the KFP system namespace and lets the existing SubjectAccessReview decide, so a shared-pipeline write is authorized like any other write. The patch in pipeline_upload_server.go:
if len(pipelineID) > 0 {
namespace, err := s.resourceManager.FetchNamespaceFromPipelineId(pipelineID)
if err != nil {
return util.Wrap(err, "Failed to authorize with the Pipeline ID")
}
if len(resourceAttributes.Namespace) == 0 {
resourceAttributes.Namespace = namespace
}
}
- if resourceAttributes.Namespace == "" {
- return nil
+ if s.resourceManager.IsEmptyNamespace(resourceAttributes.Namespace) {
+ // Shared-pipeline writes require authorization in the KFP system namespace.
+ resourceAttributes.Namespace = common.GetPodNamespace()
}
resourceAttributes.Group = common.RbacPipelinesGroup
The code now calls IsAuthorized() for shared writes, using the namespace the API server runs in. A caller that cannot create pipelines in that namespace is denied. The fix also adds a recordingSubjectAccessReviewClient test matrix. It covers shared and namespaced uploads, version uploads, allowed and denied cases, and single-user mode, so the empty-namespace branch is exercised in CI.
The fix is complete for the two HTTP endpoints the advisory names. It does not change the RBAC design: a tenant that already holds shared-pipeline write permission keeps it, and the authorization decision is the cluster's.
How do I check if I was compromised?
The vendor publishes no indicators of compromise. Check the API server log and the shared pipeline inventory:
# Pipeline upload requests and authorization failures in the API server log.
kubectl -n kubeflow logs deploy/ml-pipeline --since=720h | grep -Ei 'upload|SubjectAccessReview|Authorization Failure'
# Shared pipelines visible through the API.
kubectl -n kubeflow port-forward deploy/ml-pipeline 8888:8888 &
curl -s http://localhost:8888/apis/v2beta1/pipelines | head -c 2000
In a multi-user installation, a pipeline with no namespace is a shared pipeline. Review every shared pipeline and its versions, and compare each version's container image, command and arguments with what your teams expect. Then list the recurring runs that follow the latest version and check their recent run history for images you did not deploy.
Kubernetes audit logs record SubjectAccessReview calls when the audit policy includes them. At the Request audit level or higher, the review body includes the caller identity. Look for the upload request itself in ml-pipeline or proxy access logs, if request logging is enabled. The absence of a SubjectAccessReview is not itself logged.
A successful upload is silent in the KFP UI beyond the new pipeline or version. Treat an unexpected shared pipeline, or a new version of a shared pipeline you did not publish, as a lead to investigate.
Timeline
- 2026-09-09: the fix is merged as pull request #14352.
- 2026-10-28: Kubeflow Pipelines 2.18.0 is released and advisory GHSA-rjr3-43j3-4q8g is published.
- 2026-11-05: CVE-2026-77596 is published with CVSS v3.1 9.6.
- 2026-11-06: no exploitation in the wild is reported, and the CVE is not in CISA KEV.
FAQ
Is a single-user Kubeflow Pipelines installation affected?
No. The helper returns early when multi-user mode is off, so the authorization path and the empty-namespace branch are never used. The flaw applies to multi-user installations.
Does an attacker need a Kubernetes account?
Not when the API server is reachable directly without an authenticating proxy; the advisory says an unauthenticated caller can use the bypass in that case. Behind an authenticating proxy, the attacker needs a valid low-privilege tenant account.
Does the attacker's code always run in another tenant's namespace?
No. It runs when the target namespace has an unpinned recurring run that follows the latest version. A run that pins a version, or embeds its specification, is not substituted by this path.
Is setting REQUIRE_NAMESPACE_FOR_PIPELINES=true enough?
It blocks the empty-namespace path, which is the bypass this CVE describes. It is a mitigation, not the fix, and it does not replace the authorization check the patch adds. Upgrade when you can.
Was it exploited in the wild?
No exploitation in the wild is reported as of 2026-11-06, and the CVE is not in CISA KEV. The advisory was published on 2026-10-28, so the absence of reports is recent.
How do I know which version I run?
Check the ml-pipeline deployment image tag. The manifests pin an image tag per release, and 2.18.0 is the first fixed release. If your platform vendor bundles Kubeflow Pipelines, ask which upstream version the bundle contains.
References
- Kubeflow Pipelines security advisory GHSA-rjr3-43j3-4q8g, published 2026-10-28, with the affected versions, the vulnerable code, the fix and the impact: https://github.com/kubeflow/pipelines/security/advisories/GHSA-rjr3-43j3-4q8g (checked November 6, 2026)
- CVE-2026-77596 record, published 2026-11-05, CVSS v3.1 9.6 and the affected range: https://www.cve.org/CVERecord?id=CVE-2026-77596 (checked November 6, 2026)
- Kubeflow Pipelines pull request #14352, the fix: https://github.com/kubeflow/pipelines/pull/14352 (checked November 6, 2026)
- Kubeflow Pipelines commit 62516d8c525f146b754cb515cf441dd0290ad841, "fix(backend): authorize shared pipeline uploads (#14352)", 2026-09-09: https://github.com/kubeflow/pipelines/commit/62516d8c525f146b754cb515cf441dd0290ad841 (checked November 6, 2026)
- Kubeflow Pipelines 2.18.0 release, 2026-10-28: https://github.com/kubeflow/pipelines/releases/tag/2.18.0 (checked November 6, 2026)
- Kubeflow Pipelines
backend/src/apiserver/server/pipeline_upload_server.goat 2.17.3, the vulnerable helper: https://github.com/kubeflow/pipelines/blob/2.17.3/backend/src/apiserver/server/pipeline_upload_server.go (checked November 6, 2026) - Kubeflow Pipelines
backend/src/apiserver/server/pipeline_server.goat 2.17.3, the gRPCcanAccessPipeline()that authorizes shared writes: https://github.com/kubeflow/pipelines/blob/2.17.3/backend/src/apiserver/server/pipeline_server.go (checked November 6, 2026) - Kubeflow Pipelines
backend/src/apiserver/validation/namespace.goat 2.17.3,ValidateNamespaceRequired(): https://github.com/kubeflow/pipelines/blob/2.17.3/backend/src/apiserver/validation/namespace.go (checked November 6, 2026) - Kubeflow Pipelines
manifests/kustomize/base/pipeline/ml-pipeline-apiserver-deployment.yamlat 2.18.0, theml-pipelinedeployment: https://github.com/kubeflow/pipelines/blob/2.18.0/manifests/kustomize/base/pipeline/ml-pipeline-apiserver-deployment.yaml (checked November 6, 2026) - CISA Known Exploited Vulnerabilities catalog JSON feed, checked for CVE-2026-77596 on November 6, 2026 (not listed): https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json (checked November 6, 2026)
- GitHub,
kubeflow/pipelines, 4,262 stars and 2,185 forks: https://github.com/kubeflow/pipelines (checked November 6, 2026)