Skip to content

kubeflow/kfp-tekton

Folders and files

NameName
Last commit message
Last commit date
Feb 13, 2024
Mar 27, 2024
Jan 3, 2024
Jan 31, 2022
Aug 30, 2021
Jun 26, 2023
Feb 13, 2024
May 24, 2022
Dec 4, 2023
Jan 18, 2024
Jan 18, 2024
Jul 27, 2023
Oct 17, 2023
Feb 27, 2024
Apr 16, 2024
Oct 11, 2023
Aug 30, 2021
Jun 23, 2020
Oct 11, 2023
Aug 22, 2022
Jan 19, 2021
Aug 16, 2023
Mar 3, 2022
Mar 5, 2024
Oct 23, 2019
Jan 18, 2024
Apr 17, 2024
Jan 10, 2024
Dec 21, 2023
Jan 18, 2024
Nov 22, 2023
Nov 22, 2023

Repository files navigation

Kubeflow Pipelines on Tekton

Project bringing Kubeflow Pipelines and Tekton together. The project is driven according to this design doc. The current code allows you run Kubeflow Pipelines with Tekton backend end to end.

  • Create your Pipeline using Kubeflow Pipelines DSL, and compile it to Tekton YAML.
  • Upload the compiled Tekton YAML to KFP engine (API and UI), and run end to end with logging and artifacts tracking enabled.
  • In KFP-Tekton V2, the SDK compiler will generate the same intermediate representation as in the main Kubeflow pipelines SDK. All the Tekton related implementations are all embedded into the V2 backend API service.

For more details about the project please follow this detailed blog post . For the latest KFP-Tekton V2 implementation and supported offerings, please follow our latest Kubecon Talk and slides. For information on the KFP-Tekton V1 implementation, look at these slides as well as this deep dive presentation for demos.

Architecture

We are currently using Kubeflow Pipelines 1.8.4 and Tekton >= 0.53.2 in the master branch for this project.

For Kubeflow Pipelines 2.0.5 and Tekton >= 0.53.2 integration, please check out the kfp-tekton v2-integration branch and KFP-Tekton V2 deployment instead.

kfp-tekton

Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning (ML) workflows. More architectural details about the Kubeflow Pipelines can be found on the Kubeflow website.

The Tekton Pipelines project provides Kubernetes-style resources for declaring CI/CD-style pipelines. Tekton introduces several Custom Resource Definitions(CRDs) including Task, Pipeline, TaskRun, and PipelineRun. A PipelineRun represents a single running instance of a Pipeline and is responsible for creating a Pod for each of its Tasks and as many containers within each Pod as it has Steps. Please look for more details in the Tekton repo.

Get Started using Kubeflow Pipelines on Tekton

Install Kubeflow Pipelines with Tekton backend

KFP Tekton Pipelines User Guide

Use KFP Tekton SDK

Run Samples

Available KFP DSL Features

Tekton Specific Features

Development Guides

Backend Developer Guide

SDK Developer Guide

Compilation Tests Status Report

Design Guides

Design Doc

KFP, Argo and Tekton Features Comparison

Community

Kubeflow Slack

References

Kubeflow and TFX Pipelines

Kubeflow and TFX Pipelines talk at Tensorflow World