Skip to content

Latest commit

 

History

History
 
 

components

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Components for Kubeflow Pipelines

Kubeflow Pipelines system orchestrates pipeline graphs of containerized command-line programs. Components are the building blocks of pipelines. Component definition files describe component interface (inputs and outputs) and implementation (how to call the containerized program). Users can load components with KFP SDK, instantiate the components and compose them in a pipeline graph.

Example components

See how to build your own components. Also see the tutorials for data passing for components based on python functions and components based on command-line programs.

Index of components

/ _converters / ApacheParquet / from_ApacheArrowFeather / Convert apache arrow feather to apache parquet

/ _converters / ApacheParquet / from_CSV / Convert csv to apache parquet

/ _converters / ApacheParquet / from_TSV / Convert tsv to apache parquet

/ _converters / ApacheParquet / to_ApacheArrowFeather / Convert apache parquet to apache arrow feather

/ _converters / ApacheParquet / to_CSV / Convert apache parquet to csv

/ _converters / ApacheParquet / to_TSV / Convert apache parquet to tsv

/ _converters / KerasModelHdf5 / to_TensorflowSavedModel / Keras convert hdf5 model to tf saved model

/ _converters / OnnxModel / from_KerasModelHdf5 / To ONNX from Keras HDF5 model

/ _converters / OnnxModel / from_TensorflowSavedModel / To ONNX from Tensorflow SavedModel

/ _converters / TensorflowJSGraphModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow JS GraphModel

/ _converters / TensorflowJSGraphModel / from_TensorflowSavedModel / Convert Tensorflow SavedModel to Tensorflow JS GraphModel

/ _converters / TensorflowJSLayersModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow JS LayersModel

/ _converters / TensorflowJSLayersModel / from_TensorflowSavedModel / Convert Keras SavedModel to Tensorflow JS LayersModel

/ _converters / TensorflowLiteModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow Lite model

/ _converters / TensorflowLiteModel / from_TensorflowSavedModel / Convert Tensorflow SavedModel to Tensorflow Lite model

/ aws / athena / query / Athena Query

/ aws / emr / create_cluster / emr_create_cluster

/ aws / emr / delete_cluster / emr_delete_cluster

/ aws / emr / submit_pyspark_job / emr_submit_pyspark_job

/ aws / emr / submit_spark_job / emr_submit_spark_job

/ aws / sagemaker / batch_transform / SageMaker - Batch Transformation

/ aws / sagemaker / deploy / SageMaker - Deploy Model

/ aws / sagemaker / ground_truth / SageMaker - Ground Truth

/ aws / sagemaker / hyperparameter_tuning / SageMaker - Hyperparameter Tuning

/ aws / sagemaker / model / SageMaker - Create Model

/ aws / sagemaker / process / SageMaker - Processing Job

/ aws / sagemaker / train / SageMaker - Training Job

/ aws / sagemaker / workteam / SageMaker - Private Workforce

/ azure / azuredevops / queue-pipeline / Queue Azure Pipeline

/ azure / azureml / aml-register-model / Azure ML Register Model

/ basics / Calculate_hash / Calculate data hash

/ CatBoost / convert_CatBoostModel_to_AppleCoreMLModel / Convert CatBoostModel to AppleCoreMLModel

/ CatBoost / convert_CatBoostModel_to_ONNX / Convert CatBoostModel to ONNX

/ CatBoost / Predict_class_probabilities / from_CSV / Catboost predict class probabilities

/ CatBoost / Predict_classes / from_CSV / Catboost predict classes

/ CatBoost / Predict_values / from_CSV / Catboost predict values

/ CatBoost / Train_classifier / from_CSV / Catboost train classifier

/ CatBoost / Train_regression / from_CSV / Catboost train regression

/ dataset_manipulation / split_data_into_folds / in_CSV / Split table into folds

/ datasets / Chicago_Taxi_Trips / Chicago Taxi Trips dataset

/ deprecated / dataflow / predict / Predict using TF on Dataflow

/ deprecated / dataflow / tfdv / TFX - Data Validation

/ deprecated / dataflow / tfma / TFX - Analyze model

/ deprecated / dataflow / tft / Transform using TF on Dataflow

/ deprecated / tfx / Evaluator / Evaluator

/ deprecated / tfx / Evaluator / with_URI_IO / Evaluator

/ deprecated / tfx / ExampleGen / BigQueryExampleGen / BigQueryExampleGen

/ deprecated / tfx / ExampleGen / BigQueryExampleGen / with_URI_IO / BigQueryExampleGen

/ deprecated / tfx / ExampleGen / CsvExampleGen / CsvExampleGen

/ deprecated / tfx / ExampleGen / CsvExampleGen / with_URI_IO / CsvExampleGen

/ deprecated / tfx / ExampleGen / ImportExampleGen / ImportExampleGen

/ deprecated / tfx / ExampleGen / ImportExampleGen / with_URI_IO / ImportExampleGen

/ deprecated / tfx / ExampleValidator / ExampleValidator

/ deprecated / tfx / ExampleValidator / with_URI_IO / ExampleValidator

/ deprecated / tfx / SchemaGen / SchemaGen

/ deprecated / tfx / SchemaGen / with_URI_IO / SchemaGen

/ deprecated / tfx / StatisticsGen / StatisticsGen

/ deprecated / tfx / StatisticsGen / with_URI_IO / StatisticsGen

/ deprecated / tfx / Trainer / Trainer

/ deprecated / tfx / Trainer / with_URI_IO / Trainer

/ deprecated / tfx / Transform / Transform

/ deprecated / tfx / Transform / with_URI_IO / Transform

/ diagnostics / diagnose_me / Run diagnose me

/ filesystem / get_file / Get file

/ filesystem / get_subdirectory / Get subdirectory

/ filesystem / list_items / List items

/ gcp / automl / create_dataset_for_tables / Automl create dataset for tables

/ gcp / automl / create_model_for_tables / Automl create model for tables

/ gcp / automl / deploy_model / Automl deploy model

/ gcp / automl / export_data_to_gcs / Automl export data to gcs

/ gcp / automl / export_model_to_gcs / Automl export model to gcs

/ gcp / automl / import_data_from_bigquery / Automl import data from bigquery

/ gcp / automl / import_data_from_gcs / Automl import data from gcs

/ gcp / automl / prediction_service_batch_predict / Automl prediction service batch predict

/ gcp / automl / split_dataset_table_column_names / Automl split dataset table column names

/ gcp / bigquery / query / to_CSV / Bigquery - Query

/ gcp / bigquery / query / to_gcs / Bigquery - Query

/ gcp / bigquery / query / to_table / Bigquery - Query

/ gcp / dataflow / launch_python / Launch Python

/ gcp / dataflow / launch_template / Launch Dataflow Template

/ gcp / dataproc / create_cluster / dataproc_create_cluster

/ gcp / dataproc / delete_cluster / dataproc_delete_cluster

/ gcp / dataproc / submit_hadoop_job / dataproc_submit_hadoop_job

/ gcp / dataproc / submit_hive_job / dataproc_submit_hive_job

/ gcp / dataproc / submit_pig_job / dataproc_submit_pig_job

/ gcp / dataproc / submit_pyspark_job / dataproc_submit_pyspark_job

/ gcp / dataproc / submit_spark_job / dataproc_submit_spark_job

/ gcp / dataproc / submit_sparksql_job / dataproc_submit_sparksql_job

/ gcp / ml_engine / batch_predict / Batch predict against a model with Cloud ML Engine

/ gcp / ml_engine / deploy / Deploying a trained model to Cloud Machine Learning Engine

/ gcp / ml_engine / train / Submitting a Cloud ML training job as a pipeline step

/ git / clone / Git clone

/ google-cloud / AutoML / Tables / Create_dataset / from_CSV / Automl create tables dataset from csv

/ google-cloud / Optimizer / Add_measurement_for_trial / Add measurement for trial in gcp ai platform optimizer

/ google-cloud / Optimizer / Create_study / Create study in gcp ai platform optimizer

/ google-cloud / Optimizer / Suggest_parameter_sets_based_on_measurements / Suggest parameter sets from measurements using gcp ai platform optimizer

/ google-cloud / Optimizer / Suggest_trials / Suggest trials in gcp ai platform optimizer

/ google-cloud / storage / download / Download from GCS

/ google-cloud / storage / download_blob / Download from GCS

/ google-cloud / storage / download_dir / Download from GCS

/ google-cloud / storage / list / List blobs

/ google-cloud / storage / upload_to_explicit_uri / Upload to GCS

/ google-cloud / storage / upload_to_unique_uri / Upload to GCS

/ ibm-components / commons / config / Create Secret - Kubernetes Cluster

/ ibm-components / ffdl / serve / Serve PyTorch Model - Seldon Core

/ ibm-components / ffdl / train / Train Model - FfDL

/ ibm-components / spark / data_preprocess_spark / Preprocess Data using Spark - IBM Cloud

/ ibm-components / spark / store_spark_model / Store Spark Model - Watson Machine Learning

/ ibm-components / spark / train_spark / Train Spark Model - IBM Cloud

/ ibm-components / watson / deploy / Deploy Model - Watson Machine Learning

/ ibm-components / watson / manage / monitor_fairness / Monitor Fairness - Watson OpenScale

/ ibm-components / watson / manage / monitor_quality / Monitor quality - Watson OpenScale

/ ibm-components / watson / manage / subscribe / Subscribe - Watson OpenScale

/ ibm-components / watson / store / Store model - Watson Machine Learning

/ ibm-components / watson / train / Train Model - Watson Machine Learning

/ json / Build_dict / Build dict

/ json / Build_list / Build list

/ json / Combine_lists / Combine lists

/ json / Get_element_by_index / Get element by index from JSON

/ json / Get_element_by_key / Get element by key from JSON

/ json / Query / Query JSON using JQ

/ keras / Train_classifier / from_CSV / Keras train classifier from csv

/ kubeflow / deployer / Kubeflow - Serve TF model

/ kubeflow / dnntrainer / Train FC DNN using TF

/ kubeflow / katib-launcher / Kubeflow - Launch Experiment

/ kubeflow / kfserving / Kubeflow - Serve Model using KFServing

/ kubeflow / launcher / Kubeflow - Launch TFJob

/ local / confusion_matrix / Confusion matrix

/ local / roc / ROC curve

/ ml_metrics / Aggregate_regression_metrics / Aggregate regression metrics

/ ml_metrics / Calculate_classification_metrics / from_CSV / Calculate classification metrics from csv

/ ml_metrics / Calculate_regression_metrics / from_CSV / Calculate regression metrics from csv

/ notebooks / Run_notebook_using_papermill / Run notebook using papermill

/ nuclio / delete / nuclio delete

/ nuclio / deploy / nuclio deploy

/ nuclio / invoker / nuclio invoker

/ pandas / Transform_DataFrame / in_ApacheParquet_format / Pandas Transform DataFrame in ApacheParquet format

/ pandas / Transform_DataFrame / in_CSV_format / Pandas Transform DataFrame in CSV format

/ presto / query / Presto Query

/ sample / keras / train_classifier / Keras - Train classifier

/ tables / Remove_header / Remove header

/ tensorflow / tensorboard / prepare_tensorboard / Create Tensorboard visualization

/ web / Download / Download data

/ XGBoost / Cross_validation_for_regression / from_CSV / Xgboost 5 fold cross validation for regression

/ XGBoost / Predict / Xgboost predict

/ XGBoost / Predict / from_ApacheParquet / Xgboost predict

/ XGBoost / Train / Xgboost train

/ XGBoost / Train / from_ApacheParquet / Xgboost train

/ XGBoost / Train_and_cross-validate_regression / from_CSV / Xgboost train and cv regression on csv

/ XGBoost / Train_regression_and_calculate_metrics / from_CSV / Xgboost train regression and calculate metrics on csv