This directory contains tests and testing docs for Knative Serving:

• Unit tests currently reside in the codebase alongside the code they test
• End-to-end tests, of which there are two types:
  • Conformance tests in /test/conformance
  • Other end-to-end tests in /test/e2e
• Performance tests

The conformance tests are a subset of the end to end test with more strict requirements around what can be tested.

If you want to add more tests, see adding_tests.md.

presubmit-tests.sh is the entry point for both the end-to-end tests and the conformance tests

This script, and consequently, the e2e and conformance tests will be run before every code submission. You can run these tests manually with:

test/presubmit-tests.sh

Note that to run presubmit-tests.sh or e2e-tests.sh scripts, you'll need kubernetes kubetest installed:

go get -u k8s.io/test-infra/kubetest

To run all unit tests:

go test ./...

By default go test will not run the e2e tests, which need -tags=e2e to be enabled.

To run the e2e tests and the conformance tests, you need to have a running environment that meets the e2e test environment requirements, and you need to specify the build tag e2e.

go test -v -tags=e2e -count=1 ./test/conformance
go test -v -tags=e2e -count=1 ./test/e2e

To run one e2e test case, e.g. TestAutoscaleUpDownUp, use the -run flag with go test:

go test -v -tags=e2e -count=1 ./test/e2e -run ^TestAutoscaleUpDownUp$

These tests require:

1. A running Knative Serving cluster.
2. The namespace serving-tests:

   kubectl create namespace serving-tests
   
   kubectl label namespace serving-tests istio-injection=enabled    

3. A docker repo containing the test images

• By default the e2e tests against the current cluster in ~/.kube/config using the environment specified in your environment variables.
• Since these tests are fairly slow, running them with logging enabled is recommended (-v).
• Using --logverbose to see the verbose log output from test as well as from k8s libraries.
• Using -count=1 is the idiomatic way to disable test caching

You can use test flags to control the environment your tests run against, i.e. override your environment variables:

go test -v -tags=e2e -count=1 ./test/conformance --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo

If you are running against an environment with no loadbalancer for the ingress, at the moment your only option is to use a domain which will resolve to the IP of the running node (see #609):

go test -v -tags=e2e -count=1 ./test/conformance --resolvabledomain
go test -v -tags=e2e -count=1 ./test/e2e --resolvabledomain

Note: this is only required when you run conformance/e2e tests locally with go test commands.

The upload-test-images.sh script can be used to build and push the test images used by the conformance and e2e tests. It requires:

• DOCKER_REPO_OVERRIDE to be set
• You to be authenticated with your DOCKER_REPO_OVERRIDE
• docker to be installed

To run the script for all end to end test images:

./test/upload-test-images.sh

A docker tag may be passed as an optional parameter. This can be useful on Minikube in tandem with the --tag flag:

eval $(minikube docker-env)
./test/upload-test-images.sh any-old-tag

New test images should be placed in ./test/test_images.

These flags are useful for running against an existing cluster, making use of your existing environment setup.

Tests importing github.com/knative/serving/test recognize these flags:

• --kubeconfig
• --cluster
• --namespace
• --dockerrepo
• --tag
• --resolvabledomain
• --logverbose
• --emitmetrics

By default the tests will use the kubeconfig file at ~/.kube/config. If there is an error getting the current user, it will use kubeconfig instead as the default value. You can specify a different config file with the argument --kubeconfig.

To run the tests with a non-default kubeconfig file:

go test -v -tags=e2e -count=1 ./test/conformance --kubeconfig /my/path/kubeconfig
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig /my/path/kubeconfig

The --cluster argument lets you use a different cluster than your specified kubeconfig's active context. This will default to the value of your K8S_CLUSTER_OVERRIDE environment variable if not specified.

go test -v -tags=e2e -count=1 ./test/conformance --cluster your-cluster-name
go test -v -tags=e2e -count=1 ./test/e2e --cluster your-cluster-name

The current cluster names can be obtained by running:

kubectl config get-clusters

The --namespace argument lets you specify the namespace to use for the tests. By default, tests will use serving-tests.

go test -v -tags=e2e -count=1 ./test/conformance --namespace your-namespace-name
go test -v -tags=e2e -count=1 ./test/e2e --namespace your-namespace-name

The --dockerrepo argument lets you specify the docker repo from which images used by your tests should be pulled. This will default to the value of your DOCKER_REPO_OVERRIDE environment variable if not specified.

go test -v -tags=e2e -count=1 ./test/conformance --dockerrepo gcr.myhappyproject
go test -v -tags=e2e -count=1 ./test/e2e --dockerrepo gcr.myhappyproject

The default docker tag used for the test images is latest, which can be problematic on Minikube. To avoid having to configure a remote container registry to support the Always pull policy for latest tags, you can have the tests use a specific tag:

go test -v -tags=e2e -count=1 ./test/conformance --tag any-old-tag
go test -v -tags=e2e -count=1 ./test/e2e --tag any-old-tag

Of course, this implies that you tagged the images when you uploaded them.

If you set up your cluster using the getting started docs, Routes created in the test will use the domain example.com, unless the route has label app=prod in which case they will use the domain prod-domain.com. Since these domains will not be resolvable to deployments in your test cluster, in order to make a request against the endpoint, the test use the IP assigned to the service knative-ingressgateway in the namespace istio-system and spoof the Host in the header.

If you have configured your cluster to use a resolvable domain, you can use the --resolvabledomain flag to indicate that the test should make requests directly against Route.Status.Domain and does not need to spoof the Host.

The --logverbose argument lets you see verbose test logs and k8s logs.

go test -v -tags=e2e -count=1 ./test/e2e --logverbose

Running tests with the --emitmetrics argument will cause latency metrics to be emitted by the tests.

• To add additional metrics to a test, see emitting metrics.
• For more info on the format of the metrics, see metric format.