Skip to content

Latest commit

 

History

History
689 lines (508 loc) · 24 KB

DEV-GUIDE.md

File metadata and controls

689 lines (508 loc) · 24 KB
Raw

Developing for Istio

This document helps you get started developing code for Istio. If you follow this guide and find some problem, please submit an issue, so we can improve the document. Also check Troubleshooting.

This document is intended to be relative to the branch in which it is found. It is guaranteed that requirements will change over time for the development branch, but release branches should not change.

Prerequisites

Istio components only have few external dependencies you need to set up before being able to build and run the code.

Setting up Go

Many Istio components are written in the Go programming language. To build, you'll need a Go development environment. If you haven't set up a Go development environment, please follow these instructions to install the Go tools.

Istio currently builds with Go 1.9

Setting up Docker

Istio has a Docker build system for creating and publishing Docker images. To leverage that you will need:

  • Docker platform: To download and install Docker follow these instructions.

  • Docker Hub ID: If you do not yet have a Docker ID account you can follow these steps to create one. This ID will be used in a later step when setting up the environment variables.

Setting up dep

Istio uses dep as the dependency management tool for its Go codebase. Dep will be automatically installed as part of the build. However, if you wish to install dep yourself, use the following command:

go get -u github.com/golang/dep/cmd/dep

Setting up Kubernetes

If you are working on Istio in a Kubernetes environment, we require Kubernetes version 1.7.3 or higher. Follow the steps outlined in the prerequisites section in the Istio Quick Start to setup a Kubernetes cluster with Minikube, or launch a cluster in IBM Cloud Container Service, Google Kubernetes Engine or OpenShift.

Additional steps for GKE

  • Add --no-enable-legacy-authorization to the list of gcloud flags to fully enable RBAC in GKE.

  • Update your kubeconfig file with appropriate credentials to point kubectl to the cluster created in GKE.

  gcloud container clusters get-credentials NAME --zone=ZONE
  • Make sure you are using static client certificates before fetching cluster credentials:
  gcloud config set container/use_client_certificate True

Additional notes for Minikube

Minikube version >= v0.22.3 is required for proper certificate configuration for GenericAdmissionWebhook feature. Get the latest version from minikube release page for your platform.

If you are using kubernetes with version >= v1.8.0 for kubeadm, the extra-config and RBAC flags will be enabled by default. Explicitly enabling them will actually cause errors. Please run:

minikube start --bootstrapper kubeadm

If you are using kubernetes with version < v1.8.0, you need to specify the extra-config and RBAC flags explicitly. Please run:

minikube start \
    --extra-config=apiserver.Admission.PluginNames="Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,GenericAdmissionWebhook,ResourceQuota" \
    --kubernetes-version=v1.7.5

To enable RBAC, add --bootstrapper kubeadm --extra-config=apiserver.Authorization.Mode=RBAC to minikube start command, in addition to the flags above.

Setting up environment variables

Set up your GOPATH, add a path entry for Go binaries to your PATH, set the ISTIO path, and set your GITHUB_USER used later in this document. These exports are typically added to your ~/.profile:

export GOPATH=~/go
export PATH=$PATH:$GOPATH/bin
export ISTIO=$GOPATH/src/istio.io # eg. ~/go/src/istio.io

# Please change HUB to the desired HUB for custom docker container
# builds.
export HUB="docker.io/$USER"

# The Istio Docker build system will build images with a tag composed of
# $USER and timestamp. The codebase doesn't consistently use the same timestamp
# tag. To simplify development the development process when later using
# updateVersion.sh you may find it helpful to set TAG to something consistent
# such as $USER.
export TAG=$USER

# If your github username is not the same as your local user name (saved in the
# shell variable $USER), then replace "$USER" below with your github username
export GITHUB_USER=$USER

# Specify which Kube config you'll use for testing. This depends on whether
# you're using Minikube or your own Kubernetes cluster for local testing
# For a GKE cluster:
export KUBECONFIG=${HOME}/.kube/config
# Alternatively, for Minikube:
# export KUBECONFIG=${GOPATH}/src/istio.io/istio/.circleci/config

Execute once, to create a directory for the Istio source trees.

mkdir -p $ISTIO

As the steps recommended in this section change the $PATH and environment, you will need to reload the environment.

Setting up a personal access token

This is only necessary for core contributors in order to push changes to the main repos. You can make pull requests without two-factor authentication but the additional security is recommended for everyone.

To be part of the Istio organization, we require two-factor authentication, and you must setup a personal access token to enable push via HTTPS. Please follow these instructions for how to create a token. Alternatively you can add your SSH keys.

Using the code base

Building the code

To build Pilot, Mixer, and Istio CA for your host architecture, run

make

This build command figures out what it needs to do and does not need any input from you.

To build those components with debugger information so that a debugger such as Delve can be used to debug them, run

make DEBUG=1

TIP: To speed up consecutive builds of the project, run the following command instead:

GOBUILDFLAGS=-i make

GOBUILDFLAGS=-i causes our build system to build with go build -i, that results in significant improvements of the overall build time. Note that the use of -i flag causes Go to cache intermediate results in $GOPATH/pkg/. Depending on the situation, this behaviour may be undesirable as Golang may not erase out of date artifacts from the cache. In such a situation, erase the contents of $GOPATH/pkg/ manually before rebuilding the code.

Building and pushing the containers

Build the containers in your local docker cache:

make docker

To build the containers with the debugger information so that they can be debugged with a debugger such as Delve, run

make DEBUG=1 docker

Push the containers to your registry:

make push

Building and pushing a specific container.

If you want to make a local change and test some component, say istio-ca, you could do:

Under istio/istio repo

pwd

The path should be

.../src/istio.io/istio

Set up environment variables HUB and TAG by

export HUB=docker.io/yourrepo
export TAG=istio-ca

Make some local change of CA code, then build istio-ca

bin/gobuild.sh istio_ca istio.io/istio/pkg/version ./security/cmd/istio_ca

Note: for other images, check Makefile for more info.

And move this file to docker_temp repo

cp istio_ca $GOPATH/out/linux_amd64/release/docker_temp

Push docker image

make push.docker.istio-ca

Building the Istio manifests

Use updateVersion.sh to generate new manifests with mixer, pilot, and ca_cert custom built containers:

install/updateVersion.sh -a ${HUB},${TAG}

Cleaning outputs

You can delete any build artifacts with:

make clean

Debug an Istio container with Delve

To debug an Istio container with Delve in a Kubernetes environment:

  • Locate the Kubernetes node on which your container is running.
  • Make sure that the node has Go tool installed as described in above.
  • Make sure the node has Delve installed.
  • Clone the Istio repo from which your debuggable executables have been built onto the node.
  • Log on to the node and find out the process id that you'd like to debug. For example, if you want to debug Pilot, the process name is pilot-discovery. Issue command ps -ef | grep pilot-discovery to find the process id.
  • Issue the command sudo dlv attach <pilot-pid> to start the debug session.

You may find this Delve tutorial is useful.

Alternatively, you can use Squash with Delve to debug your container. You may need to modify the Istio Dockerfile to use a base image such as alpine (versus scratch in Pilot Dockerfiles). One of the benefits of using Squash is that you don't need to install Go tool and Delve on every Kubernetes nodes.

Running tests

You can run all the available tests with:

make test

Note on Pilot unit tests: For tests that require systems integration, such as invoking the Envoy proxy with a special configuration, we capture the desired output as golden artifacts and save the artifacts in the repository. Validation tests compare generated output against the desired output. For example, Envoy configuration test data contains auto-generated proxy configuration. If you make changes to the configuration generation, you also need to create or update the golden artifact in the same pull request. The test library can automatically refresh all golden artifacts if you pass a special environment variable:

env REFRESH_GOLDEN=true make pilot-test

Getting coverage numbers

You can get the current unit test coverage numbers on your local repo by going to the top of the repo and entering:

make coverage

Auto-formatting source code

You can automatically format the source code to follow our conventions by going to the top of the repo and entering:

make format

Running the linters

You can run all the linters we require on your local repo by going to the top of the repo and entering:

make lint
# To run only on your local changes
bin/linters.sh -s HEAD^

Race detection tests

You can run the test suite using the Go race detection tools using:

make racetest

Adding dependencies

It will occasionally be necessary to add a new external dependency to the system. If the dependent Go package does not have to be pinned to a specific version, run dep ensure to update the Gopkg.lock files and commit them along with your code. If the dependency has to be pinned to a specific version, run

dep ensure -add github.com/foo/bar

The command above adds a version constraint to Gopkg.toml and updates Gopkg.lock. Inspect Gopkg.toml to ensure that the package is pinned to the correct SHA. Please pin to COMMIT SHAs instead of branches or tags.

You will need to commit the vendor/ change through a separate PR, please see https://github.com/istio/istio/wiki/Vendor-FAQ#how-do-i-add--change-a-dependency

About testing

Before sending pull requests you should at least make sure your changes have passed both unit and integration tests. We only merge pull requests when all tests are passing.

  • Unit tests should be fully hermetic
    • Only access resources in the test binary.
  • All packages and any significant files require unit tests.
  • Unit tests are written using the standard Go testing package.
  • The preferred method of testing multiple scenarios or input is table driven testing
  • Concurrent unit test runs must pass.

Working with CircleCI

We use CircleCI as one of the systems for continuous integration. Any PR will have to pass all CircleCI tests (in addition to Prow tests) before being ready to merge. When you fork the Istio repository, you will automatically inherit the CircleCI testing environment as well, allowing you to fully reproduce our testing infrastructure. If you have already signed up for CircleCI, you can test your code changes in your fork against the full suite of tests that we run for every PR.

Please refer to the wiki for a detailed guide on using CircleCI with Istio.

Writing reference docs

Our users depend on having quality documentation for our product. In addition to the prose you should be creating in the istio/istio.github.io repo, you need to also write quality reference documentation throughout the product.

Our reference documentation comes from two places:

  • Comments on proto files for our config formats and API definitions. The various protos in the istio/api repo, the Mixer adapter configuration protos, and the Mixer template definitions are all examples of this.

  • Cobra commands for our CLI docs. The various CLI commands we build generally use the Cobra framework to parse our command-lines. Cobra is also used to produce reference documentation for the individual commands.

Here's how this works:

  • Developers comment every element in proto files they author

  • Developers provide quality help & usage text in Cobra command definitions.

  • Whenever we compile protos, in addition to producing .pb.go files, our build system also produces .pb.html files which hold the documentation for the particular proto package.

  • Within the istio/istio.github.io repo, the script file scripts/grab_reference_docs.sh does the work necessary to harvest all the generated .pb.html files from the istio/api and istio/istio repos and installs them in the right place in the website hierarchy. The script also builds and runs the various CLI commands we have in order to extract their documentation, and copies their docs to the right place in the website hierarchy. Once the script is done running, the changes to the website just need to be pushed and istio.io will shortly thereafter reflect the changes.

About proto documentation

Writing proto documentation is a simple matter of adding comments to elements within the input proto files. You can put comments directly above individual elements, or to the right. For example:

// A package-level comment
package pkg;

// This documents the message as a whole
message MyMsg {
    // This documents this field
    // It can contain many lines.
    int32 field1 = 1;

    int32 field2 = 2;       // This documents field2
}

Comments are treated as markdown. You can thus embed classic markdown annotations within any comment.

Linking to types and elements

In addition to normal markdown links, you can also use special proto links within any comment. Proto links are used to create a link to other types or elements within the set of protos. You specify proto links using two pairs of square brackets such as:

// This is a comment that links to another type: [MyOtherType][MyPkg.MyOtherType]
message MyMsg {

}

The first square brackets contain the name of the type to display in the resulting documentation. The second square brackets contain the fully qualified name of the type or element being referenced, including the package name.

Annotations

Within a proto file, you should insert special comments which provide additional metadata to use in producing quality documentation. Within a package, include an unattached comment of the form:

// $title: My Title
// $overview: My Overview
// $location: https://mysite.com/mypage.html

$title provides a title for the generated package documentation. This is used for things like the title of the generated HTML. $overview is a one-line description of the package, useful for tables of contents or indexes. Finally, $location indicates the expected URL for the generated documentation. This is used to help downstream processing tools to know where to copy the documentation, and is used when creating documentation links from other packages to this one.

You can also use the $front_matter annotation to introduce new Jekyll front matter which can be useful for special tuning of the output. For example:

// $front_matter: order: 10

The above will include the front matter order: 10 in the generated Jekyll HTML document.

If a comment for an element contains the annotation $hide_from_docs, then the associated element will be omitted from the output. This is useful when staging the introduction of new features that aren't quite ready for use yet. The annotation can appear anywhere in the comment for the element. For example:

message MyMsg {
    int32 field1 = 1; // $hide_from_docs
}

The comment for any element can contain the annotation $class: <foo> which is used to insert a specific HTML class around the generated element. This is useful to give particular styling to particular elements. Common examples of useful classes include

message MyMsg {
    int32 field1 = 1; // $class: alpha
    int32 field2 = 2; // $class: beta
    int32 field3 = 3; // $class: experimental
}

The specific class used can be used to control the rendering of the element. At the moment, Istio doesn't provide any specific rendering controls for these classes, but it would be easy to add some. As shown in the example, we could readily have different rendering for alpha elements vs. beta vs. experimental. If you find this would be a useful feature, file an issue in istio/istio.github.io to introduce the CSS necessary to support the particular class you want to use.

Proto doc checklist

  • Ensure each package contains the appropriate $title, $overview, and $location annotations such that your package's docs are properly published to istio.io.

  • Ensure each package contains exactly one package-level comment. This comment is displayed at the top of the documentation page.

  • Ensure each element (message, enum, message field, enum value, etc) has good operator-centric documentation.

Git workflow

Below, we outline one of the more common Git workflows that core developers use. Other Git workflows are also valid.

Fork the main repository

  1. Go to https://github.com/istio/istio
  2. Click the "Fork" button (at the top right)

Clone your fork

The commands below require that you have $GOPATH set ($GOPATH docs). We highly recommend you put Istio's code into your GOPATH. Note: the commands below will not work if there is more than one directory in your $GOPATH.

cd $ISTIO
git clone https://github.com/$GITHUB_USER/istio
cd istio
git remote add upstream 'https://github.com/istio/istio'
git config --global --add http.followRedirects 1

Enable pre-commit hook

NOTE: The precommit hook is not functional as of 11/08/2017 following the repo reorganization. It should come back alive shortly.

Istio uses a local pre-commit hook to ensure that the code passes local tests before being committed.

Run

./bin/pre-commit
Installing pre-commit hook

This hook is invoked every time you commit changes locally. The commit is allowed to proceed only if the hook succeeds.

Create a branch and make changes

git checkout -b my-feature
# Make your code changes

Keeping your fork in sync

git fetch upstream
git rebase upstream/master

Note: If you have write access to the main repositories (e.g. github.com/istio/istio), you should modify your Git configuration so that you can't accidentally push to upstream:

git remote set-url --push upstream no_push

Committing changes to your fork

When you're happy with some changes, you can commit them to your repo:

git add .
git commit

Then push the change to your fork (typically called origin). When prompted for authentication, use your GitHub username as usual but the personal access token as your password if you have not setup ssh keys. Please follow these instructions if you want to cache the token.

git push origin my-feature

Creating a pull request

  1. Visit https://github.com/$GITHUB_USER/istio if you created a fork in your own github repository, or https://github.com/istio/istio and navigate to your branch (e.g. "my-feature").
  2. Click the "Compare" button to compare the change, and then the "Pull request" button next to your "my-feature" branch.

Getting a code review

Once your pull request has been opened it will be assigned to one or more reviewers. Those reviewers will do a thorough code review, looking for correctness, bugs, opportunities for improvement, documentation and comments, and style.

Very small PRs are easy to review. Very large PRs are very difficult to review. GitHub has a built-in code review tool, which is what most people use.

When to retain commits and when to squash

Upon merge, all Git commits should represent meaningful milestones or units of work. Use commits to add clarity to the development and review process.

Before merging a PR, squash any "fix review feedback", "typo", and "rebased" sorts of commits. It is not imperative that every commit in a PR compile and pass tests independently, but it is worth striving for. For mass automated fixups (e.g. automated doc formatting), use one or more commits for the changes to tooling and a final commit to apply the fixup en masse. This makes reviews much easier.

Please do not use force push after submitting a PR to resolve review feedback. Doing so results in an inability to see what has changed between revisions of the PR. Instead submit additional commits until the PR is suitable for merging. Once the PR is suitable for merging, the commits will be squashed to simplify the commit.