Golang Packaging Guidelines

This document details best practices for packaging Golang packages. Most of it is automated by an extensive use of macros.

go2rpm

go2rpm is tool that automates many of these steps. It is advisable to try go2rpm import_path first before attempting to write a SPEC by hand.

Import Path

In Golang, packages are referenced by full URLs. Since this URL is referenced in several places throughout the rpmspec, set the base import path as a global define at the top of the spec file

%global goipath     github.com/kr/pretty

All macros, including package name, source URL, will be computed from this value.

Take the time to identify it accurately. Changing it later will be inconvenient.

it may differ from the repository URL;
generally, the correct value will be the one used by the project in its documentation, coding examples, and build assertions;
use the gopkg import path for all code states when a project uses it.

If upstream confused itself after multiple forks and renamings, you will need to fix references to past names in the Go source files, unit tests included. Perform this fixing in %prep.

Naming

Source packages (src.rpm)

Golang source packages dedicated to providing code MUST be named after their main import path. This process is automated by the %{goname} macro. This macro will remove any capitalization, "go" keywords, and any duplication in the import path.

For example:
- the import path github.com/kr/pretty will become golang-github-kr-pretty
- the import path github.com/DATA-DOG/go-txdb will become golang-github-data-dog-txdb
- the import path github.com/gopherjs/gopherjs will become golang-github-gopherjs
The filename of spec MUST match the name of the package.

If you’re not sure what will the name processed by the %{goname} macro be, simply build the SRPM with:
```
fedpkg --release f31 srpm
```
The SRPM filename will be the one to use in your rpmspec and spec filename.
Source packages that provide a well-known application such as etcd MUST be named after the application. End users do not care about the language their applications are written in. But do not name packages after an obscure utility binary that happens to be built by the package.

Implementation: `%{gorpmname}`

%gometa uses the %{gorpmname} macro to compute the main %{goname} from %{goipath}.

%{gorpmname} can produce collisions

%{gorpmname} tries to compute human-friendly and rpm-compatible naming from Go import paths. It simplifies them, removing redundancies and common qualifiers. As a result it is possible for two different import paths to produce the same result. In that case, feel free to adjust this result manually to avoid the collision.

Go code packages: `%{goname}-devel`

In a source package dedicated to providing Go code

Packages that ship Go code in %{goipath} should be named %{goname}-devel. If your source package is already named %{goname} then:

%package devel
[…]
%description devel
[…]
%files devel -f devel.file-list

This has been automated by the %{gopkg} and %{gopdevelkg} macros described in the Package metadata section below.

In a another kind of source package

If your source package is named something other than %{goname}, you SHOULD use:

%package -n %{goname}-devel
[…]
%description -n %{goname}-devel
[…]
%files -n %{goname}-devel -f devel.file-list

Separate code packages

And, finally, if you wish to split the project Go code in multiple packages, you can compute the corresponding names with:

%global goname1 %gorpmname importpath1
[…]
%package -n %{goname1}-devel
[…]
%description -n %{goname1}-devel
[…]
%files -n %{goname1}-devel -f %{goname1}.file-list

See also the Dealing with cyclic dependencies chapter.

Do remember that for Go, each directory is a package. Never separate the .go files contained in a single directory in different packages.

Import path compatibility packages: `%{compat-%{oldgoname}-devel}`

When a project can be referenced under multiple import paths, due to forks, renamings, rehostings, organizational changes, or confusing project documentation, it is possible to generate compatibility sub-packages to connect code that uses one of the other import paths to the canonical one.

The canonical import path SHOULD always be the one referenced in the project documentation. However some projects do not document import path changes, and rely on HTTPS redirections (for example https://github.com/docker/docker → https://github.com/moby/moby). Such a redirection is a sufficient indicator the canonical import path has changed (but please make sure with upstream).

The new import path SHOULD be reflected in %{goipath} and compatibility import paths MUST be declared with the goaltipaths macro:

# A space-separated list of import paths to simulate.
%global goaltipaths

For example, the Go library github.com/Sirupsen/logrus was renamed github.com/sirupsen/logrus and the documentation reflects this new import path. The packager SHOULD thus request a renaming of his package with a new import path and a compatibility import path:

%global goipath     github.com/sirupsen/logrus
%global goaltipaths github.com/Sirupsen/logrus

If a project has gone through multiple rename, multiple compatibility import paths can be specified as well.

Never defer renamings

Packagers MUST NOT use import path compatibility sub-packages to alias the canonical import path to one of the previous namings. Packagers MUST apply upstream renaming choices to the main %{goipath} spec variable and everything that derives from it, such as %{goname}. Deferred renamings introduce friction with upstream and other packagers.

Go binary packages

The binaries produced by your rpmspec SHOULD generally be listed in the main package. However, if you want a more appropriate name or split binaries among different packages, you can create additional binary subpackage. Of course these package MUST NOT be noarch.

For example we can create the package bbolt that will contain the binary of the same name:

%package -n bbolt
[…]
%description -n bbolt
[…]
%files -n bbolt
%license LICENSE
%{_bindir}/bbolt

Versioning

Many Go libraries do not use package versions or have regular releases and are instead maintained in public version control. In this case, follow the standard Fedora version conventions. This means that often Go packages will have a version number of 0 and a release number like 0.10.20190319git27435c6.

Most of this process is automated by macros so that you don’t have to specify the release number yourself.

You first specify either a Version, tag or commit in the header.

Version:
%global tag
%global commit

Then use the %gometa macro:

%gometa

The %gometa macro will automatically process the %{?dist} tag of the Release field to take into account the commit if any.

Commits vs releases

You SHOULD package releases in priority. Please reward the projects that make an effort to identify stable code states. Only fall back to commits when the project does not release, when the release is more than six months old, or if you absolutely need one of the changes of a later commit. In the later cases please inform the project politely of the reason you needed to give up on their official releases. Promoting releases is a way to limit incompatible commit hell.

Go Language Architectures

To compile on various architectures, golang and gcc-go compilers are available. The golang compiler currently supports x86, x86_64, ppc64le, ppc64 (partially, see upstream issue#13192), s390x, armv7hl and aarch64.

Binaries SHOULD set ExclusiveArch so that we only attempt to build packages on those arches. This is now automatically added by the %gometa macro by leveraging the %{golang_arches} macro. Packagers can exclude %ix86 (see Changes/EncourageI686LeafRemoval) by passing -f to the %gometa macro. The -f flag tells %gometa to set ExclusiveArch: %{golang_arches_future} instead of ExclusiveArch: %{golang_arches}. %{golang_arches_future} includes the same architectures as {golang_arches} sans %ix86.

Dependencies

Packages MUST have BuildRequires: go-rpm-macros.

This is automated by the %gometa macro.

Automatic Dependency Generation

Most of the golang-* packages are source code only. The *-devel sub-package that includes the source code should explicitly have provides for the golang imports that it includes. These provides are automatically deduced from import paths.

Binary builds that include these imports will use them in BuildRequires, for example:

BuildRequires: golang(github.com/gorilla/context)

Bundled or unbundled

At the moment golang projects packaged in Fedora SHOULD be unbundled by default. It means projects are built from dependencies packaged in Fedora.

For some project it can be reasonable to build from bundled dependencies. Every bundling needs a proper justification.

BuildRequires

The BuildRequires of the project contains the dependencies needed by unit tests and binaries.

You can gather them manually with golist. For example:

 export GOPATH=/home/user/go
 export GO111MODULE=off
 export goipath="github.com/sirupsen/logrus"
 go get $goipath
 (sort -u | xargs -I{} echo "BuildRequires:  golang({})") <<< "$(
  golist --imported --package-path $goipath --skip-self
  golist --imported --package-path $goipath --skip-self --tests
 )"

outputs:

BuildRequires:  golang(github.com/stretchr/testify/assert)
BuildRequires:  golang(github.com/stretchr/testify/require)
BuildRequires:  golang(golang.org/x/crypto/ssh/terminal)

If automatic buildrequires are available on your build target, you can use the %go_generate_buildrequires macro in %generate_buildrequires:

%generate_buildrequires
%go_generate_buildrequires

This macro leverages golist to gather build dependencies and tests dependencies from the package source.

Testing

You MUST run unit tests. Due to the nature of Go development, especially the lack of use of semantic versioning, API breakages are frequent.These need to be detected early and dealt with upstream.

Some tests may be disabled, especially the following kinds of unit tests are incompatible with a secure build environment such as mock:

tests that call a remote server or API over the Internet,
tests that attempt to reconfigure the system,
tests that rely on a specific app running on the system, like a database or syslog server.

If a test is broken for some other reason, you can disable it the same way. However, you SHOULD also report the problem upstream. Remember to trace in a comment why each check was disabled, with links to eventual upstream problem reports.

Walkthrough

This chapter will present a typical Go spec file step by step, with comments and explanations.

Spec preamble: `%{goipath}`, `%{forgeurl}` and `%gometa`

Usual case

A Go package is identified by its import path. A Go spec file will therefore start with the %{goipath} declaration. Don’t get it wrong, it will control the behaviour of the rest of the spec file.

%global goipath    google.golang.org/api

If your package is hosted on a forge like GitHub, GitLab, Bitbucket or Pagure, the hosting of the Go package will be automatically deduced from this variable (typically by prefixing it with https://). If that is not the case, you need to declare explicitly the hosting URL with the %{forgeurl} macro

%global forgeurl    https://github.com/google/google-api-go-client

The %{forgeurl} declaration is followed by either Version, commit or tag. Use the combination that matches your use-case.

%global commit     2dc3ad4d67ba9a37200c5702d36687a940df1111

You can also define date to override the mtime of the Source archive.

Now that we have all the required variables, the %gometa macro can be run

%gometa

It will compute and set the following variables if they are not already set by the packager:

goname: an rpm-compatible package name derived from goipath
gosource: a URL that can be used as SourceX: value
gourl: a URL that can be used as URL: value

It will delegate processing to the %forgemeta macro for:

forgesource: a URL that can be used as SourceX: value
forgesetupargs: the correct arguments to pass to %setup for this source used by %forgesetup and %forgeautosetup
archivename: the source archive filename, without extensions
archiveext: the source archive filename extensions, without leading dot
archiveurl: the URL that can be used to download the source archive, without renaming
topdir: the source archive top directory (can be empty)
extractdir: the source directory created inside %{_builddir} after using %forgesetup, %forgeautosetup or %{forgesetupargs}
repo: the repository name
owner: the repository owner (if used by another computed variable)
shortcommit: the commit hash clamping used by the forge, if any
scm: the scm type, when packaging code snapshots: commits or tags
distprefix: the prefix that needs adding to dist to trace non-release packaging

Most of the computed variables are both overridable and optional.

Now we can add the remaining elements of the preamble. First, we can define a multiline description block shared between subpackages:

%global common_description %{expand:
cmux is a generic Go library to multiplex connections based on their payload.
Using cmux, you can serve gRPC, SSH, HTTPS, HTTP, Go RPC, and pretty much any
other protocol on the same TCP listener.}

This description MUST stay within 80 characters per line.

If you need specific devel summary and description, you can also define:

%global godevelsummary
%global godeveldescription

Then we MUST specify the license files that will be added to the devel subpackages:

%global golicenses: A space-separated list of shell globs matching the project license files.

And the possible documentation that SHOULD be included:

%global godocs: A space-separated list of shell globs matching the project documentation files. The Go rpm macros will pick up “.md” files by default without this.

You can also exclude files from %golicense and %godocs with:

%global golicensesex
%global godocsex

For example you might want to exclude INSTALL* files from %godocs as these should not be provided.

Source package metadata: `%{goname}`, `%{gourl}` and `%{gosource}`

We can declare the usual rpm headers, using the values computed by %gometa:

Name:      %{goname}
# If not set before
Version:
Release:   1%{?dist}
Summary:
License:
URL:       %{gourl}
Source:    %{gosource}

You can replace them with manual definitions. For example, replace %{gourl} with the project homepage if it exists separately from the repository URL. Be careful to only replace %{go*} variables when it adds value to the specfile and you understand the consequences. Otherwise you will just add maintenance-intensive discrepancies in the distribution.

BuildRequires

If they are not automatically generated, you can now add the dependencies needed for package building and unit testings:

BuildRequires:  golang(github.com/stretchr/testify/assert)
BuildRequires:  golang(github.com/stretchr/testify/require)
BuildRequires:  golang(golang.org/x/crypto/ssh/terminal)

See this section on how to get the BuildRequires list manually.

Description

Now add the main package description:

%description
%{common_description}

Package metadata

The process of declaring Go code packages has been automated with the %{gopkg} macro. It will automatically generate a %package and %description for all primary import paths and compatibility ones.

%gopkg

If you only need to generate Go devel subpackages without compat, use:

%godevelpkg

If necessary, you can then define one or multiple Go packages that will contain the binaries being built. See the previous Go binary packages section.

%prep: `%goprep` , removing vendoring

%goprep unpacks the Go source archives and creates the project “GOPATH” tree used in the rest of the spec file. It removes vendored (bundled) code: use the -k flag if you wish to keep this vendored code, and deal with the consequences in the rest of the spec.

%goprep

%goprep only performs basic vendoring detection. It will miss inventive ways to vendor code. Remove manually missed vendor code after the %goprep line.

%goprep will not fix upstream sources for you. Since all the macro calls that follow %goprep assume clean problem-free sources, you need to correct them just after the %goprep line:

replace calls to deprecated import paths with their correct value
patch code problems
remove dead code (some upstreams deliberately ship broken source code in the hope someone will get around to fix it)

You SHOULD send fixes and problem reports upstream. Any patch used SHOULD contain a link to an upstream bug report or merge request. At minimum, a comment SHOULD be added to explain the patch rationale.

When you package an import path, that participates in a dependency loop, you need bootstrapping to manage the initial builds: https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping For Go code, that means your bootstrap section should:

remove unit tests that import other parts of the loop
remove code that imports other parts of the loop

Sometimes one can resolve dependency loops just by splitting specific subdirectories in a separate -devel subpackage. See also Dealing with cyclic dependencies.

Automatic BuildRequires

If BuildRequires generator are supported, you can now add them to your build:

%generate_buildrequires
%go_generate_buildrequires

Packaging a binary: the %build section

If your package is a source package only, you can skip this %build section entirely.

Otherwise, you first need to identify manually the project parts that can be built, and how to name the result. Practically, it’s any directory containing a main() Go section. Nice projects put those in cmd subdirectories named after the command that will be built, which is what we will document here, but it is not a general rule. Sometimes the whole %goipath builds as a single binary.

for cmd in cmd/* ; do
  %gobuild -o %{gobuilddir}/bin/$(basename $cmd) %{goipath}/$cmd
done

Installing the packages

Source package installation

If you package a library, %gopkginstall will perform installation steps for all primary import paths and compatibility ones.

%gopkginstall

If you only need to install Go devel subpackages without compat, use:

%godevelinstall

Binary package installation

For binaries, we simply create the %{_bindir} directory in the buildroot and install the commands as executable in it:

install -m 0755 -vd                     %{buildroot}%{_bindir}
install -m 0755 -vp %{gobuilddir}/bin/* %{buildroot}%{_bindir}/

Be wary of command names which might already exist in Fedora. If you have any doubt, you can check if the command is already provided in Fedora:

dnf whatprovides --disablerepo="*" --enablerepo=rawhide "/usr/bin/$cmd"

If this is the case, use your best judgment to rename the command.

Running the unit tests: `%gocheck`

As said before, you MUST run unit tests in %check:

%gocheck

However it is often necessary to disable some of them. You have 3 exclusion flags to do so:

-d <directory>: exclude the files contained in <directory> non-recursively (subdirectories are not excluded)
-t <tree root>: exclude the files contained in <tree root> recursively (subdirectories are excluded)
-r <regexp>: exclude files matching <regexp>

Remember to document why a test has been disabled.

%files declaration

When shipping source code

The %gopkgfiles will process the file list produced in %install and add the necessary license and documentation files:

%gopkgfiles

When shipping binaries

Binaries are usually shipped in the main package. This package MUST include legal files and documentation associated with those binaries.

%files
%license LICENSE
%doc cmd/foo/README.md
%{_bindir}/*

Examples

Simple source package

golang-github-stretchr-testify.spec

# https://github.com/stretchr/testify
%global goipath         github.com/stretchr/testify
Version:                1.2.2

%gometa

%global common_description %{expand:
Golang set of packages that provide many tools for testifying
that your code will behave as you intend.

Features include:

 - Easy assertions
 - Mocking
 - Testing suite interfaces and functions}

%global golicenses    LICENSE

Name:           %{goname}
Release:        1%{?dist}
Summary:        Tools for testifying that your code will behave as you intend
License:        MIT
URL:            %{gourl}
Source:         %{gosource}

BuildRequires: golang(github.com/davecgh/go-spew/spew)
BuildRequires: golang(github.com/pmezard/go-difflib/difflib)
BuildRequires: golang(github.com/stretchr/objx)

%description
%{common_description}

%gopkg

%prep
%goprep

%install
%gopkginstall

%check
%gocheck

%gopkgfiles


%changelog
* Thu Mar 21 22:20:22 CET 2019 Robert-André Mauchin <[email protected]> - 1.2.2-1
- First package for Fedora

Handling package renames

golang-github-sirupsen-logrus.spec

# https://github.com/sirupsen/logrus
%global goipath         github.com/sirupsen/logrus
Version:                1.4.0

%gometa

%global goaltipaths     github.com/Sirupsen/logrus

%global common_description %{expand:
Logrus is a structured logger for Go (golang), completely API compatible with
the standard library logger.}

%global golicenses    LICENSE
%global godocs        *.md

Name:           %{goname}
Release:        1%{?dist}
Summary:        Structured logger for Go
License:        MIT
URL:            %{gourl}
Source:         %{gosource}

BuildRequires: golang(golang.org/x/crypto/ssh/terminal)
BuildRequires: golang(github.com/stretchr/testify/assert)

%description
%{common_description}

%gopkg

%prep
%goprep

%install
%gopkginstall

%check
%gocheck

%gopkgfiles


%changelog
* Wed Oct 31 2018 Robert-André Mauchin <[email protected]> - 1.4.0-1
- First package for Fedora

Simple binary package

golang-github-boltdb-bolt.spec

# https://github.com/square/go-jose
%global goipath         gopkg.in/square/go-jose.v2
%global forgeurl        https://github.com/square/go-jose
Version:                2.1.9

%gometa

%global common_description %{expand:
Package jose aims to provide an implementation of the Javascript Object
Signing and Encryption set of standards. This includes support for JSON Web
Encryption, JSON Web Signature, and JSON Web Token standards.}

%global golicenses    LICENSE
%global godocs        *.md

%global godevelheader %{expand:
# The devel package will usually benefit from corresponding project binaries.
Requires:  %{name} = %{version}-%{release}
}

Name:           %{goname}
Release:        1%{?dist}
Summary:        An implementation of JOSE standards (JWE, JWS, JWT) in Go
# Detected licences
# - *No copyright* Apache License (v2.0) at 'LICENSE'
# json/ is BSD-3-Clause
License:        Apache-2.0 AND BSD-3-Clause
URL:            %{gourl}
Source:         %{gosource}

BuildRequires: golang(golang.org/x/crypto/ed25519)
BuildRequires: golang(golang.org/x/crypto/pbkdf2)
BuildRequires: golang(github.com/stretchr/testify/assert)
BuildRequires: golang(gopkg.in/alecthomas/kingpin.v2)

%description
%{common_description}

%gopkg

%prep
%goprep

%build
for cmd in jose-util jwk-keygen; do
  %gobuild -o %{gobuilddir}/bin/$(basename $cmd) %{goipath}/$cmd
done

%install
%gopkginstall
install -m 0755 -vd                     %{buildroot}%{_bindir}
install -m 0755 -vp %{gobuilddir}/bin/* %{buildroot}%{_bindir}/

%check
%gocheck

%files
%license %{golicenses}
%doc %{godocs}
%{_bindir}/*

%gopkgfiles

%changelog
* Thu Mar 21 21:59:10 CET 2019 Robert-André Mauchin <[email protected]> - 2.1.9-1
- First package for Fedora

Additional resources

Advanced uses cases

Additional annotated templates

Want to help? Learn how to contribute to Fedora Docs ›

Golang Packaging Guidelines

go2rpm

Import Path

Naming

Source packages (src.rpm)

Implementation: %{gorpmname}

Go code packages: %{goname}-devel

In a source package dedicated to providing Go code

In a another kind of source package

Separate code packages

Import path compatibility packages: %{compat-%{oldgoname}-devel}

Go binary packages

Versioning

Go Language Architectures

Dependencies

Automatic Dependency Generation

Bundled or unbundled

BuildRequires

Testing

Walkthrough

Spec preamble: %{goipath}, %{forgeurl} and %gometa

Usual case

Source package metadata: %{goname}, %{gourl} and %{gosource}

BuildRequires

Description

Package metadata

%prep: %goprep , removing vendoring

Automatic BuildRequires

Packaging a binary: the %build section

Installing the packages

Source package installation

Binary package installation

Running the unit tests: %gocheck

%files declaration

When shipping source code

When shipping binaries

Examples

Simple source package

Handling package renames

Simple binary package

Additional resources

Advanced uses cases

Additional annotated templates

Implementation: `%{gorpmname}`

Go code packages: `%{goname}-devel`

Import path compatibility packages: `%{compat-%{oldgoname}-devel}`

Spec preamble: `%{goipath}`, `%{forgeurl}` and `%gometa`

Source package metadata: `%{goname}`, `%{gourl}` and `%{gosource}`

%prep: `%goprep` , removing vendoring

Running the unit tests: `%gocheck`