Source Control, Git, and

Industrial Automation

TABLE OF CONTENTS

1. What is Source Control?

2. Source Control Benefits
3. Adopting Source Control
4. Source Control vs. "The Archive Folder"
5. Not a Revolution, Just Responsible
6. Available Tools + Challenges

1
 Overview

Source control systems have become ubiquitous in the IT world

but have not become a major component of operational
...building systems technology (in particular, PLC programming).
without source control There is an increasing, popular movement in the industrial
feels like “walking a controls world to bring modern source control systems to PLC
programming. This document will review the benefits of these
tightrope without a net.” systems and why people have begun to push for their adoption
across many industrial organizations.

We argue here that source control isn’t a revolutionary concept,

but instead provides multiple tangible benefits to organizations
and should be strongly considered given the relative ease of
adoption and low implementation cost. When benefits include
reduced downtime, improvements to engineering efficiency,
and greatly optimized workflows these systems easily provide
ROI for any organization.

More than that, they ultimately improve the development

experience for controls engineers and automation experts. Once
familiar with these systems, we have heard from engineers that
the idea of building systems without source control feels like
“walking a tightrope without a net.”

2
 What is Source Control?

Put simply, a source control system is really just an advanced file management
system that enables people and teams to manage the revision history of their
documents.

Source control systems tend to feature:

REVISION HISTORY
This enables people to take meaningful snapshots of their files which they can
Git - a source control
system created by the
easily retrieve or return to.
creator of Linux.
VERSION CONTEXT
This enables teams to understand who changed what and why. Often these
systems require credentials that enable traceability.

SHARED REPOSITORIES
Centralized storage systems (either in the cloud or on-premise) which allows
teams to access relevant files.
...the source
code files for ACCESS CONTROLS
the application Mechanisms to enable access to different files with different permissioning,
you are reading such as read and write access for different stakeholders.
this in are more
FILE REVIEW
than likely
Mechanisms to enable teams to review changes before they are committed to
stored in a Git the shared repository.
repository.
These are common sense features that build on top of concepts that many
controls professionals are familiar with such as a shared network drive or cloud
hosted storage. The most popular source control system is Git, which was
invented by the creator of the Linux operating system, and has been adopted by
millions of engineers in the last 15-years. In fact, the source code files for the
application you are reading this in are more than likely stored in a Git repository.

3
Source Control Benefits Adopting Source Control

The adoption of a strong source control system provides

numerous benefits to organizations that can be mapped to Figure 1.
ROI. Code review with
differentials of routines.
OPERATIONAL UPTIME
Possibly the biggest benefit of strong version control
practices is the ability to understand what code changed in
the case of unplanned downtime. Investment in version
control systems can enable teams to easily recognize who
made a change to a system, revert that change, and even
understand the context of that change. When the cost of
unplanned downtime is on average $250,000 / hour, every
minute counts in terms of root causing and mitigating an
issue.

One of the major benefits of source control systems is that

ENGINEERING EFFICIENCY
they typically integrate at the file system layer. This means
Source control systems unlock new efficiencies between
that technical adoption is straightforward. We’ve seen
engineers within the same team and organization. For
companies fully migrate to a modern source control system
example, we’ve seen organizations invest in extensive AOI
in less than an hour.
libraries, templated “master” copies of their machine builds,
and share reusable components that enable them to REGULAR COMMITS
quickly kick off new projects. A strong version control Engineers need to develop a best practice of always
practice enables teams to ensure that all of their members committing changes to their source control system before
are using the correct version of these reusable components deploying to production.
and can even provide a centralized system for
documentation. CODE REVIEW
Engineering teams can begin to develop a code review
AVOID COPY + PASTE best practices that creates a record of feedback on code
Good version control systems will integrate directly with a changes and enables deeper understanding of what
computer’s file system so that you do not have to copy and changed and why certain decisions were made (see figure
paste files in order to store back-ups. They’ll automatically 1).
pick up on and store meaningful changes. This means
you’re less likely to lose a file. One controls engineer told REUSABLE CODE
us “my hard drive failed literally the day after adopting a Engineers can invest in centralized repositories and
version control system.” Fortunately, they were backed up documentation to provide reusable code for their team and
in a central repository! organization.

AUDITING + TRACEABILITY CODE SHARING

Increasingly many contracts and regulatory environments Engineers can invest in access management protocols to
require strong version control practices. Organizations that share their code with external and internal stakeholders in
seek SOC 2 compliance have to have a system for code ways that protect their IP but also allow them to accelerate
review. Many industrial contracts require suppliers to and improve their collaboration.
provide first-order traceability as part of their contract.
We’ve heard of organizations who've considered WORKFLOWS
implementing version control practices for auditing for Very advanced engineering teams will invest in automated
multi-million dollar contracts. practices around code changes such as internal reporting
and even integration with their deployment model.
While high degrees of security and the need for self-hosted
solutions can make version control systems expensive to Overall, the basics of source control are extremely easy to
implement for some enterprise manufacturers, most adopt, but strong version management systems (like Git)
reasonable version control systems cost in the hundreds of can scale to organizations of virtually any size with virtually
dollars a month for a single team to adopt. any sort of collaborative workflow.

4
Source Control vs. “The Archive Folder”

In a recent Reddit post on source control systems a commenter made this

point:

“I don’t get it - this is so you can keep track of versions of your PLC
program? At my plant, we have a folder named “archive” for each
project. We just name the old project with the date and save it before
When we asked [...] “how
making edits, etc.” did you get people to
A responder gave a “quick and dirty” explanation as to why a true source finally adopt [source
control system is so much more powerful than an “archive folder.”
control]?” He simply
1. Only saves the differences between versions saving space.
responded, “bad things
2. Keeps a history of the changes and who made them
3. Allows multiple people to work on different parts of a project at
happening.”
the same time and merge them together without manually going
through and splicing in code
4. On commits must provide a message so helps keep track of why
a change was made.

To expand on their excellent answer, we distinguish between a “backup

system” and a “source control system.” A backup system provides you
with regular backups of a production system. This is useful, but it’s
difficult to know what has changed, who has changed it, and when it
changed.

With a proper source control system, it becomes easy to trace meaningful

changes and thus root cause regressions and reduce production
downtime. Furthermore, source control systems often provide powerful
mechanisms for code sharing and collaboration which streamline
operations and save engineers time.
In a typical archive folder, there is no
context about changes except for file name Ultimately, these sorts of arguments against source control are not new.
changes. We spoke with a principal software engineer at a major technology
company who explained the history of adoption of Git at Microsoft. When
we asked him “how did you get people to finally adopt [source
control]?” He simply responded, “bad things happening.”

Good source control practices save engineers when they need it the
most.

5
Not a Revolution, Just Responsible

Recently an engineer commented in an online discussion about source control, “I have been using my own system to manage files for
years, I don’t understand why this is revolutionary.”

We look at this through a different lens. It’s best captured by an embedded systems programmer who over the course of a year
convinced their team to adopt source control practices. He eventually convinced his team by telling them that it was “frankly
irresponsible” to not use version control.

In a world where controls professionals are often responsible for multi-million dollar production environments, and where the cost
of downtime can easily be hundreds of thousands of dollars or even human life, we believe that organizations simply cannot afford
to not adopt systems that will easily allow them to identify who changed what, where, when, and why.

Available Tools + Challenges

INDUSTRIAL FILES
For many industries, the standard for 15-years has been “Git,” Industrial file types were designed well before modern version
a distributed source control system that makes collaboration control systems. While some systems have moved to XML,
around code extremely easy. Git is available for free as an these are still hard for engineers to parse and understand.
open source product. Furthermore, common file types like ACD files for Allen-Bradley
PLCs are still entirely in binary, and are not human
However, Git by itself has some major downsides that make comprehensible.
simple adoption by controls engineering orgs difficult:

COMPLEXITY
Git is an extremely powerful tool with a very intense learning
curve. For most control system professionals, just learning
the command line is a barrier to entry, especially on windows
machines which require special plug-ins to work with a tool
originally designed for Linux.

From left to right: GitHub, GitLab, and Copia

HOSTING
To make Git truly work, it’s important to have a remote server
that hosts a Git repository. Maintaining and managing a Git The complexity of using and difficulty of hosting Git is a well-
server for a standard team is expensive, and often not worth known problem, with major Git services such as Github and
the benefit that it provides. Gitlab becoming multi-billion dollar companies as they have
solved this problem for millions of engineers. However, the
VISUAL PROGRAMMING problems specific to industrial automation have thus far
Whereas many programming languages are entirely text- impeded adoption in industrial automation broadly.
based the IEC 61131 specification provides definitions for
multiple visual programming languages. Ladder logic and While it’s possible to introduce a Git-based workflow on top of
FBD are nearly ubiquitous in the controls engineering world, Github or Gitlab, only recently have companies begun to
and that is likely not going to change any time soon. Certain solve this problem for controls professionals. In particular,
features in Git, such as showing the “diff” or “difference” companies like Copia Automation provide a Git-based
between text based files entirely breaks in the context of workflow built directly to support controls engineers with first
visual programming paradigms. This makes features such as order support for IEC 61131 languages and vendor-specific
version history that presents diffs to understand changes less industrial file types.
meaningful.

6
 Conclusion

We hope that [...] we Source control systems are easy to adopt, affordable, and
provide immediate benefits to organizations in terms of
begin to see major operational uptime and efficiency. Adoption of source control
adoption of powerful systems have been curtailed by the lack of support for industrial
source control tools for use cases. We hope that as we move into the new decade of
the foundational industrial automation, we begin to see major adoption of powerful
source control tools for the foundational systems that power the
systems that power the industrial economy.
industrial economy.

Copia Automation

Copia Automation is a company dedicated to building developer

tools to improve efficiency, reduce time to market, and improve
ROI for engineers and organizations in industrial automation.

To learn more visit us at our website and feel free to email with
any questions or comments at [email protected].

Specifications subject to change without notice. Copia Automation. All rights reserved. 7