Externalize PGF

Question

I have a bunch of figures made using Matplotlib's PGF backend. To avoid regenerating all these plots with Matplotlib if there is a style change, I prefer to just \input the PGF files directly in my document. This works fine, except it means arXiv disallows my 16 MB of sources.

Is there a way I can externalize the PGF files just like I would TikZ files? I'd really like to avoid regenerating all the plots, as this is quite time-consuming.

Answer 1

A workaround could be to generate a separate PDF with the pgfpictures, one on each page, and replace the inputs with \includegraphics. The standalone class makes this easy, for example:

\documentclass{standalone}
\usepackage{pgf}
\standaloneenv{pgfpicture}
\begin{document}
\input{onefigure.pgf}
\input{anotherfigure.pgf}
\end{document}

Save this as e.g. figs.tex and compile. In your manuscript, say \includegraphics[page=1]{figs} instead of \input{onefigure}, and similar for the others. Of course you could create one PDF per figure if that is preferable.

Answer 2

I had the same problem, and wrote my own package to deal with it automatically. You can find it here. If you don't want to install the package, you can essentially put this code into your preamble:

\makeatletter
\edef\@figdir{_pgfcache}

\catcode`\#=11
\catcode`\|=6
\newcommand{\@basicpgfpreamble}[1]{%
    \unexpanded{%
        \documentclass{standalone}^^J
        \usepackage{pgf}^^J
        \let\oldpgfimage\pgfimage^^J
        \renewcommand{\pgfimage}[2][]{\oldpgfimage[#1]{|1/#2}}^^J
    }%
}
\catcode`\#=6
\catcode`\|=12

\let\@pgfpreamble\@basicpgfpreamble

\newcommand{\setpgfpreamble}[1]{%
    \renewcommand{\@pgfpreamble}[1]{\@basicpgfpreamble{##1}\unexpanded{#1}}
}

\newcounter{@pgfcounter}
\newwrite\@pgfout
\newread\@pgfin

\newcommand{\importpgf}[3][]{%
    \IfFileExists{#2/#3}{}{\errmessage{importpgf: File #2/#3 not found}}%
    \edef\@figfile{\jobname-\the@pgfcounter}%
    \providecommand{\@writetempfile}{}%
    \renewcommand{\@writetempfile}[1]%
    {%
        \immediate\openout\@pgfout=##1%
        \immediate\write\@pgfout{\@pgfpreamble{#2}}%
        \immediate\write\@pgfout{\string\begin{document}}%
        \immediate\openin\@pgfin=#2/#3%
        \begingroup\endlinechar=-1%
            \loop\unless\ifeof\@pgfin%
                \readline\@pgfin to \@fileline%
                \ifx\@fileline\@empty\else%
                    \immediate\write\@pgfout{\@fileline}%
                \fi%
            \repeat%
        \endgroup%
        \immediate\closein\@pgfin%
        \immediate\write\@pgfout{\string\end{document}}%
        \immediate\closeout\@pgfout%
    }%
    \def\@compile%
    {%
        \immediate\write18{pdflatex -interaction=batchmode -output-directory="\@figdir" \@figdir/\@figfile.tex}%
    }%
    \IfFileExists{\@figdir/\@figfile.pdf}%
    {%
        \@writetempfile{\@figdir/tmp.tex}%
        \edef\@hashold{\pdfmdfivesum file {\@figdir/\@figfile.tex}}%
        \edef\@hashnew{\pdfmdfivesum file {\@figdir/tmp.tex}}%
        \ifnum\pdfstrcmp{\@hashold}{\@hashnew}=0%
            \relax%
        \else%
            \@writetempfile{\@figdir/\@figfile.tex}%
            \@compile%
        \fi%
    }%
    {%
        \@writetempfile{\@figdir/\@figfile.tex}%
        \@compile%
    }%
    \IfFileExists{\@figdir/\@figfile.pdf}%
    {\includegraphics[#1]{\@figdir/\@figfile.pdf}}%
    {\errmessage{Error during compilation of figure #2/#3}}%
    \stepcounter{@pgfcounter}%
}
\makeatother

This provides you with two commands, \setpgfpreamble and \importpgf. \importpgf does two things:

1. Checks whether a pre-compiled PDF version of the figure is already present in a folder called _figurecache (you might have to create this one). If so, it just includes that PDF with \includegraphics. If not, it calls pdflatex to compile the figure, then includes it. The figure is also re-compiled if the source of the figure changes (checked via the MD5 checksum of the file).

2. It patches the \pgfimage command inside your PGF figures, so you can include rasterized parts created with Matplotlib even if they reside in a subfolder (as described here).

You can use the command like this:

\importpgf{path/to/file}{myfigure.pgf}

With \setpgfpreamble, you can define additional packages that are needed to compile your PGF images (such as fonts or math packages):

\setpgfpreamble{%
    \usepackage{libertine}
    \usepackage{amsmath}
    \usepackage{siunitx}
}

The only dependencies are the standalone and pgf packages. The first compilation will take a long time if you have a lot of figures, but subsequent compilations will be much faster.

Caveats: It's probably not very portable (only works if you're using PDFLaTeX; the pdflatex executable must be on your path; and you have to compile with -shell-escape). Also, error-checking is limited, so if you run into trouble, check the logs in the _figurecache folder for errors (you might have forgotten to include a package with \setpgfpreamble).

Answer 3

This is a hacky solution that probably will not work in any cases. The idea is to replace pgfpicture environment with tikzpicture one, and then to use externalization.

For this you can :

1. read your file in a macro using catchfile package;
2. patch this macro using patchcmd from etoolbox to replace pgfpicture with tikzpicture;
3. put all this in a macro \mypgfimport that set \tikzsetfigurename.

Here is the code:

% ------------------------------------
% example file blablatest.pgf
\begin{filecontents}{blablatest.pgf}
\begin{pgfpicture}
\pgfpathsvg{M 0 0 l 20 0 0 20 -20 0 q 10 0 10 10
t 10 10 10 10 h -50 z}
\pgfusepath{stroke}
\end{pgfpicture}
\end{filecontents}
% ------------------------------------
\documentclass[border=7mm]{standalone}
\usepackage{catchfile}
\usepackage{etoolbox}
\usepackage{tikz}
\usepgflibrary{svg.path}
\usetikzlibrary{external}\tikzexternalize

\def\mypgfimport#1{%
  \tikzsetfigurename{#1}
  \CatchFileDef{\tempmacro}{#1.pgf}{}%
  \patchcmd{\tempmacro}{\begin{pgfpicture}}{\begin{tikzpicture}[red]}{}{}%
  \patchcmd{\tempmacro}{\end{pgfpicture}}{\end{tikzpicture}}{}{}%
  \tempmacro%
}

\begin{document}
  \mypgfimport{blablatest}
\end{document}