\nopagebreak and \penalty

Question

What is the difference between \nopagebreak and \penalty? (Other than the obvious fact that their arguments are measured on different scales.) When should one use one, and when the other?

Answer 1

Look at the definition of \nopagebreak (in latex.ltx):

\def\nopagebreak{\@testopt\@no@pgbk4}
\def\@no@pgbk #1[#2]{%
  \ifvmode
    \penalty #1\@getpen{#2}%
  \else
    \@bsphack
    \vadjust{\penalty #1\@getpen{#2}}%
    \@esphack
  \fi}

Discounting optional-argument crud, this basically says

If in vertical mode, issue \penalty\@getpen{#2}, otherwise put \penalty\@getpen{#2} in vertical mode after the current line of the current paragraph.

What does \@getpen do?

\def\@getpen#1{\ifcase #1 \z@ \or \@lowpenalty\or
         \@medpenalty \or \@highpenalty
         \else \@M \fi}

This basically says

If #1==0, return 0, if #1==1, return \@lowpenalty, if #1==2, return \@medpenalty, if #1==3, return \@highpenalty, if #1==4, return 10000.

How are \@lowpenalty etc. defined? Their definition is found in the class file:

\@lowpenalty   51
\@medpenalty  151
\@highpenalty 301

So, putting all together, the answer is

\penalty is a low-level command which just inserts a penalty value into the current output list, regardless of the current mode (horizontal or vertical). \nopagebreak is a user-level command which always adds a non-negative penalty in vertical mode. The concrete value of the penalty is "abstracted" in five levels from zero to infinite, and furthermore dependent on the document class used.

Answer 2

The \penalty command is a TeX primitive that inserts a penalty node into the current list (e.g., a paragraph) while \nopagebreak is a high-level LaTeX command that inspects its suroundings (to some extent) and does different things depending on whether it is used in vertical or in horizontal mode.

If you are in the middle of a paragraph, then saying \penalty 10000 is inserting a penalty that directs the linebreaking, but not one that affects the page breaking. In contrast \nopagebreak would insert more or less \vadjust{\penalty 10000} and it would do some additional magic to ensure that you don't end up with two spaces in that place.

So bottom line, as a user you shouldn't use \penalty at all, there are good reasons why LaTeX provides higher-level user commands. As a developer of a package, you may use penalty if you know what situation are in and of course using \penalty directly removes a tiny bit of overhead.