Computational Tractability: The View From Mars: The Problem of Systematically Coping With Computational Intractability
Computational Tractability: The View From Mars: The Problem of Systematically Coping With Computational Intractability
Computational Tractability: The View From Mars: The Problem of Systematically Coping With Computational Intractability
1. Introduction
There are two dierent ways that one can view the theory of parameterized
complexity. The easiest is as a kind of \rst aid" that can sometimes be
applied to problems that are NP-hard, PSPACE-hard or undecidable. That
is, it can be viewed as a potential means of coping with intractability as it
is classically diagnosed.
The second way that one can view parameterized complexity is as a
fundamentally richer and generally more productive primary framework for
problem analysis and algorithm design, including the design of heuristic and
approximation algorithms.
It is interesting and humbling to consider the big picture of the development of theoretical computer science over the past 30 years. Above all, this
seems to be dominated by continent-sized discoveries and cartography of
various kinds of computational intractability. For some time, the practical
folks portrayed in the amusing cartoons of Garey and Johnson[GJ79], Chapter 1, have been somewhat disappointed with theoretical computer science,
and there have been numerous incidents of \restless drums" in the jungles
Rodney G. Downey. Research supported by a grant from the United States/New
Zealand Cooperative Science Foundation and by the New Zealand Marsden Fund for
Basic Science.
Michael R. Fellows. Research supported by the National Science and Engineering
Research Council of Canada.
This a shortened version of a survey that appeared in the AMS-DIMACS Proceedings
Volume, The Future of Discrete Mathematics, edited by F. Roberts and J. Nesetril .
1
Example 1: Yet Another Call for Reform. One of the plenary addresses at the AAAI meeting in 1996 was concerned with the broad theme
of how computer science practice and theory interact [DKL96]. The discussion in [DKL96] can be summarized as:
like to shoot the messenger who brings so much bad news! In this dicult
situation, computer science theory has articulated a few general programs
for systematically coping with the ubiquitous phenomena of computational
intractability. We list these famous basic approaches, and add to this list
our less well known candidate.
The idea of focusing on average-case as opposed to worst-case analysis
of problems.
The idea of settling for approximate solutions to problems, and of
looking for ecient approximation algorithms.
The idea of using randomization in algorithms.
2. Fixed-Parameter Tractability
(2) The sum over the edges uv of T of the Hamming distance between the
labels l(u) 2 f0; 1gn and l(v ) 2 f0; 1gn is at most m.
The Maximum Agreement Subtree Problem (MAST)
Instance: A set of rooted trees T1; :::; Tr (r 3) with the leaf set of each Ti
labeled 1:1 with a set of species X , and a positive integer k.
Parameter: k
Question: Is there a subset S X of size at most k such that Ti restricted
to the leaf set X 0 = X ? S is the same (up to label-preserving isomorphism
and ignoring vertices of degree 2) for i = 1; :::; r?
All of these problems are NP-complete and they are described above in
the standard way for the parameterized complexity framework. Part of
the input (which may be some aggregate of various aspects) is identied
as the parameter for the problem specication. (In order to consider a
parameterized problem classically, just ignore the parameter part of the
specication. Note that a single classical problem may give rise to many
dierent parameterized problems.) All of the four problems above can be
solved in time O(nf (k)) by simple brute force algorithms.
Papadimitriou and Yannakakis showed that Vertex Cover can be
solved in time O(3k n) [PY96]. Balasubramanian, Fellows and Raman gave
an algorithm with running time O((53=40)kk2 + kn) [BFR98]. It is possible
to do even better. Note that since the exponential dependence on the parameter k in the last expression is additive, Vertex Cover is well-solved for
input of any size so long as k is no more than around 60. The dierence between the situation for Dominating Set and Vertex Cover is displayed
in Table 1.
n = 50
n = 100
n = 150
k=2
625
2,500
5,625
k = 3 15,625 125,000 421,875
k = 5 390,625 6,250,000 31,640,625
k = 10 1:9 1012 9:8 1014 3:7 1016
k = 20 1:8 1026 9:5 1031 2:1 1035
k +1
Table 1. The Ratio n2k n for Various Values of n and k.
There are myriad ways in which numbers that are small or moderately large
(e.g., k 40) arise naturally in problem specications. For example, graph
linear layout width metrics are of interest in VLSI layout and routing problems and have important applications for width values of k 10. Interval
graphs of pathwidth k 10 have applications in DNA sequence reconstruction problems [BDFHW95]. Logic and database problems frequently are
dened as having input consisting a formula (which may be small and relatively invariant), and some other structure (such as a database) which is
typically quite large and changeable. Formula size, or other aspects of formula structure may be a relevant parameter [Yan95]. Hardware constraints
are a common source of natural parameters. The number of processors or
machines to be scheduled may be bounded by a value such as k 20. The
number of degrees of freedom in a robot motion-planning problem is commonly in the range k 10. The number of wiring layers in VLSI chip
manufacture is typically bounded by k 30. Important distributional parameters may also arise in ways that are not at all obvious. Thorup, for
example, has shown that the
ow graphs of structured programs for the
major computer languages have treewidth k 7 [Th97].
are as follows.
Denition. A parameterized language L is a subset L . If L
is a parameterized language and (x; y ) 2 L then we will refer to x as the
main part, and refer to y as the parameter. It makes no dierence to the
theory and is occasionally more convenient to consider that y is an integer,
or equivalently to dene a parameterized language to be a subset of IN .
An Illustrative Non-Example. It can be helpful to observe how parametric transformations dier from ordinary polynomial-time reductions.
Recall that for a graph G = (V; E ) on n vertices, a set of vertices V 0 V is
a k-clique in G if and only if V ? V 0 is a vertex cover in the complementary
graph G0 where vertices are adjacent if and only if they are not adjacent in G.
This gives an easy polynomial-time reduction of the naturally parameterized
Clique problem to the naturally parameterized Vertex Cover problem,
transforming the instance (G; k) of Clique into the instance (G0; k0) of Vertex Cover. But this is not a parametric transformation, since k0 = n ? k
is not purely a function of k. The available evidence suggests that there is
no parametric transformation in this direction.
An Illustrative Example. There is a fairly elaborate parametric transformation from the naturally parameterized Clique problem to the naturally
parameterized Dominating Set problem, mapping (G; k) to (G0; k0) where
k0 = 2k [DF95, DF98]. The evidence is that there is no such parametric
transformation in the other direction.
The essential property of parametric transformations is that if L transforms to L0 and L0 2 FPT , then L 2 FPT . This naturally leads to a completeness program based on a hierarchy of parameterized problem classes:
FPT W [1] W [2] W [SAT ] W [P ] AW [P ] XP
The parameterized analog of NP is W [1], and W [1]-hardness is the basic
evidence that a parameterized problem is likely not to be xed-parameter
tractable. The k-Step Halting Problem for Nondeterministic Turing Machines is W [1]-complete (where the amount of nondeterminism at
each step of the Turing machine computation is unbounded). Since the
q(n)-Step Halting Problem is essentially the dening problem for NP,
the analogy between NP and W [1] is quite strong. The parameterized complexity class W [P ] is an analog of PSPACE [ADF95]. The class XP is the
set of all parameterized languages L for which there are functions f and g
and an algorithm to determine whether (x; k) 2 L in time f (k)jxjg(k).
How Compelling is Fixed-Parameter Tractability? The notion of
xed-parameter tractability is the basic concept of the theory | but how
good, really, is this notion of good complexity behaviour? It might be objected that Table 1 is misleading, unless FPTk parameter functions such as
2k are typical. Certainly functions such as 222 are allowed by the denition,
and would be impractical for k = 3, which suggests that the basic denition
allows too much pathology.
There are two main responses. First of all, we are already used to some
risk-taking in denitions, since the notion of polynomial time allows for, e.g.,
O(n12) and O(n120) and O(n1200), all of which are completely impractical.
We forget how controversial the notion of asymptotic polynomial time was
when rst introduced, but the notion has thrived because the universe of
natural computational problems has been kind, in some sense. A parameterized problem is just an ordinary problem for which some aspect of the
input has been designated as the parameter. Ignoring the parameter, if the
problem can be solved in polynomial time, that is, in time polynomial in
both the total input size n and the parameter k, then trivially this is an
FPT algorithm. In other words, considered classically, FPT is a superset
of P, and it is intended to be a generalization that allows us to do something for problems that are not in P and that may even be PSPACE hard
or undecidable. We have to expect to risk something in formulating such a
general attack on intractability. The denition simply asks whether the difculty of the problem can be conned to a function of the parameter, while
the other costs are polynomial. How else would one naturally formulate a
generalization of P having such ambitions?
The second response is that there are many good FPT examples other
than Vertex Cover suggesting that \reasonable" (e.g., single exponential)
parameter functions are usually obtainable for natural problems (possibly
after some rounds of improvement). For example, consider the problem
Maximum Satisfiability where the parameter k denotes the number of
clauses to be satised. This was shown by Cai and Chen[CC97] to be in
FPT with the parameter function 22ck , when the clause size is bounded by c.
The parameter function was improved by Mahajan and Raman[MR98] to
k (without
assuming a bound on the clause size), where is the golden ratio
p
(1 + 5)=2. Franco and others in [Fr97] have shown that the falsiability
problem for pure implicational formulas having k negations is in FPT with
parameter function kk . (Can this be improved to 2k ?) Although the type
checking problem for the programming language ML is PSPACE-complete,
this is handled in implementations in linear time with a parameter function
of 2k , where k is the nesting depth of let's, a very natural parameter for this
problem (one that explains why the problem did not seem hard in practice).
We will describe an FPT algorithm for a natural parameterization of the
Maximum Agreement Subtree problem having the parameter function
3k . Many more examples can be found in [DF98]. The improvement of
parameter functions for FPT problems seems to be a productive area for
research, where many dierent ideas and techniques can be employed.
The point of view that parameterized complexity adopts can be summarized in a metaphorical picture. There is an assumption that most interesting
problems are hard, so we can picture them as stones, or perhaps planets in
the generally hostile environment of outer space. The trick is to identify and
develop thin zones of computational viability, as suggested in Figure 1.
the
Rock
of
Intractability
Increasingly viable
range of k
Figure 1. The view from Mars: interesting computations
Approximation. Most of the discussion in the chapter on coping with NPcompleteness in [GJ79] is devoted to explaining the basic ideas of polynomial
Randomized Polynomial Time. Randomization is discussed in Chapter 6 of Garey and Johnson[GJ79] as a means of avoiding one of the weak-
10
workhorse of cryptography, and have important applications in computational geometry, pattern matching, on-line algorithms and computer algebra
(see [Karp86] and [MR95a] for surveys), in part because they are often
simple to program. Approximate counting is another area of notable success. Randomization is an important new idea that is now applied in many
dierent kinds of algorithms, including approximations and heuristics.
Despite these successes, it now seems that randomized polynomial time
is better at delivering good algorithms for dicult problems that \probably" are in P to begin with, than at providing a general means for dealing with intractable problems. There have recently been a number of important results replacing fast probabilistic algorithms with ordinary polynomial time algorithms through the use of sophisticated derandomization
techniques [Rag88, MR95b]. Even more to the point, Impagliazzo and
Wigderson have recently shown that if there merely exists a language in
the exponential time class EXP that requires circuits of size 2
(n) then
BPP = P [IW97]. The main weaknesses of randomization (in the sense of
algorithms with performance guarantees) as a general program for coping
with intractability are:
time algorithms are any more eective against problems that are truly
hard than ordinary polynomial time algorithms.
Although the positive toolkit of methods for designing and analyzing
randomized algorithms is rich, there is no specically corresponding
negative toolkit that can be used in tandem to negotiate problem
complexity and guide the design of eective algorithms.
New Forms of Computation: DNA and Quantum Mechanics. Although these programs have been launched with great fanfare, they so far
oer much less of substance than the other items on this list in terms of
a general mathematically-powered program for coping with intractability.
So far, DNA computing essentially amounts to computation by molecular
brute force. Combinatorial explosion can quickly force one to contemplate
a very large test tube for brute force computations, despite the fact that
information can be stored in molecules with a factor of 1012 improved efciency compared to magnetic tape storage. Mathematically, the program
seems to come down to the potential for signicant constant speed-ups by
means of this physical miniaturization of computing.
It is still unclear whether quantum computers useful for any kind of
computation can actually be built. The notion of quantum polynomial time
is mathematically interesting, but so far appears to be applicable only to a
few special kinds of problems.
The main weakness of these approaches are:
Practical implementation of the new computing devices seems to be
far in the future.
11
12
2
where k is ... unfortunately, it doesn't matter ... while the townspeople are
cheering on another, who is marching towards the Dragon on foot waving a
wooden cudgel (not a knight, actually just a hired ruan).
Leaving fanciful impressions aside, it seems fair to say that the central
problems of theoretical computer science, both structural and concrete, have
turned out to be much harder to crack than was hoped in the early years of
the eld.
In this section we sketch the reasons why we think that parameterized complexity oers an inevitable, general, mathematical program for dealing with
computational intractability. There are two main points to the argument.
13
1. The parameterized complexity perspective can lead to useful algorithms in several dierent ways, including:
Directly and analytically, when the parameter function is reasonable (i.e., not too fast-growing) and the parameter describes
a restriction of the general problem that is still useful.
Directly and empirically, in cases where the analytic bound on
the running time of the FPT algorithm turns out to be too
crude or pessimistic, that is, where the algorithm turns out
to be useful for larger parameter ranges than the parameter
function would indicate.
By supplying systematic guidance in the design of heuristic algorithms in the form of explicit general methods based on FPT
techniques, using the theory to understand \the internal structure of the complexity of the problem" by identifying those
parameterizations of the problem that are FPT and those that
probably are not. (E.g., local search algorithms that iterate
FPT k-step explorations of the neighborhood structure of the
solution space.)
Via methodological connections between FPT and the design
of ecient polynomial-time approximation schemes (where the
relevant implicit parameter is k = 1=, for approximations to
within a factor of (1 + ) of optimal) and other approximation
heuristics.
2. The notion of FPT, in many cases, simply provides a new name and a
new perspective on heuristic algorithms already in use. Where natural
instance distributions exhibit limited parameter ranges, these have
frequently been exploited in the design of useful heuristics in a way
that the notion of FPT names and systematizes.
We have already reported on a steadily growing list of examples of FPT
problems with \reasonable" parametric costs, such as 2k . Currently the best
known algorithm for Vertex Cover runs in time O(kn + maxf(1:25542)kk2,
(1:2906)k2:5kg) which directly indicates that it is useful for graphs of any
size, so long as k 157 (the running time of O(kn + maxf(1:25542)kk2 ,
(1:2906)k2:5kg) is a result mainly of a combination of the algorithm by
Niedermeier and Rossmanith[NR99], the algorithm presented in [DFS99],
and a more careful analysis [Ste99, SF99]). In fact, experiments with the
algorithm indicate that it is useful for k at least up to 200 [HGS98].
FPT-algorithm design has a distinctive toolkit of positive techniques,
including bounded treewidth and pathwidth methods [Bod96], well quasiordering [RS85], color-coding [AYZ94] and important elementary methods
such as search trees and reduction to a problem kernel. This is unfortunately not the place to exposit this interesting collection of methods. What
is important to realize, however, is that all of these methods can be systematically adapted in the design of heuristic algorithms. The essential theme is
14
to obtain heuristics from FPT algorithms by strategies for de
ating the parametric costs by truncating or sampling the search trees, obstruction sets,
test sets, etc. This is summarized in the following table.
FPT Technique
15
X to obtain a set X 0 on which all of the trees agree. In considering this pro-
16
implies that the problem is in FPT, since the question can be answered by
brute force for (x0; k0). Surprisingly, it can be shown that every problem in
FPT can be kernelized. Such reductions seem to be closely related to ideas
explored in [CKT91, KS94]. Since such reductions simplify the input, it
seems that they cannot hurt, and thus constitute a reasonable \preprocessing" heuristic regardless of what comes after that.
The Steiner Problem for Hypercubes is of interest to biologists
in the computation of phylogenetic trees under the criterion of minimum
evolution / maximum parsimony. The set S corresponds to a set of species,
and the binary vectors correspond to information about the species, each
component recording the answer to some question (as 0 or 1), such as: \Does
it have wings?" or \Is there a thymine at a certain position in the DNA sequence?" Each such bit of information is termed a character of the species.
In realistic applications the number k of species may usefully be around 40
or 60, while the number of characters n may be very large. We consider a
slightly more general problem than the one described in x1.
The Steiner Problem for Generalized Hypercubes
Instance: The input to the problem consists of the following pieces of information:
(1) A set of complete weighted digraphs Di for i = 1; :::; n, each described
by a set of vertices Vi and a function
ti : Vi Vi ! IN
(We refer to the vertices of Di as character states, to Di as the character
state digraph, and to ti as the state transition cost function for the ith character.)
(2) A positive integer k1 such that jVi j k1 for i = 1; :::; n.
(3) A set X of k2 length n vectors xj for j = 1; :::; k2, where the ith component xj [i] 2 Vi. That is, for j = 1; :::; k2,
n
xj 2
= =n Vi
i=1
X t (y [i]; y [i])
n
is bounded by M ?
i=1
i u
17
18
The parameter function for this simple kernelization algorithm is not very
good and can probably be much improved. We remark that most of the
expense is in determining when two transition digraph indices i and i0 are
equivalent by testing them on `the relevant set of trees with k2 leaves. This
suggests a heuristic algorithm that combines indices when they fail to be
distinguished by a (much smaller) random sample of trees and leaf-labelings.
In x1 we reported an encounter with an evolutionary biologist who reported earlier, rather fruitless interactions with theoretical computer scientists who proved that his problems were NP-complete and \went away".
We claimed that we were dierent! and that we had a result on one of
his computational problems (The Steiner Problem for Hypercubes)
that might be of interest. After we described the FPT algorithm he said
simply [Fel97]:
\That's what I already do!"
5. Parametric Intractability
Linear Inequalities
Minimum Axiom Set
Short Satisfiability
Weighted Circuit Satisfiability
W [SAT ] Weighted Satisfiability
Longest Common Subsequence
(k = h number of seqs.,jji) (hard)
W [t], Feasible Register Assignment (hard)
for all t Triangulating Colored Graphs (hard)
Bandwidth (hard)
Proper Interval Graph Completion (hard)
Weighted t{Normalized Satisfiability
Weighted f0; 1g Integer Programming
W [2] Dominating Set
Tournament Dominating Set
Unit Length Precedence Constrained
Scheduling (hard)
Shortest Common Supersequence (k seqs.) (hard)
Maximum Likelihood Decoding (hard)
Weight Distribution in Linear Codes (hard)
Nearest Vector in Integer Lattices (hard)
Short Permutation Group Factorization (hard)
W [1] Short Post Correspondence
Weighted q {CNF Satisfiability
Vapnik{Chervonenkis Dimension
Longest Common Subsequence
(length m common subseq. for k seqs.,
parameter (k; m))
Independent Set
Square Tiling
Monotone Data Complexity for
Relational Databases
k-Step Derivation for Context Sensitive Grammars
Clique
Short NTM Computation
Feedback Vertex Set
FPT Graph Genus
Minor Order Test
Treewidth
Vertex Cover
Table 3. A Sample of Parametric Complexity Classica-
W [P ]
19
20
21
The pair of notions (10) and (20) are actually rather straightforward mutations of (1) and (2), and they inherit many of the properties that have made
the framework provided by (1) and (2) so successful. We note the following
in support of this position.
22
and that many of the problems in FPT admit kernelization algorithms that provide useful start-ups for any algorithmic attack on the
problem.
The complexity of approximation is handled more elegantly than in
the classical theory, with W [1]-hardness immediately implying that
there is no ecient PTAS. Moreover, FPT algorithm design techniques appear to be fruitful in the design of approximation algorithms
(e.g., bounded treewidth techniques in the planar graph PTAS results
of Baker[Ba94]).
Parameterization is a very broad idea. It is possible to formulate
and explore notions such as randomized FPT [FK93], parameterized parallel complexity [Ces96], parameterized learning complexity [DEF93], parameterized approximation, parameterized cryptosystems based on O(nk ) security, etc.
We feel that the parametric complexity notions, with their implicit ultranitism, correspond better to the natural universe of computational complexity, where we nd ourselves overwhelmingly among hard problems, dependent on identifying and exploiting thin zones of computational viability.
Many natural problem distributions are generated by processes that inhabit
such zones themselves (e.g., computer code that is written in a structured
manner so that it can be comprehensible to the programmer), and these
distributions then inherit limited parameter ranges because of the computational parameters that implicitly govern the feasibility of the generative
processes, though the relevant parameters may not be immediately obvious.
It seems that we have a whole new world of complexity issues to explore!
References
23
[BDFHW95] H. Bodlaender, R. Downey, M. Fellows, M. Hallett and H. T. Wareham, \Parameterized Complexity Analysis in Computational Biology," Computer Applications
in the Biosciences 11 (1995), 49{57.
[BFR98] R. Balasubramanian, M. Fellows and V. Raman, \An Improved Fixed-Parameter
Algorithm for Vertex Cover," Information Processing Letters 65 (1998), 163{168.
[BFRS98] D. Bryant, M. Fellows, V. Raman and U. Stege, \On the Parameterized Complexity of MAST and 3-Hitting Sets," manuscript, 1998.
[Bod96] H. Bodlaender, \A Linear Time Algorithm for Finding Tree Decompositions of
Small Treewidth," SIAM J. Comp. 25 (1996), 1305{1317.
[Bry97] D. Bryant, \Building Trees, Hunting for Trees, and Comparing Trees," Ph.D. dissertation, Department of Mathematics, Univ. Canterbury, Christchurch, New
Zealand, 1997.
[Bry98] D. Bryant, private communication, 1998.
[CC97] L. Cai and J. Chen. \On Fixed-Parameter Tractability and Approximability of
NP-Hard Optimization Problems," J. Computer and Systems Sciences 54 (1997),
465{474.
[CCDF96] L. Cai, J. Chen, R. G. Downey and M. R. Fellows, \On the Parameterized Complexity of Short Computation and Factorization," Arch. for Math. Logic 36 (1997),
321{337.
[CCDF97] L. Cai, J. Chen, R. Downey and M. Fellows, \Advice Classes of Parameterized
Tractability," Annals of Pure and Applied Logic 84 (1997), 119{138.
[Ces96] M. Cesati, \Structural Aspects of Parameterized Complexity," Ph.D. dissertation,
University of Rome, 1995.
[CKT91] P. Cheeseman, B. Kanefsky and W. Taylor, \Where the Really Hard Problems
Are," Proc. 12th International Joint Conference on Articial Intelligence (1991), 331{
337.
[CT97] M. Cesati and L. Trevisan, \On the Eciency of Polynomial Time Approximation
Schemes," Information Processing Letters 64 (1997), 165{171.
[DKL96] T. Dean, J. Kirman and S.-H. Lin, \Theory and Practice in Planning," Technical
Report, Computer Science Department, Brown University, 1996.
[DEF93] R. Downey, P. Evans and M. Fellows, \Parameterized Learning Complexity,"
Proc. 6th ACM Workshop on Computational Learning Theory (1993), 51{57.
[DF95] R. G. Downey and M. R. Fellows, \Fixed Parameter Tractability and Completeness I: Basic Theory," SIAM Journal of Computing 24 (1995), 873{921.
[DF98] R. G. Downey and M. R. Fellows, Parameterized Complexity, Springer-Verlag,
1998.
[DFKHW94] R. G. Downey, M. Fellows, B. Kapron, M. Hallett, and H. T. Wareham. \The
Parameterized Complexity of Some Problems in Logic and Linguistics," Proceedings
Symposium on Logical Foundations of Computer Science (LFCS), Springer-Verlag,
Lecture Notes in Computer Science vol. 813 (1994), 89{100.
[DFS99] R. G. Downey, M. R. Fellows, and U. Stege. \Parameterized Complexity: A
Framework for Systematically Confronting Parameterized Complexity," in The Future of Discrete Mathematics: Proceedings of the First DIMATIA Symposium, Stirin
Castle, Czech Republic, June, 1997, F. Roberts, J. Kratochvil and J. Nesetril (eds.),
AMS-DIMACS Proceedings Series, 1999 (in print).
[Fel97] J. Felsenstein. Private communication, 1997.
[FK93] M. Fellows and N. Koblitz, \Fixed-Parameter Complexity and Cryptography,"
Proceedings of the 10th Intl. Symp. on Applied Algebra, Algebraic Algorithms and
Error-Correcting Codes, Springer-Verlag, Berlin, Lecture Notes in Computer Science
vol. 673 (1993), 121{131.
[FPT95] M. Farach, T. Przytycka, and M. Thorup. \On the agreement of many trees"
Information Processing Letters 55 (1995), 297{301.
24
25
Michael R. Fellows, Department of Computer Science, University of Victoria, Victoria, B.C. V8W 3P6, Canada.