When looking for the best course of management decisions to efficiently conserve metapopulation s... more When looking for the best course of management decisions to efficiently conserve metapopulation systems, a classic approach in the ecology literature is to model the optimisation problem as a Markov decision process and find an optimal control policy using exact stochastic dynamic programming techniques. Stochastic dynamic programming is an iterative procedure that seeks to optimise a value function at each timestep by evaluating the benefits of each of the actions in each state of the system defined in the Markov decision process.
Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal.
We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the optimal solution to the results of the on-line sparse sampling algorithm for a simple 3-waterhole case. We find that three look-ahead steps minimises the error between the optimal solution and the approximation algorithm. This paper introduces a new algorithm to conservation management that provides a way to avoid the effects of the curse of dimensionality. The work has the potential to allow us to approximate solutions to much more complex metapopulation management problems in the future.
Managers of species that exist as metapopulations are faced with many decisions. In this paper we... more Managers of species that exist as metapopulations are faced with many decisions. In this paper we use a decision-theory framework to examine a fundamental management question: Should we focus on decreasing the local extinction probability of subpopulations by increasing the size of their patch, or should metapopulation viability be improved by constructing more patches? Using a spatially implicit stochastic metapopulation model and stochastic dynamic programming (SDP), we found the optimal solution to this problem for both the finite- and infinite-time horizon cases. We showed that the SDP solutions outperform a range of heuristic management strategies. The optimal strategy for a given parameter set depends heavily on metapopulation parameters, and it is difficult to make generalizations about the optimal restoration strategy a priori. Although heuristic strategies perform well in some cases, it is not possible to judge their performance until the SDP solution has been computed, and for this reason we advocate the use of SDP as a management tool in restoration. We demonstrate the use of SDP by deriving an optimal management strategy for a population of the Mount Lofty Ranges Southern Emu-wren (Stipiturus malachurus intermedius).
Centre for Water Research, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009... more Centre for Water Research, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: ...
The need to make correct management decisions in the face of an uncertain future motivates conser... more The need to make correct management decisions in the face of an uncertain future motivates conservation biologists to opti-mize decision making using mathematical techniques. For models with small state spaces the best technique is stochastic dynamic programming (SDP), ...
When looking for the best course of management decisions to efficiently conserve metapopulation s... more When looking for the best course of management decisions to efficiently conserve metapopulation systems, a classic approach in the ecology literature is to model the optimisation problem as a Markov decision process and find an optimal control policy using exact stochastic dynamic programming techniques. Stochastic dynamic programming is an iterative procedure that seeks to optimise a value function at each timestep by evaluating the benefits of each of the actions in each state of the system defined in the Markov decision process.
Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal.
We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the optimal solution to the results of the on-line sparse sampling algorithm for a simple 3-waterhole case. We find that three look-ahead steps minimises the error between the optimal solution and the approximation algorithm. This paper introduces a new algorithm to conservation management that provides a way to avoid the effects of the curse of dimensionality. The work has the potential to allow us to approximate solutions to much more complex metapopulation management problems in the future.
Managers of species that exist as metapopulations are faced with many decisions. In this paper we... more Managers of species that exist as metapopulations are faced with many decisions. In this paper we use a decision-theory framework to examine a fundamental management question: Should we focus on decreasing the local extinction probability of subpopulations by increasing the size of their patch, or should metapopulation viability be improved by constructing more patches? Using a spatially implicit stochastic metapopulation model and stochastic dynamic programming (SDP), we found the optimal solution to this problem for both the finite- and infinite-time horizon cases. We showed that the SDP solutions outperform a range of heuristic management strategies. The optimal strategy for a given parameter set depends heavily on metapopulation parameters, and it is difficult to make generalizations about the optimal restoration strategy a priori. Although heuristic strategies perform well in some cases, it is not possible to judge their performance until the SDP solution has been computed, and for this reason we advocate the use of SDP as a management tool in restoration. We demonstrate the use of SDP by deriving an optimal management strategy for a population of the Mount Lofty Ranges Southern Emu-wren (Stipiturus malachurus intermedius).
Centre for Water Research, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009... more Centre for Water Research, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: ...
The need to make correct management decisions in the face of an uncertain future motivates conser... more The need to make correct management decisions in the face of an uncertain future motivates conservation biologists to opti-mize decision making using mathematical techniques. For models with small state spaces the best technique is stochastic dynamic programming (SDP), ...
Uploads
Papers by Sam Nicol
Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal.
We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the optimal solution to the results of the on-line sparse sampling algorithm for a simple 3-waterhole case. We find that three look-ahead steps minimises the error between the optimal solution and the approximation algorithm. This paper introduces a new algorithm to conservation management that provides a way to avoid the effects of the curse of dimensionality. The work has the potential to allow us to approximate solutions to much more complex metapopulation management problems in the future.
Read More: http://www.esajournals.org/doi/abs/10.1890/08-2216.1
Although stochastic dynamic programming methods provide an optimal solution to conservation management questions in a stochastic world, their applicability in metapopulation problems has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. The high computational requirements of stochastic dynamic programming methods mean that only simple metapopulation management problems can be analysed. In this paper we overcome the complexity burden of exact stochastic dynamic programming methods and present the benefits of an on-line sparse sampling algorithm proposed by Kearns, Mansour and Ng (2002). The algorithm is particularly attractive for problems with large state spaces as the running time is independent of the size of the state space of the problem. This appealing improvement is achieved at a cost: the solutions found are no longer guaranteed to be optimal.
We apply the algorithm of Kearns et al. (2002) to a hypothetical fish metapopulation problem where the management objective is to maximise the number of occupied patches over the management time horizon. Our model has multiple management options to combat the threats of water abstraction and waterhole sedimentation. We compare the performance of the optimal solution to the results of the on-line sparse sampling algorithm for a simple 3-waterhole case. We find that three look-ahead steps minimises the error between the optimal solution and the approximation algorithm. This paper introduces a new algorithm to conservation management that provides a way to avoid the effects of the curse of dimensionality. The work has the potential to allow us to approximate solutions to much more complex metapopulation management problems in the future.
Read More: http://www.esajournals.org/doi/abs/10.1890/08-2216.1