International Journal on Artificial Intelligence Tools, Aug 1, 2013
Feature selection is a multi-objective problem with the two main conflicting objectives of minimi... more Feature selection is a multi-objective problem with the two main conflicting objectives of minimising the number of features and maximising the classification performance. However, most existing feature selection algorithms are single objective and do not appropriately reflect the actual need. There are a small number of multi-objective feature selection algorithms, which are wrapper based and accordingly are computationally expensive and less general than filter algorithms. Evolutionary computation techniques are particularly suitable for multi-objective optimisation because they use a population of candidate solutions and are able to find multiple non-dominated solutions in a single run. However, the two well-known evolutionary multi-objective algorithms, nondominated sorting based multi-objective genetic algorithm II (NSGAII) and strength Pareto evolutionary algorithm 2 (SPEA2) have not been applied to filter based feature selection. In this work, based on NSGAII and SPEA2, we develop two multi-objective, filter based feature selection frameworks. Four multi-objective feature selection methods are then developed by applying mutual information and entropy as two different filter evaluation criteria in each of the two proposed frameworks. The proposed multi-objective algorithms are examined and compared with a single objective method and three traditional methods (two filters and one wrapper) on eight benchmark datasets. A decision tree is employed to test the classification performance. Experimental results show that the proposed multi-objective algorithms can automatically evolve a set of non-dominated solutions that include a smaller number of features and achieve better classification performance than using all features. NSGAII and SPEA2 outperform the single objective algorithm, the two traditional filter algorithms and even the traditional wrapper algorithm in terms of both the number of features and the classification performance in most cases. NSGAII achieves similar performance to SPEA2 for the datasets that consist of a small number of features and slightly better results when the number of features is large. This work represents the first study on NSGAII and SPEA2 for filter feature selection in classification problems with both providing field leading classification performance.
Cooperative co-evolutionary algorithms have addressed many large-scale problems successfully, but... more Cooperative co-evolutionary algorithms have addressed many large-scale problems successfully, but the nonseparable large-scale problems with overlapping subcomponents are still a serious difficulty that has not been conquered yet. First, the existence of shared variables makes the problem hard to be decomposed. Second, existing cooperative co-evolutionary frameworks usually cannot maintain the two crucial factors, high cooperation frequency and effective computing resource allocation, simultaneously when optimizing the overlapping subcomponents. Aiming at these two issues, this paper proposes a new contribution-based cooperative co-evolutionary algorithm to decompose and optimize the non-separable large-scale problems with overlapping subcomponents effectively and efficiently. 1) A contribution-based decomposition method is proposed to assign the shared variables. Among all the subcomponents containing a shared variable, the one that contributes the most to the whole problem will include the shared variable. 2) To achieve the two crucial factors at the same time, a new contribution-based optimization framework is designed to award the important subcomponents based on the round-robin structure. Experimental studies show that the proposed algorithm performs significantly better than the state-of-the-art algorithms due to the effective grouping structure generated by the proposed decomposition method and the fast optimizing speed provided by the new optimization framework.
The Uncertain Capacitated Arc Routing Problem (UCARP) has a variety of real-world applications. G... more The Uncertain Capacitated Arc Routing Problem (UCARP) has a variety of real-world applications. Genetic Programming Hyper-heuristic (GPHH) is considered a promising technique to handle UCARP. Many scholars have shown the power of GPHH of evolving effective routing policies. However, the size of the evolved routing policies is ignored. Typically, smaller routing policies can have better interpretability and generalisation. Thus, it is necessary to optimise the size along with the effectiveness. The objective selection bias issue arises as the size is much easier to be optimised than effectiveness. The Pareto front is biased to the size gradually during the evolutionary process. To address this issue, we develop an α dominance criteria based Multi-Objective GP with a self-adaptive α scheme (αMOGP-sa). The basic idea of the α dominance criteria is to set tradeoff rates between objectives. For different instances, the search space can be very different. In this case, the self-adaptive α scheme is employed to automatically tuning the α value during the evolutionary process so that we can identify a valid α value for different instances. This paper examines the proposed algorithm in eight different problem instances. The experimental results showed that αMOGP-sa could effectively handle the objective selection bias issue, and evolve much better Pareto front on Hyper-Volume and Inverted Generational Distance than the current state-of-the-art MOGP approach for UCARP in terms of effectiveness and size on all instances. Also, αMOGP-sa can evolve much smaller routing policies than the state-of-art single-objective GPHH without sacrificing effectiveness.
Proceedings of the 2020 Genetic and Evolutionary Computation Conference, 2020
Potable water distribution networks are requisites of modern cities. Because of the city expansio... more Potable water distribution networks are requisites of modern cities. Because of the city expansion, nowadays, the scale of the network grows rapidly, which brings great difficulty to its optimization. Evolutionary computation methods have been widely investigated on small-scale networks, but their performance is far from satisfactory on large-scale networks. Aimed at addressing this difficulty, a new memetic algorithm called level-based learning swarm optimizer with restart and local search is proposed in this paper to solve the large-scale water distribution network optimization problem. Instead of using traditional evolutionary computation algorithms, the level-based learning swarm optimizer that is especially proposed for large-scale optimization problems is applied as the populationbased optimizer. Two restart strategies are incorporated to make the algorithm more effective. They can help the algorithm jump out from local optima thus to increase its exploration ability. Moreover, a simple yet effective local search algorithm is proposed based on the domain knowledge to further refine the solutions after the algorithm converges. Experimental results on both single-source and multi-source large-scale water distribution networks show that the proposed algorithm is more effective than the state-of-the-art evolutionary computation algorithms. CCS CONCEPTS • Computing methodologies → Discrete space search; Randomized search; • Applied computing → Computer-aided design;
Evolutionary multitask learning has achieved great success due to its ability to handle multiple ... more Evolutionary multitask learning has achieved great success due to its ability to handle multiple tasks simultaneously. However, it is rarely used in the hyper-heuristic domain which aims at generating a heuristic for a class of problems rather than solving one specific problem. The existing multitask hyperheuristic studies only focus on heuristic selection, which is not applicable to heuristic generation. To fill the gap, we propose a novel multitask generative hyper-heuristic approach based on genetic programming in this paper. Specifically, we introduce the idea in evolutionary multitask learning to genetic programming hyper-heuristics with a suitable evolutionary framework and individual selection pressure. In addition, an origin-based offspring reservation strategy is developed to maintain the quality of individuals for each task. To verify the effectiveness of the proposed approach, comprehensive empirical studies have been conducted on the homogeneous and heterogeneous multitask dynamic flexible job shop scheduling. The results show that the proposed algorithm can significantly improve the quality of scheduling heuristics for each task in all the examined scenarios. In addition, the evolved scheduling heuristics verify the mutual help among the tasks in a multitask scenario.
Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2019
Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation prob... more Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation problem. Genetic Programming (GP) has shown effectiveness in automatically evolving routing policies to handle the uncertain environment in UCARP. However, when the scenario changes, current routing policy can no longer work effectively, and one has to retrain a new policy for the new scenario which is time consuming. On the other hand, knowledge from solving the previous similar scenarios may be helpful in improving the efficiency of the retraining process. In this paper, we propose different knowledge transfer methods from a source scenario to a similar target scenario and examine them in different settings. The experimental results showed that by knowledge transfer, the retraining process is made more efficient and the same performance one can obtained within a much shorter time without having any negative transfer.
2020 IEEE Symposium Series on Computational Intelligence (SSCI), 2020
Amid continuing increases in purchasing power and Internet shopping, demands for more efficient a... more Amid continuing increases in purchasing power and Internet shopping, demands for more efficient algorithms to solve the Large-Scale Vehicle Routing Problems are to be expected. The problem, which consists of finding routes for visiting customers, however, is very hard to solve due to its combinatorial nature. This becomes more notable when looking at large scale instances, i.e at least 200 customers, requiring sophisticated solution methods to solve, mostly heuristic-based. The existing meta-heuristics apply some limits to the solution space for handling the large scale nature. However, this is typically done manually, based on extensive domain knowledge. Hyper-Heuristics aim to reduce domain knowledge dependence, but current works disregard search space size, presenting poor scalability. In this paper, we propose a new Hyper-Heuristic to automatically evolve effective heuristics, based on a Genetic Algorithm framework. The new Hyper-Heuristic adds limits to the neighbourhoods of Low-Level Heuristics. In the method, the large search space is dealt with an adaptive chromosome which limits the neighbourhood size for each operator. The goal is to show whether more effective limits can be found if done automatically. Results show that the automatically evolved neighbourhood size limits outperform fixed ones for most instances tested and manually designed limits.
Dynamic flexible job shop scheduling has received widespread attention from academia and industry... more Dynamic flexible job shop scheduling has received widespread attention from academia and industry due to its practical application value. It requires complex routing and sequencing decisions under unpredicted dynamic events. Genetic programming, as a hyper-heuristic approach, has been successfully applied to evolve scheduling heuristics for job shop scheduling due to its flexible representation. However, the simulationbased evaluation is computationally expensive since there are many calculations based on individuals for making decisions in the simulation. To improve training efficiency, this paper proposes a novel multi-fidelity based surrogate-assisted genetic programming. Specifically, multi-fidelity based surrogate models are first designed by simplifying the problem expected to be solved. In addition, this paper proposes an effective collaboration mechanism with knowledge transfer for utilising the advantages of multi-fidelity based surrogate models to solve the desired problems. This paper examines the proposed algorithm in six different scenarios. The results show that the proposed algorithm can dramatically reduce the computational cost of genetic programming without sacrificing the performance in all scenarios. With the same training time, the proposed algorithm can achieve significantly better performance than its counterparts in most scenarios while no worse in others.
2020 IEEE Congress on Evolutionary Computation (CEC), 2020
The Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation ... more The Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation problem with extensive real-world applications. Genetic Programming (GP) has shown effectiveness in automatically evolving routing policies to handle the uncertain environment in UCARP. However, whenever a UCARP scenario changes, e.g. when a new vehicle is bought, the previously trained routing policy may no longer work effectively, and one has to retrain a new policy. Retraining a new policy from scratch can be time-consuming but the transfer of knowledge gained from solving the previous similar scenarios may help improve the efficiency of the retraining process. In this paper, we propose a novel transfer learning method by learning the probability distribution of good solutions from source domains and modelling it as a probabilistic prototype tree. We demonstrate that this approach is capable of capturing more information about the source domain compared to transfer learning based on (sub-)tree transfers and even create good trees that are not seen in source domains. Our experimental results showed that our method made the retraining process more efficient and one can obtain an initial state for solving difficult problems that is significantly better than existing methods. The final performance of all algorithms, were comparable, implying that there was no negative transfer.
Dynamic flexible job shop scheduling (DFJSS) is a challenging combinational optimisation problem ... more Dynamic flexible job shop scheduling (DFJSS) is a challenging combinational optimisation problem that takes the dynamic environment into account. Genetic programming hyperheuristics (GPHH) have been widely used to evolve scheduling heuristics for job shop scheduling. A proper selection of the terminal set is a critical factor for the success of GPHH. However, there is a wide range of features that can capture different characteristics of the job shop state. Moreover, the importance of a feature is unclear from one scenario to another. The irrelevant and redundant features may lead to performance limitations. Feature selection is an important task to select relevant and complementary features. However, little work has considered feature selection in GPHH for DFJSS. In this paper, a novel two-stage GPHH framework with feature selection is designed to evolve scheduling heuristics only with the selected features for DFJSS automatically. Meanwhile, individual adaptation strategies are proposed to utilise the information of both the selected features and the investigated individuals during the feature selection process. The results show that the proposed algorithm can successfully achieve more interpretable scheduling heuristics with fewer unique features and smaller sizes. In addition, the proposed algorithm can reach comparable scheduling heuristic quality with much shorter training time.
Due to its direct relevance to post-disaster operations, meter reading and civil refuse collectio... more Due to its direct relevance to post-disaster operations, meter reading and civil refuse collection, the Uncertain Capacitated Arc Routing Problem (UCARP) is an important optimisation problem. Stochastic models are critical to study as they more accurately represent the real world than their deterministic counterparts. Although there have been extensive studies in solving routing problems under uncertainty, very few have considered UCARP, and none consider collaboration between vehicles to handle the negative effects of uncertainty. This article proposes a novel Solution Construction Procedure (SCP) that generates solutions to UCARP within a collaborative, multi-vehicle framework. It consists of two types of collaborative activities: one when a vehicle unexpectedly expends capacity ( route failure), and the other during the refill process. Then, we propose a Genetic Programming Hyper-Heuristic (GPHH) algorithm to evolve the routing policy used within the collaborative framework. The ...
The uncertain capacitated arc routing problem is of great significance for its wide applications ... more The uncertain capacitated arc routing problem is of great significance for its wide applications in the real world. In the uncertain capacitated arc routing problem, variables such as task demands and travel costs are realised in real time. This may cause the predefined solution to become ineffective and/or infeasible. There are two main challenges in solving this problem. One is to obtain a high-quality and robust baseline task sequence, and the other is to design an effective recourse policy to adjust the baseline task sequence when it becomes infeasible and/or ineffective during the execution. Existing studies typically only tackle one challenge (the other being addressed using a naive strategy). No existing work optimises the baseline task sequence and recourse policy simultaneously. To fill this gap, we propose a novel proactive-reactive approach, which represents a solution as a baseline task sequence and a recourse policy. The two components are optimised under a cooperative ...
Dynamic job shop scheduling (DJSS) problems are combinatorial optimisation problems that have bee... more Dynamic job shop scheduling (DJSS) problems are combinatorial optimisation problems that have been extensively studied in the literature due to their difficulty and their applicability to real-world manufacturing systems, e.g., car manufacturing systems. In a DJSS problem instance, jobs arrive on the shop floor to be processed on specific sequences of machines on the shop floor and unforeseen events such as dynamic job arrivals and machine breakdown occur that affect the properties of the shop floor. Many researchers have proposed genetic programming based hyper-heuristic (GP-HH) approaches to evolve high quality dispatching rules for DJSS problems with dynamic job arrivals, outperforming good man-made rules for the problems. However, no GP-HH approaches have been proposed for DJSS problems with dynamic job arrivals and machine breakdowns, and it is not known how well GP generalises over both DJSS problem instances with no machine breakdown to problem instances with machine breakdown. Therefore, this paper investigates the generality of GP for DJSS problem with dynamic job arrivals and machine breakdowns. To do this, a machine breakdown specific DJSS dataset is proposed, and an analysis procedure is used to observe the differences in the structures of the GP rules when evolved under different machine breakdown scenarios. The results show that performance and the distributions of the terminals for the evolved rules is sensitive to the frequency of machine breakdowns in the training instances used to evolve the rules.
Evolutionary Computation in Combinatorial Optimisation, 2014
Feature selection is an important but difficult task in classification, which aims to reduce the ... more Feature selection is an important but difficult task in classification, which aims to reduce the number of features and maintain or even increase the classification accuracy. This paper proposes a new particle swarm optimisation (PSO) algorithm using statistical clustering information to solve feature selection problems. Based on Gaussian distribution, a new updating mechanism is developed to allow the use of the clustering information during the evolutionary process of PSO based on which a new algorithm (GPSO) is developed. The proposed algorithm is examined and compared with two traditional algorithms and a PSO based algorithm which does not use clustering information on eight benchmark datasets of varying difficulty. The results show that GPSO can be successfully used for feature selection to reduce the number of features and achieve similar or even better classification performance than using all features. Meanwhile, it achieves better performance than the two traditional feature selection algorithms. It maintains the classification performance achieved by the standard PSO for feature selection algorithm, but significantly reduces the number of features and the computational cost.
This paper describes two innovations that improve the efficiency and effectiveness of a genetic p... more This paper describes two innovations that improve the efficiency and effectiveness of a genetic programming approach to object detection problems. The approach uses genetic programming to construct object detection programs that are applied, in a moving window fashion, to the large images to locate the objects of interest. The first innovation is to break the GP search into two phases with the first phase applied to a selected subset of the training data, and a simplified fitness function. The second phase is initialised with the programs from the first phase, and uses the full set of training data with a complete fitness function to construct the final detection programs. The second innovation is to add a program size component to the fitness function. This approach was applied to three object detection problems of increasing difficulty. The results indicate that the innovations increased both the effectiveness and the efficiency of the genetic programming search, and also that the genetic programming approach was more effective than a neural network approach.
Abstract. The cooperative coevolution paradigm decomposes a large problem into a set of subcompon... more Abstract. The cooperative coevolution paradigm decomposes a large problem into a set of subcomponents and solves them independently in order to collectively solve the large problem. This work introduces a novel encoding scheme for building subcomponents based on functional prop-erties of ...
Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search proble... more Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search problem by decomposing it into low-dimensional subcomponents. Efficient problem decomposition methods or encoding schemes group interacting variables into separate subcomponents in order to solve them separately where possible. It is important to find out which encoding schemes efficiently group subcomponents and the nature of the neural network training problem in terms of the degree of nonseparability. This paper introduces a novel encoding scheme in cooperative coevolution for training recurrent neural networks. The method is tested on grammatical inference problems. The results show that the proposed encoding scheme achieves better performance when compared to a previous encoding scheme.
The 2011 International Joint Conference on Neural Networks, 2011
In this paper, an adaptive modularity cooperative coevolutionary framework is presented for train... more In this paper, an adaptive modularity cooperative coevolutionary framework is presented for training feedforward neural networks. The modularity adaptation framework is composed of different neural network encoding schemes which transform from one level to another based on the network error. The proposed framework is compared with canonical cooperative coevolutionary methods. The results show that the proposal outperforms its counterparts in terms of training time, success rate and scalability.
The 2011 International Joint Conference on Neural Networks, 2011
Memetic algorithms and cooperative coevolution are emerging fields in evolutionary computation wh... more Memetic algorithms and cooperative coevolution are emerging fields in evolutionary computation which have shown to be powerful tools for real-world application problems and for training neural networks. Cooperative coevolution decomposes a problem into subcomponents that evolve independently. Memetic algorithms provides further enhancement to evolutionary algorithms with local refinement. The use of crossover-based local refinement has gained attention in memetic computing. This paper employs a cooperative coevolutionary framework that utilises the strength of local refinement via crossover. The framework is evaluated by training recurrent neural networks on grammatical inference problems. The results show that the proposed approach can achieve better performance than the standard cooperative coevolution framework.
International Journal on Artificial Intelligence Tools, Aug 1, 2013
Feature selection is a multi-objective problem with the two main conflicting objectives of minimi... more Feature selection is a multi-objective problem with the two main conflicting objectives of minimising the number of features and maximising the classification performance. However, most existing feature selection algorithms are single objective and do not appropriately reflect the actual need. There are a small number of multi-objective feature selection algorithms, which are wrapper based and accordingly are computationally expensive and less general than filter algorithms. Evolutionary computation techniques are particularly suitable for multi-objective optimisation because they use a population of candidate solutions and are able to find multiple non-dominated solutions in a single run. However, the two well-known evolutionary multi-objective algorithms, nondominated sorting based multi-objective genetic algorithm II (NSGAII) and strength Pareto evolutionary algorithm 2 (SPEA2) have not been applied to filter based feature selection. In this work, based on NSGAII and SPEA2, we develop two multi-objective, filter based feature selection frameworks. Four multi-objective feature selection methods are then developed by applying mutual information and entropy as two different filter evaluation criteria in each of the two proposed frameworks. The proposed multi-objective algorithms are examined and compared with a single objective method and three traditional methods (two filters and one wrapper) on eight benchmark datasets. A decision tree is employed to test the classification performance. Experimental results show that the proposed multi-objective algorithms can automatically evolve a set of non-dominated solutions that include a smaller number of features and achieve better classification performance than using all features. NSGAII and SPEA2 outperform the single objective algorithm, the two traditional filter algorithms and even the traditional wrapper algorithm in terms of both the number of features and the classification performance in most cases. NSGAII achieves similar performance to SPEA2 for the datasets that consist of a small number of features and slightly better results when the number of features is large. This work represents the first study on NSGAII and SPEA2 for filter feature selection in classification problems with both providing field leading classification performance.
Cooperative co-evolutionary algorithms have addressed many large-scale problems successfully, but... more Cooperative co-evolutionary algorithms have addressed many large-scale problems successfully, but the nonseparable large-scale problems with overlapping subcomponents are still a serious difficulty that has not been conquered yet. First, the existence of shared variables makes the problem hard to be decomposed. Second, existing cooperative co-evolutionary frameworks usually cannot maintain the two crucial factors, high cooperation frequency and effective computing resource allocation, simultaneously when optimizing the overlapping subcomponents. Aiming at these two issues, this paper proposes a new contribution-based cooperative co-evolutionary algorithm to decompose and optimize the non-separable large-scale problems with overlapping subcomponents effectively and efficiently. 1) A contribution-based decomposition method is proposed to assign the shared variables. Among all the subcomponents containing a shared variable, the one that contributes the most to the whole problem will include the shared variable. 2) To achieve the two crucial factors at the same time, a new contribution-based optimization framework is designed to award the important subcomponents based on the round-robin structure. Experimental studies show that the proposed algorithm performs significantly better than the state-of-the-art algorithms due to the effective grouping structure generated by the proposed decomposition method and the fast optimizing speed provided by the new optimization framework.
The Uncertain Capacitated Arc Routing Problem (UCARP) has a variety of real-world applications. G... more The Uncertain Capacitated Arc Routing Problem (UCARP) has a variety of real-world applications. Genetic Programming Hyper-heuristic (GPHH) is considered a promising technique to handle UCARP. Many scholars have shown the power of GPHH of evolving effective routing policies. However, the size of the evolved routing policies is ignored. Typically, smaller routing policies can have better interpretability and generalisation. Thus, it is necessary to optimise the size along with the effectiveness. The objective selection bias issue arises as the size is much easier to be optimised than effectiveness. The Pareto front is biased to the size gradually during the evolutionary process. To address this issue, we develop an α dominance criteria based Multi-Objective GP with a self-adaptive α scheme (αMOGP-sa). The basic idea of the α dominance criteria is to set tradeoff rates between objectives. For different instances, the search space can be very different. In this case, the self-adaptive α scheme is employed to automatically tuning the α value during the evolutionary process so that we can identify a valid α value for different instances. This paper examines the proposed algorithm in eight different problem instances. The experimental results showed that αMOGP-sa could effectively handle the objective selection bias issue, and evolve much better Pareto front on Hyper-Volume and Inverted Generational Distance than the current state-of-the-art MOGP approach for UCARP in terms of effectiveness and size on all instances. Also, αMOGP-sa can evolve much smaller routing policies than the state-of-art single-objective GPHH without sacrificing effectiveness.
Proceedings of the 2020 Genetic and Evolutionary Computation Conference, 2020
Potable water distribution networks are requisites of modern cities. Because of the city expansio... more Potable water distribution networks are requisites of modern cities. Because of the city expansion, nowadays, the scale of the network grows rapidly, which brings great difficulty to its optimization. Evolutionary computation methods have been widely investigated on small-scale networks, but their performance is far from satisfactory on large-scale networks. Aimed at addressing this difficulty, a new memetic algorithm called level-based learning swarm optimizer with restart and local search is proposed in this paper to solve the large-scale water distribution network optimization problem. Instead of using traditional evolutionary computation algorithms, the level-based learning swarm optimizer that is especially proposed for large-scale optimization problems is applied as the populationbased optimizer. Two restart strategies are incorporated to make the algorithm more effective. They can help the algorithm jump out from local optima thus to increase its exploration ability. Moreover, a simple yet effective local search algorithm is proposed based on the domain knowledge to further refine the solutions after the algorithm converges. Experimental results on both single-source and multi-source large-scale water distribution networks show that the proposed algorithm is more effective than the state-of-the-art evolutionary computation algorithms. CCS CONCEPTS • Computing methodologies → Discrete space search; Randomized search; • Applied computing → Computer-aided design;
Evolutionary multitask learning has achieved great success due to its ability to handle multiple ... more Evolutionary multitask learning has achieved great success due to its ability to handle multiple tasks simultaneously. However, it is rarely used in the hyper-heuristic domain which aims at generating a heuristic for a class of problems rather than solving one specific problem. The existing multitask hyperheuristic studies only focus on heuristic selection, which is not applicable to heuristic generation. To fill the gap, we propose a novel multitask generative hyper-heuristic approach based on genetic programming in this paper. Specifically, we introduce the idea in evolutionary multitask learning to genetic programming hyper-heuristics with a suitable evolutionary framework and individual selection pressure. In addition, an origin-based offspring reservation strategy is developed to maintain the quality of individuals for each task. To verify the effectiveness of the proposed approach, comprehensive empirical studies have been conducted on the homogeneous and heterogeneous multitask dynamic flexible job shop scheduling. The results show that the proposed algorithm can significantly improve the quality of scheduling heuristics for each task in all the examined scenarios. In addition, the evolved scheduling heuristics verify the mutual help among the tasks in a multitask scenario.
Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2019
Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation prob... more Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation problem. Genetic Programming (GP) has shown effectiveness in automatically evolving routing policies to handle the uncertain environment in UCARP. However, when the scenario changes, current routing policy can no longer work effectively, and one has to retrain a new policy for the new scenario which is time consuming. On the other hand, knowledge from solving the previous similar scenarios may be helpful in improving the efficiency of the retraining process. In this paper, we propose different knowledge transfer methods from a source scenario to a similar target scenario and examine them in different settings. The experimental results showed that by knowledge transfer, the retraining process is made more efficient and the same performance one can obtained within a much shorter time without having any negative transfer.
2020 IEEE Symposium Series on Computational Intelligence (SSCI), 2020
Amid continuing increases in purchasing power and Internet shopping, demands for more efficient a... more Amid continuing increases in purchasing power and Internet shopping, demands for more efficient algorithms to solve the Large-Scale Vehicle Routing Problems are to be expected. The problem, which consists of finding routes for visiting customers, however, is very hard to solve due to its combinatorial nature. This becomes more notable when looking at large scale instances, i.e at least 200 customers, requiring sophisticated solution methods to solve, mostly heuristic-based. The existing meta-heuristics apply some limits to the solution space for handling the large scale nature. However, this is typically done manually, based on extensive domain knowledge. Hyper-Heuristics aim to reduce domain knowledge dependence, but current works disregard search space size, presenting poor scalability. In this paper, we propose a new Hyper-Heuristic to automatically evolve effective heuristics, based on a Genetic Algorithm framework. The new Hyper-Heuristic adds limits to the neighbourhoods of Low-Level Heuristics. In the method, the large search space is dealt with an adaptive chromosome which limits the neighbourhood size for each operator. The goal is to show whether more effective limits can be found if done automatically. Results show that the automatically evolved neighbourhood size limits outperform fixed ones for most instances tested and manually designed limits.
Dynamic flexible job shop scheduling has received widespread attention from academia and industry... more Dynamic flexible job shop scheduling has received widespread attention from academia and industry due to its practical application value. It requires complex routing and sequencing decisions under unpredicted dynamic events. Genetic programming, as a hyper-heuristic approach, has been successfully applied to evolve scheduling heuristics for job shop scheduling due to its flexible representation. However, the simulationbased evaluation is computationally expensive since there are many calculations based on individuals for making decisions in the simulation. To improve training efficiency, this paper proposes a novel multi-fidelity based surrogate-assisted genetic programming. Specifically, multi-fidelity based surrogate models are first designed by simplifying the problem expected to be solved. In addition, this paper proposes an effective collaboration mechanism with knowledge transfer for utilising the advantages of multi-fidelity based surrogate models to solve the desired problems. This paper examines the proposed algorithm in six different scenarios. The results show that the proposed algorithm can dramatically reduce the computational cost of genetic programming without sacrificing the performance in all scenarios. With the same training time, the proposed algorithm can achieve significantly better performance than its counterparts in most scenarios while no worse in others.
2020 IEEE Congress on Evolutionary Computation (CEC), 2020
The Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation ... more The Uncertain Capacitated Arc Routing Problem (UCARP) is an important combinatorial optimisation problem with extensive real-world applications. Genetic Programming (GP) has shown effectiveness in automatically evolving routing policies to handle the uncertain environment in UCARP. However, whenever a UCARP scenario changes, e.g. when a new vehicle is bought, the previously trained routing policy may no longer work effectively, and one has to retrain a new policy. Retraining a new policy from scratch can be time-consuming but the transfer of knowledge gained from solving the previous similar scenarios may help improve the efficiency of the retraining process. In this paper, we propose a novel transfer learning method by learning the probability distribution of good solutions from source domains and modelling it as a probabilistic prototype tree. We demonstrate that this approach is capable of capturing more information about the source domain compared to transfer learning based on (sub-)tree transfers and even create good trees that are not seen in source domains. Our experimental results showed that our method made the retraining process more efficient and one can obtain an initial state for solving difficult problems that is significantly better than existing methods. The final performance of all algorithms, were comparable, implying that there was no negative transfer.
Dynamic flexible job shop scheduling (DFJSS) is a challenging combinational optimisation problem ... more Dynamic flexible job shop scheduling (DFJSS) is a challenging combinational optimisation problem that takes the dynamic environment into account. Genetic programming hyperheuristics (GPHH) have been widely used to evolve scheduling heuristics for job shop scheduling. A proper selection of the terminal set is a critical factor for the success of GPHH. However, there is a wide range of features that can capture different characteristics of the job shop state. Moreover, the importance of a feature is unclear from one scenario to another. The irrelevant and redundant features may lead to performance limitations. Feature selection is an important task to select relevant and complementary features. However, little work has considered feature selection in GPHH for DFJSS. In this paper, a novel two-stage GPHH framework with feature selection is designed to evolve scheduling heuristics only with the selected features for DFJSS automatically. Meanwhile, individual adaptation strategies are proposed to utilise the information of both the selected features and the investigated individuals during the feature selection process. The results show that the proposed algorithm can successfully achieve more interpretable scheduling heuristics with fewer unique features and smaller sizes. In addition, the proposed algorithm can reach comparable scheduling heuristic quality with much shorter training time.
Due to its direct relevance to post-disaster operations, meter reading and civil refuse collectio... more Due to its direct relevance to post-disaster operations, meter reading and civil refuse collection, the Uncertain Capacitated Arc Routing Problem (UCARP) is an important optimisation problem. Stochastic models are critical to study as they more accurately represent the real world than their deterministic counterparts. Although there have been extensive studies in solving routing problems under uncertainty, very few have considered UCARP, and none consider collaboration between vehicles to handle the negative effects of uncertainty. This article proposes a novel Solution Construction Procedure (SCP) that generates solutions to UCARP within a collaborative, multi-vehicle framework. It consists of two types of collaborative activities: one when a vehicle unexpectedly expends capacity ( route failure), and the other during the refill process. Then, we propose a Genetic Programming Hyper-Heuristic (GPHH) algorithm to evolve the routing policy used within the collaborative framework. The ...
The uncertain capacitated arc routing problem is of great significance for its wide applications ... more The uncertain capacitated arc routing problem is of great significance for its wide applications in the real world. In the uncertain capacitated arc routing problem, variables such as task demands and travel costs are realised in real time. This may cause the predefined solution to become ineffective and/or infeasible. There are two main challenges in solving this problem. One is to obtain a high-quality and robust baseline task sequence, and the other is to design an effective recourse policy to adjust the baseline task sequence when it becomes infeasible and/or ineffective during the execution. Existing studies typically only tackle one challenge (the other being addressed using a naive strategy). No existing work optimises the baseline task sequence and recourse policy simultaneously. To fill this gap, we propose a novel proactive-reactive approach, which represents a solution as a baseline task sequence and a recourse policy. The two components are optimised under a cooperative ...
Dynamic job shop scheduling (DJSS) problems are combinatorial optimisation problems that have bee... more Dynamic job shop scheduling (DJSS) problems are combinatorial optimisation problems that have been extensively studied in the literature due to their difficulty and their applicability to real-world manufacturing systems, e.g., car manufacturing systems. In a DJSS problem instance, jobs arrive on the shop floor to be processed on specific sequences of machines on the shop floor and unforeseen events such as dynamic job arrivals and machine breakdown occur that affect the properties of the shop floor. Many researchers have proposed genetic programming based hyper-heuristic (GP-HH) approaches to evolve high quality dispatching rules for DJSS problems with dynamic job arrivals, outperforming good man-made rules for the problems. However, no GP-HH approaches have been proposed for DJSS problems with dynamic job arrivals and machine breakdowns, and it is not known how well GP generalises over both DJSS problem instances with no machine breakdown to problem instances with machine breakdown. Therefore, this paper investigates the generality of GP for DJSS problem with dynamic job arrivals and machine breakdowns. To do this, a machine breakdown specific DJSS dataset is proposed, and an analysis procedure is used to observe the differences in the structures of the GP rules when evolved under different machine breakdown scenarios. The results show that performance and the distributions of the terminals for the evolved rules is sensitive to the frequency of machine breakdowns in the training instances used to evolve the rules.
Evolutionary Computation in Combinatorial Optimisation, 2014
Feature selection is an important but difficult task in classification, which aims to reduce the ... more Feature selection is an important but difficult task in classification, which aims to reduce the number of features and maintain or even increase the classification accuracy. This paper proposes a new particle swarm optimisation (PSO) algorithm using statistical clustering information to solve feature selection problems. Based on Gaussian distribution, a new updating mechanism is developed to allow the use of the clustering information during the evolutionary process of PSO based on which a new algorithm (GPSO) is developed. The proposed algorithm is examined and compared with two traditional algorithms and a PSO based algorithm which does not use clustering information on eight benchmark datasets of varying difficulty. The results show that GPSO can be successfully used for feature selection to reduce the number of features and achieve similar or even better classification performance than using all features. Meanwhile, it achieves better performance than the two traditional feature selection algorithms. It maintains the classification performance achieved by the standard PSO for feature selection algorithm, but significantly reduces the number of features and the computational cost.
This paper describes two innovations that improve the efficiency and effectiveness of a genetic p... more This paper describes two innovations that improve the efficiency and effectiveness of a genetic programming approach to object detection problems. The approach uses genetic programming to construct object detection programs that are applied, in a moving window fashion, to the large images to locate the objects of interest. The first innovation is to break the GP search into two phases with the first phase applied to a selected subset of the training data, and a simplified fitness function. The second phase is initialised with the programs from the first phase, and uses the full set of training data with a complete fitness function to construct the final detection programs. The second innovation is to add a program size component to the fitness function. This approach was applied to three object detection problems of increasing difficulty. The results indicate that the innovations increased both the effectiveness and the efficiency of the genetic programming search, and also that the genetic programming approach was more effective than a neural network approach.
Abstract. The cooperative coevolution paradigm decomposes a large problem into a set of subcompon... more Abstract. The cooperative coevolution paradigm decomposes a large problem into a set of subcomponents and solves them independently in order to collectively solve the large problem. This work introduces a novel encoding scheme for building subcomponents based on functional prop-erties of ...
Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search proble... more Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search problem by decomposing it into low-dimensional subcomponents. Efficient problem decomposition methods or encoding schemes group interacting variables into separate subcomponents in order to solve them separately where possible. It is important to find out which encoding schemes efficiently group subcomponents and the nature of the neural network training problem in terms of the degree of nonseparability. This paper introduces a novel encoding scheme in cooperative coevolution for training recurrent neural networks. The method is tested on grammatical inference problems. The results show that the proposed encoding scheme achieves better performance when compared to a previous encoding scheme.
The 2011 International Joint Conference on Neural Networks, 2011
In this paper, an adaptive modularity cooperative coevolutionary framework is presented for train... more In this paper, an adaptive modularity cooperative coevolutionary framework is presented for training feedforward neural networks. The modularity adaptation framework is composed of different neural network encoding schemes which transform from one level to another based on the network error. The proposed framework is compared with canonical cooperative coevolutionary methods. The results show that the proposal outperforms its counterparts in terms of training time, success rate and scalability.
The 2011 International Joint Conference on Neural Networks, 2011
Memetic algorithms and cooperative coevolution are emerging fields in evolutionary computation wh... more Memetic algorithms and cooperative coevolution are emerging fields in evolutionary computation which have shown to be powerful tools for real-world application problems and for training neural networks. Cooperative coevolution decomposes a problem into subcomponents that evolve independently. Memetic algorithms provides further enhancement to evolutionary algorithms with local refinement. The use of crossover-based local refinement has gained attention in memetic computing. This paper employs a cooperative coevolutionary framework that utilises the strength of local refinement via crossover. The framework is evaluated by training recurrent neural networks on grammatical inference problems. The results show that the proposed approach can achieve better performance than the standard cooperative coevolution framework.
Artificial intelligence (AI) emphasises the creation of intelligent machines/systems that functio... more Artificial intelligence (AI) emphasises the creation of intelligent machines/systems that function like humans. AI has been applied to many real-world applications. Machine learning is a branch of AI based on the idea that systems can learn from data, identify hidden patterns, and make decisions with little/minimal human intervention. Evolutionary computation is an umbrella of population-based intelligent/learning algorithms inspired by nature, where New Zealand has a good international reputation. This paper provides a review on evolutionary machine learning, i.e., evolutionary computation techniques for major machine learning tasks such as classification, regression and clustering, and emerging topics including combinatorial optimisation, computer vision, deep learning, transfer learning, and ensemble learning. The paper also provides a brief review of evolutionary learning applications, such as supply chain and manufacturing for milk/dairy, wine and seafood industries, which are important to New Zealand. Finally, the paper presents current issues with future perspectives in evolutionary machine learning.
Uploads
Papers by Mengjie Zhang