Multi-agent manufacturing control: an industrial case study

MULTI-AGENT MANUFACTURING CONTROL: AN INDUSTRIAL CASE STUDY Bart Saint Germain, Paul Valckenaers, Hendrik Van Brussel, Hadeli, Olaf Bochmann, Constantin Zamfirescu, Paul Verstraete Abstract: The paper discusses the development of a multi-agent manufacturing control system for an industrial application. This multi-agent system follows the PROSA reference architecture and applies an ant colony design (stigmergy) to provide coordination and emergent forecasting services. The paper first presents the experimental set-up: • The properties of this shop and its particular challenges to control systems • The emulation of an existing production shop in industry • The multi-agent system connected to this emulation Next, the paper discusses the real-time aspect of the emulation. Third, the multiagent control system is presented. Finally, the paper discusses the browser to analyze the results generated during simulation runs. Keywords: Multi-agent systems, manufacturing control, industrial case study. 1. INTRODUCTION A manufacturing control system manages the internal logistics in a production system; it determines the routings of product instances, the assignment of workers and components, the starting of the processes on not-yet-finished product instances, aso. Manufacturing control does not control the manufacturing processes themselves, but has to cope with the consequences of the processing results (e.g. the routing of products to a repair station). Past experience revealed that manufacturing control cannot be implemented through ’one shot scheduling solutions’ (Wegner, 1997). On the shop floor, schedules have acquired a reputation for rapidly becoming invalid because of frequent changes and disturbances. Therefore, the multi-agent manufacturing control, which is discussed in this paper, treats changes and disturbances as business-as-usual. It considers manufacturing as a non-stop ’going concern’. Moreover, the multi-agent design incorporates the approach from object-oriented design which states that the problem has to be a part of the solution in order to make robust and flexible software (Jackson, 1997). 2. THE PROPERTIES OF THE SHOP AND ITS PARTICULAR CHALLENGES TO CONTROL SYSTEMS The case study in this paper concerns making photographic products out of large rolls of photographic foil. The factory in the case study can be categorized as a flow shop with some routing and storage flexibilities. Furthermore, the manufacturing processes in the case study strongly resemble disassembly operations in which one product unit turns into multiple units when it is cut into smaller pieces. This section first introduces a general overview of the different product types in the shop. Then, some items like the slitting plan and priorities are discussed. (see figure 1) ProductionOrder * CustomerOrder * 1 MasterRoll 1 * Reel 1 * SmallRoll Unit * StackOfSheets Fig. 1. The topology of this case study 2.1 The case study 2.3 The slitting plan In the case study, the system receives customer orders. A customer order comprises one or more product units. A product unit can either be a stack of sheets or a roll of a given kind of photographic paper. These rolls and sheets are made out of big rolls, called master rolls, by dividing these big rolls into smaller pieces. In this way, these master rolls can be associated with multiple customer orders, which only require some part of the big roll. Conversely, a customer order can be associated with multiple master rolls since the units in a single customer order can specify different types of foil (each master roll only contains one type of foil). In other words, there is a manyto-many relationship between master rolls and customer orders in the system. All the ordered production units have to be mapped on the master rolls. The material on a master roll is expensive; even recycling this material represents a very significant expense. Therefore, the company has to reduce losses on the roll as much as possible when the customer orders are mapped onto the master rolls. The strategy used to minimize these losses is to combine several customer orders into one master roll such that almost every space is used. For further optimization, make-to-stock orders are used to fill the gaps. Instead of recycling the material, asyet-unsold products are made; their selection is based on market forecasts, current stock levels and the properties of the gaps that remain after the customer orders have been fitted to the master rolls. To form a product, the master roll has to be split first lengthwise, this is called slitting. This operation divides the master roll into pieces, called reels. The second operation splits the reels along the width of the original master roll. Depending on the products to be made, the operation is either cutting or rewinding. The final product is a stack of sheets or a small roll respectively. 2.2 Priorities Only customer orders have a visible priority. It is very difficult to give static priorities to the master rolls, reels... The only way to give real priorities to the master roll or the reel is to compose all priorities of the associated customer orders. To compose the customer priorities, the system must be as traceable as possible so that the due date may be recalculated whenever the situation changes. For example: a reel composed out of three customer order, two with low priority and the third one with a high priority will get a high priority. 2.4 Disturbances In reality, production frequently suffers disturbances. These disturbances include planned and unexpected maintenance, which results in variations in production speeds. More seriously, some disturbances imply damage to the photographic film. For example: a burr in a slitting machine can make a scratch on the foil. In this case, the slitting plan becomes invalid and, possibly, some customer orders may not be produced any more out of this master roll. In such situations, the control and planning systems have to cooperate to re-insert this customer order elsewhere and salvage as much as possible from the damaged foil by making another product out of it. 3. THE EMULATION OF AN EXISTING PRODUCTION SHOP IN INDUSTRY While developing a new factory control system, there are some properties that needs to be investigated: 3.2 Discrete event emulation DES Control Emulation reflects Reality Fig. 2. Simulation versus Emulation • key performance indicators (K.P.I.) such as lead time, due date, and throughput; • computational parameters such as the complexity of the algorithms; • the impact of specific control system parameters such as information sampling rates. For practical reasons, it is impossible to perform the investigation on a real system. This is not only prohibitively costly and time-consuming. It would also be impossible to reproduce the same conditions for a number of test runs when trying to compare alternative designs or control parameter settings. One way to overcoming this is to build a simulation. By using such a simulation model, numerous controlled experiments may be performed at low cost. The research developments discussed in this manuscript use a simulation with a specific internal structure, which is discussed next. 3.1 Emulation versus Simulation The simulation in the case study comprises of two parts: an emulation and a control subsystem (Figure 2) . The emulation provides the state of an emulated world to the control and the control sends the appropriate commands to the emulation. To the control subsystem, being connected to the emulation is indistinguishable from being connected to the real world. Consequently, it is easy to connect the control system to the real factory instead of its emulation. Furthermore, validation of the simulation observes that the one-toone correspondence between the real world manufacturing system and its emulation is sufficiently accurate. It is not necessary to check whether the control behaviour is adequately captured by the simulation since it is the same software code in all situations. The industrial case study in this paper uses a discrete event emulation to implement the emulation part of the overall simulation. This is different from discrete event simulation (Law and Kelton, 1991). Discrete event simulation concerns the modeling of a system, including its control, as it evolves over time by a representation in which the state variables change suddenly at certain points over time. These points in time are the ones at which an event occurs, where an event is defined as an instantaneous occurrence that may change the state of the system. In contrast, a discrete event emulation only reflects the behaviour of the underlying production system. The control system is not included. This differs from the typical discrete event simulation where the control is implemented into the discrete event model itself. For the remainder, the techniques used for the discrete event emulation are the same as those used in discrete event simulation software. 4. THE MULTI-AGENT SYSTEM CONNECTED TO THIS EMULATION The simulation approach presented in part 3 creates an opportunity for a novel type of service for shop floor operations. As presented above, the multi-agent manufacturing control is first connected to an emulation for development purposes. Afterward, the control systems will be connected to the real system. However, the management of the shop floor operations often has to take decisions without solid information about the consequences of its choices. To remedy this situation, the actual deployment of the multi-agent manufacturing control system no longer consists of replacing the emulation by the real world. The emulation remains available and can be synchronized with the current situation in the factory or with a short-term forecast, made available by the control, of the situation on the shop floor. From that point on, a copy of the control with the envisaged control settings and the emulation execute faster than in reality and provide feedback on how the production system would behave; multiple sessions in parallel allow a comparison between alternatives. In this way, unwise ideas are discovered before they do real damage. An example: client X asks to change the due date to get his products earlier. For the manager it is very difficult to foresee the impact of this intervention. This manager can start a simulation with the current or forecasted state and with the adapted due date. After some minutes the result of the simulation is available so that this manager has solid information to take a decision and/or to give a quote to the customer. 5. THE REAL-TIME ASPECT OF THE SIMULATION As mentioned in section 3, a discrete event emulation connected to the multi-agent control is the most appropriate way to build a simulation to test a factory control system. There is, however, one issue that prevents the straightforward use of an off-the-shelf discrete event simulation software to design the emulation. To enable fast execution of the test runs, the emulation aims to process events as fast as possible. Therefore, it ’jumps’ from the current event to the next event without delay. More over conventional simulation software assumes that the control system takes its decisions in a negligible amount of time. This assumption does not hold for the typical multi-agent manufacturing control system; the control needs time to think and to perform actions. As a consequence, control and emulation need to cooperate. The system tracks all control decisions that have to be taken and all control activities to be executed at a given point of time in the emulation. When there are no control system decisions or actions pending, the emulation jumps to the next event. In the other case, the emulation operates in real-time mode where one time unit in emulation will take one time unit in reality. Moreover, the control system may insert its own events into the emulation system (e.g. schedule the next sampling of information, the expiration date of a service offer...). 6. THE MULTI-AGENT CONTROL SYSTEM The introduction states that a one shot solution is insufficient; manufacturing control addresses the ’going concern’ of making production fast and smoothly. Furthermore, organizations become more and more flexible in order to react much faster to the demands of customers. To devise a control system capable of coping with this complexity, a stable layer is needed. This stable layer act as the foundation in the software so that the system does not break whenever a new situation is arises. A suitable architecture for such control systems is PROSA (Valckenaers, 2001). On this foundation ant-based systems are implemented in order to enhance the quality of decisions. 6.1 PROSA The PROSA architecture is built around three types of basic agents: order, product and resource agents. Each of them is responsible, respectively, for one aspect of manufacturing control: (i) internal logistics, (ii) recipes or process plans, and (iii) resource handling. Optionally, staff agents may be added to assist the basic agents with expert knowledge. 6.1.1. Product Agent Product Agents provide the process knowledge of a product. This means that a Product Agent can identify all possible subsequent production steps for certain state of an order. The implementation and representation of the recipe is internal to the Product Agent. The product agent is responsible for the process plan of different products. In the case study the process plan corresponds to the slitting plan. The Product Agent actually encapsulates the functions and the means of implementing the slitting plan. This slitting plan may be implemented either in a static or a dynamic way see (2.3). It could also have multiple possibilities like a choice between a cutter or a rewinder at the end of the process. 6.1.2. Resource Agent Each resource agent corresponds to a physical part in the manufacturing system. A resource agent corresponding to a machine provides the prior knowledge of this machine; it knows the supported operations, the production capacity, queuing strategy, aso. Machines like slitters, cutters and rewinders in the case study are Resource Agents. Other types of resource agents may also be integrated into the model. In our example, the speed of the machines and the accuracy of the operation depends on number of operators. To reflect the accuracy of the process parameters and the fluctuation of the speed, there are two options. A first solution is to model the number of operators as a parameter of the machine together with the impact of this parameter on the machine’s performance. The other option is to consider operators as separate resources, which is the more flexible and robust solution. Note however that this requires the order agents to manage multiple resource allocation, meaning that several resources must be available to run an operation. 6.1.3. Order Agent The order agent manages the physical product instance being produced, the product state model, and all logistic information processing related to the job. It is responsible for performing the assigned work correctly and on time. An order agent may represent customer orders, make-to-stock orders, production orders and orders to maintain and repair resources. The last type of orders, maintain and repair orders, have the highest priority to ensure the maintenance is done on time. The order agents in the case study reflect the structure of the actual situation in the plant. Order agents that correspond to a master roll create one order agent for each of the reels that emerge when their master roll goes through the slitting process. Likewise, reel order agents spawn the corresponding stack or small-roll agents when they undergo cutting or rewinding. All agents keep references to the agent that created them and delegate the proper enquiries to them. For instance, the customer order agent is systematically consulted and informed about due date information. In this way, a change of due date (by the customer or because some part of the delivery suffered a delay) will be seen rapidly and automatically by all agents involved. The two following subsections shortly discuss the control functionality that is developed on top of these PROSA agent by means of an ant colony design. (Valckenaers, 2001) gives a more detailed discussion. 6.2 Feasibility The primary concern of order agents is to get their product instance(s) manufactured. To achieve this goal, the hard constraints in the manufacturing system have to be accounted for. Hard constraints are constraints like the process sequence (reflected in the product agent) and deadlocks. Deadlocks do not present a problem in this case because there are no loops into the system. To make sure that the right sequence of processes is reached, the subnet capabilities of the shop are made available at each routing point in the system. To spread the capabilities of subnets of the shop around, the resource agent at the end of the production line creates, at a certain frequency, a mobile resource agent and sends it upstream. These mobile agents post information on local blackboards, which are attached to resource agents, describing the processing capabilities of the corresponding path discovered so far. This enables order agents to distinguish which processing capabilities are available along a given route. Whenever a transportation resource has multiple entries these mobile agents clone themselves to ensure the entire factory is covered by the mechanism. The feasibility information has a finite lifespan after which it is wiped from the local blackboard. The frequency at which the mobile resource agents are created by the resource at the end of the production system, is sufficiently high to ensure that the feasibility information is always available and up-to-date (in case of changes to the production system). 6.3 Exploring and forecasting The order agents explore the available possibilities to get themselves produced and select the most suitable one. This is done in an ant colony design (Dorigo, 2000). The main advantage of the antbased design is that individual agents are not exposed to the complexity and dynamics of the global situation. None of the agents need a mental map of the environment nor do these agents communicate amongst themselves about the environment. Also the communication is carried out in a very robust way by means of an evaporaterefresh mechanism. The exploring function works as follows. Each order agent creates ants (mobile agents) and sends them downstream through the system along possible paths which the order should follow. The ants act in a virtual way, without actually executing the operations. Along his path, the ant writes information on the local blackboards about the (lack of) success while following this solution. To have an idea of the queuing time on the different resources, intentions are sent through the factory. If all intention agents have the same decision criteria as their corresponding order agent and the order agent is likely to follow his intention, resource agents can calculate reliable forecasts of their load in the near future. These load forecasts influence the outcome of explorations by the ants and in turn the decision process of the Order Agent. 7. THE BROWSER TO ANALYZE THE RESULTS GENERATED DURING SIMULATION RUNS Finding a suitable way to visualize the emulation is very difficult. When the speed of the emulation increases the visualization becomes less and less realistic. And, there will be a point where the visualization is not visible anymore because the speed of the emulation is higher than the computer’s ability to visualize. This section briefly presents how to utilize a flexible browser to synchronize the visualization with the discrete event emulation. 7.1 Painters in the event calendar Section 3.2 explains how an discrete event emulation jumps from event to event as fast as possible. Executing an operation in the discrete event emulation takes no considerable time, but the animation needs some time to visualize this operation. A proper way to overcome this problem may be found in the discrete event emulation itself. Every time the event calendar arrives at a point where an event is inserted, all updaters are called to calculate their new state and these states are eventually painted in a visualisation window on the computer screen. If a smooth visualization is needed, the updater has to insert sufficient events, thus ensuring that the emulation progresses in small steps. To separate the logic part and the visualization part of the emulation, the event calendar is split up into several layers. The logic layer is responsible for the starting and the stopping of the machines. The animation layers are responsible for inserting the animation updaters. Synchronizing these two layers presents no difficulty because the states reflect the reality and every updater works only with these states. 7.2 Off-line analysis Section 7.1 describes a way in which the emulation may be visualized in a pure discrete event way. The animation is provided during the simulation run. Having only an on-line browser, which visualizes what is happening in the factory, has the following drawbacks: • adding or changing a K.P.I. implies running a new simulation; • the user may want to ’rewind’ the simulation to examine the information more closely to understand what is happening; • the user may not want to sit through the whole simulation run; more likely, s/he would prefer to be able to jump forward to moments when something interesting is happening. To manage these problems an off-line browser was developed. While simulating, all states of the emulation entities are written into serialized objects. These objects are recallable after the simulation process. In this way the above problems are solved. By using the analyzer, it is possible to observe the results more intentionally and thoroughly. 8. EPILOGUE In this paper, conceptual concepts like PROSA and stigmergy are mapped on a real case. The case represents a shop which has some special characteristics similar to a disassembly shop. The strength of PROSA, the ability to reflect the situation, is used to model the complex structure of the orders, customer orders and all types of production orders. PROSA, especially the cooperation between the product and the order agents, tackled also the disassembly property of the shop and the links between the different order agents formed a nice traceable system. To overcome the cost and time to do tests in the real system, a simulation approach is presented which is able to reuse the control part and where the emulation creates new possibilities. The visualization as presented above is done in two ways, either an on line browser to interact with the simulation and an off line browser to analyze the simulation run in more detail. To conclude, PROSA and stigmergy established a flexible and traceable system able to handle the dynamics of the system. REFERENCES Dorigo, M., Bonabeau E. Theraulaz G. (2000). Ant algorithms and stigmergy. In: Future Generation Computer Systems. pp. 851–871. Number 16. Jackson, M. A. (1997). Principles of Program Design. Academic Press. Law, A. M. and W. D. Kelton (1991). Simulation Modeling and Analysis. M-H series in industrial engineering and management science. second ed.. McGraw-Hill, Inc. Valckenaers, P, Van Brussel H. Kollingbaum M. Bochmann O. (2001). Multi-agent coordination and control using stigmergy applied to manufacturing control.. In: Lecture Notes in Artivicial Intelligence. pp. 317–334. Number 2086. Springer-Verlag. Wegner, P. (1997). Why interaction is more powerful than algoritms.. Communications of the ACM 40(5), 80–91.