An Improved Complexity Measure in Hierarchical Fuzzy Systems

An Improved Complexity Measure in Hierarchical Fuzzy Systems Tajul Rosli Razak1,2 , Jonathan M. Garibaldi1 and Christian Wagner1 1 Laboratory for Uncertainty in Data and Decision Making (LUCID), School of Computer Science, University of Nottingham, United Kingdom. 2 Faculty of Computer and Mathematical Science, Universiti Teknologi MARA, Perlis Branch, Malaysia. Email: 1 {tajul.razak,jon.garibaldi,christian.wagner}@nottingham.ac.uk, 2 [email protected] Abstract—Interpretability is an important and necessary topic that needs to be discussed in relation to the fields of Artificial Intelligence and Machine Learning. Within fuzzy logic systems (FLSs), hierarchical fuzzy systems (HFSs) have been suggested as a key component to help improve the interpretability of FLSs. In this context, complexity is a key component in the interpretability of FLSs. In FLSs, the complexity is commonly expressed using in a rule-based manner, considering the number of rules, variables, and fuzzy terms. Several studies have used indicators (for example, the number of rules) to measure the complexity of FLSs. However, this is not a perfect way of assessing complexity in HFSs that have the structure of multiple subsystems, layers and different topologies. Thus far, complexity assessment associated with the structure of HFSs has not been discussed. In this paper, we aim to put forward a new approach in assessing the complexity of HFSs, which will combine rule-based complexity and structural complexity. A detailed measurement of complexity for different HFSs’ topologies, namely parallel and serial, will be presented to showcase the features of the new approach. The contribution of this paper is the introduction of a combined rule-based and structural complexities-based approach in order to establish a comprehensive measurement of complexity in HFSs. Index Terms—Hierarchical fuzzy systems, Complexity, Rulebased complexity, Structural complexity. I. I NTRODUCTION The field of FLSs has been making rapid progress in recent years. There has been an increasing number of works in many areas such as science, manufacturing, business and also in the medical domain for decision making. Many researchers [1]–[3] have used fuzzy systems as a tool for controlling and modelling in many fields, proving it to be a useful technology. One of the most important motivations for using FLSs for system modelling is that an FLS uses linguistic variables and rules [4] that are easy to understand. Moreover, FLSs are good at capturing the complexity of a wide range of problems through their linguistic modelling and approximate reasoning capabilities [5]. However, the FLS rule-based structure poses significant challenges, including the curse of dimensionality, in which the number of required rules and the model complexity commonly increase exponentially with the number of input variables [6], [7], thus potentially reducing the transparency and interpretability of FLSs. In exploring this problem, several methods have been proposed for optimising the size of the rule-based structure in FLSs, such as rule selection [8], feature selection [9], rule interpolation [10], singular-value 978-1-7281-6932-3/20/$31.00 ©2020 IEEE decomposition-QR [11], evolutionary algorithms [12], fuzzy similarity measures [7], rule learning [13] and hierarchical fuzzy systems (HFSs) [14]. HFSs have been shown to be an effective way to reduce the number of rules in FLSs, which in turn reduces the model complexity and improves its interpretability. Indeed, in literature, it has been claimed that reducing the complexity is a way of improving the model interpretability in FLSs [7]. Thus, complexity appears to be a key component in the interpretability of FLSs. However, to fully understand the concept of interpretability and complexity in HFSs is a challenge and still remains uncertain. In our previous work, as an introductory step to understanding the interpretability of HFSs, an initial index was proposed as a way to measure their interpretability [15] and further proposes a generic index in [16]; this considered the structure of multiple subsystems, layers and different topologies. However, since interpretability is of a subjective nature, it is not clear as to how different people may perceive interpretability in HFSs. Hence, user perception of interpretability and complexity in HFSs was then investigated with the aid of a user study [17]. It was found that users had different perceptions regarding interpretability and complexity. For instance, the rules and structure affected the users’ perceptions of interpretability and complexity. Therefore, the current paper aims to focus on understanding the complexity of HFSs, and to identify the requirement of measuring the complexity in HFSs. Although researchers usually measure complexity in FLSs by looking principally at the rule-based complexity level [7], [18]–[20], this is not necessarily strong enough research especially for HFSs that have a structure of multiple subsystems, layers and different topologies. Therefore, as an initial approach, we introduced a way of assessing the structural complexity in HFSs adapted from Software Engineering [21]. In this paper, we propose a more comprehensive way of assessing the complexity of HFSs. This extends our previous work [21] that only covered the structural complexity of HFSs. Here, a new approach to complexity measurement in HFSs is discussed, focusing on the combination of rule-based complexity and structural complexity, in order to reach a more comprehensive understanding (together with measures of) the overall interpretability of HFSs. The rest of this paper is organised as follows. Section II discusses background on the state-of- the-art of interpretability, complexity in FLSs and other fields, and also hierarchical fuzzy systems. Section III introduces a new approach to complexity measurement in HFSs that consider both rule-based complexity and structural complexity. Section IV demonstrates the new approach of complexity measurement with a realworld example, specifically seesaw control systems. Finally, Section V present results of the complexity measure, with Section VI providing concluding remarks together with some suggestions for future work. II. BACKGROUND In this section, we briefly provide background in respect to interpretability, complexity in FLSs, complexity in other fields and HFSs. A. Interpretability Interpretability refers to the capability of FLSs to express the behavior of the system in an understandable way [22]. In recent years, the interest of researchers in obtaining more interpretable fuzzy models has increased. However, the choice of an appropriate interpretability measure is still an open question due to its subjective nature and the large number of factors involved. Gacto et al. [23] introduced a taxonomy for assessing interpretability of FLSs, that includes the two key components; (i) complexity-based interpretability; and (ii) semantic-based interpretability. Complexity-based interpretability is devoted to decreasing the complexity of the obtained model (usually measured as the number of rules, variables, labels per rule, and other factors). Meanwhile, semantic-based interpretability is dedicated to preserving the semantics associated with the membership functions (MFs), to ensure semantic integrity by imposing constraints on the MFs or approaches considering measures such as distinguishability, coverage and other factors. Also, Gacto’s taxonomy identifies that complexity is an essential component to determine the interpretability of FLSs. Thus far, complexity has often been used as an indirect measurement of the interpretability of FLSs. Several researchers claim that the reduction of complexity in a system can lead to better interpretability of fuzzy systems [7]. B. Complexity in FLSs In FLSs, complexity could be related to the specific problem described by the fuzzy model. In other words, from the structural analysis of a knowledge base, we should expect to gain information concerning the complexity of the underlying problem [24]. Complexity is usually evaluated by a simple measure mainly looking at the rule-based complexity. 1) Rule-based complexity: Rule-based complexity is primarily related to the readability of the knowledge base [25]; that is, the complexity of the rule base is minimised so as to improve its readability [24]. Standards which are used to measure rule-based complexity include the number of rules, variables, and labels per rule, amongst others [26], [27]. Furthermore, the study by Razak et al. [17] has shown the relationship between the number of inputs, the number of rules and the complexity in FLSs to be as follows: ↑ Number of inputs (FLSs), ↑ Number of rules (FLSs) (1) ↑ Number of rules (FLSs), ↑ Complexity (FLSs) (2) where ↑ denotes increasing and ↓ indicates decreasing. The relationships of (1) and (2), indicate that increasing the number of inputs in FLSs will increase the number of rules in FLSs and consequently, this will also increase the complexity of FLSs. In the related literature, there are several ways to measure the rule-based complexity in FLSs. For example, Nauck [28] has defined the complexity of an FLS as measurable by the number of classes (NC) divided by the total number of premises (NP). This complexity measurement is a part of his proposed interpretability index – Nauck’s index. The complexity, Comp, is computed as: NC (3) NP An FLS model is less complex when the rule-based complexity Comp gets higher and more complex when the rule-based complexity Comp decreases. Another study, by Alonso et al. [29], measured the complexity by one minus the complexity value in (3), which can be expressed as follows: Comp = C = 1 − Comp (4) An FLS model is more complex when the rule-based complexity C is close to 1 and less complex when the rule-based complexity C is close to 0. C. Complexity in others fields Complexity arises from either the structure of the interactions between very similar units, or from the units and the interactions themselves having specific characteristics. In both cases, the abstract representation of a complex system can be achieved by a collection of nodes (units) and edges (representing interactions between the units) forming a network (or graph) [30]. One common view of a complexity measure is that it is dependent on the number of structural features contained within an organisation rather than simply on the number of its basic elements [31]; this idea is also known as structural complexity. 1) Structural complexity: Structural complexity is an attribute of any general type of system. This attribute can be assessed by different measures, and it is often linked to interaction among various properties of the given system, such as nodes, edges and network (topology structure) [30], [32]. McCabe in [33] proposed a graph-theoretic structural complexity measure that measures and controls the number of paths through a programme. The complexity measure developed here is defined in terms of basic paths that when taken in combination will generate every possible path. The cyclomatic 1) Limpid - Hierarchical Fuzzy Systems (L-HFS): Lee at al. [36] proposed a new method to overcome the issue of an intermediate output that does not possess physical meaning in the middle layer. They introduced a new mapping rule base called L-HFS in order to obtain the HFS rule base. They claim that, by implementing L-HFS, a smaller rule base is achievable with the same input-output model as in the original FLS. Furthermore, the design of the fuzzy rule involved in this approach is easier than in a conventional HFS. Fig. 1. One main program M, two subroutine A and B. Adapted from McCabe [33]. number v(G) of graph G with n vertices, e edges, and p connected components is: v(G) = e − n + 2p (5) The overall strategy involves measuring the complexity of a programme by computing the number of linearly independent paths v(G). However, calculating the complexity of a collection of programmes (p 6= 1 ), particularly a hierarchical nest of subroutines, (5) becomes: V G  [ = e − n + 2p (6) i=1 S where, denotes a summation complexity of collection of programmes. For instance, assuming a main program M and two called subroutines A and B having a control structure as shown in Fig. 1. Let us denote the total graph above with 3 connected components as M ∪ A ∪ B. Now, since p = 3, the complexity can be computed using (6) as follows: v(M ∪ A ∪ B) = e − n + 2p = 13 − 13 + 2 × 3 = 6 A graph is more complex when the structural complexity v(G) value is large and less complex when the v(G) value is small. In this paper, we will assess the structural complexity of HFS using a measure in [21], which is adapted from McCabe’s measure. D. Hierarchical Fuzzy Systems HFSs are characterized by composing the input variables into a collection of low-dimensional fuzzy logic subsystems [14], [34]. HFSs can be illustrated as a cascade structure in which the output of each layer is considered as an input to the following layer, as shown in Fig. 2 (see Section IV). Also, a system that goes from one layer as shown in Fig. 2 to two or more layers as in Fig. 4 (see Section IV) has fewer rules than those in a FLS with one layer. The most extreme reduction of rules will be if the structure of the HFS has two input variables for each subsystem and has (n − 1) layers [14]. However, the fundamental issue with this HFS is to handle the (potentially) reduced physical meaning of the intermediate output variable(s) — which consequently makes them more difficult to design, with increased model complexity [35]. III. A C OMPREHENSIVE M EASUREMENT OF C OMPLEXITY IN H IERARCHICAL F UZZY S YSTEMS A comprehensive measurement of complexity is needed to evaluate complexity in HFSs that have multiple subsystems, layers and different topologies. A new approach to complexity measurement referred to as CHF S (for HFS complexity) is proposed that combines rule-based complexity and structural complexity. The CHF S process for measuring complexity in HFSs is explained in the following steps. A. Step 1: Calculating the Rule-based Complexity of HFSs Although there are several ways of measuring rule-based complexity in FLSs, it is challenging to measure the complexity of HFSs that have multiple subsystems. It should be noted that each subsystem will have a small number of inputs and outputs, and a small rule base that will serve a single purpose [37]. For an initial solution, in this step, equation (4) is used to calculate the rule-based complexity of each subsystem in a given HFS. However, another challenge is to aggregate all the rule-based complexity values at each subsystem across and within layers, in obtaining the overall rule-based complexity of HFSs. For this challenge, an aggregation strategy namely a layer-weighted average strategy has been proposed in [15] which considers all subsystems across and within layers. Therefore, in this step, the layer-weighted average strategy is used to obtain the final rule-based complexity of a given HFS. This strategy now becomes as follows:  sj q  X X Cjk /sj , lj (7) CRB = j=1 k=1 where Cjk is the complexity value associated with the subsystem k at layer j, for example as shown in (4), lj is the weight associated with layer j of the HFS, sj is the number of subsystems located in layer j, s is the total number of subsystems, q is the number of layers of the HFS and CRB is the rule-based complexity of the HFS. An HFS model is less complex when the CRB is close to 0 and more complex when CRB is close to 1. B. Step 2: Calculating the Structural Complexity of HFSs As discussed in Section II, structural complexity in the case of HFSs is related to the interaction amongst its elements; that is, the multiple subsystems, layers and different topologies. However, to evaluate structural complexity in HFSs is a challenging task, particularly when quantifying the interaction between the HFSs’ elements. So far, however, assessment on the structural complexity in HFSs has not been investigated. Therefore, in this step, as an initial solution, a wellestablished measure, namely the McCabe measure [33] (as discussed in Section II-C) is used to measure the complexity in the structure of HFSs. In order to that, an approach of mapping the McCabe measure to the HFSs’ design was introduced previously as follows [21]: s  [ CS = l − x + 2s (8) i=1 where x indicates the number of input variables in HFSs, l indicates the number of layers in HFSs, and s indicates the number of subsystems in HFSs. Also, CS indicates the nonnormalised structural complexity in HFSs. C. Step 3: Overall Measurement of the Complexity of HFSs The final challenge is to combine the rule-based complexity and structural complexity calculation from Step 1 and Step 2 respectively in order to obtain the overall complexity of the HFS. However, to combine both rule-based complexity and structural complexity is not a straightforward task. In this case, two important questions are examined: (i) “What approach might be used to make both rule-based complexity and structural complexity comparable in that both should have the same scale of output?” (ii) “What is the best operator to combine rule-based complexity and structural complexity?”. Therefore, in order to answer these questions, a method to address the challenges is proposed and explained in the following subsection. 1) Normalising the structural complexity values to be in the same range as rule-based complexity values: McCabe’s measure ranges from [1, ∞] whereas CS range from [−∞, ∞]. Whilst there is no inherent problem with this, other measures of interpretability for FLSs (such a Nauk’s index [28] or Alonso et al’s index [38]) have traditionally been defined over [0, 1]. Given this, it will be easier to subsequently combine an HFS complexity measure with other components of interpretability if it lies over the same range. Thus, in this paper, the complexity values of CS will be normalised to the range [0, 1]. There are several alternative functions that can be used but for this paper, one of the principal functions in mathematics, which is an inverse trigonometric function is chosen. This function can be expressed as follows: C̃S = 2 arctan(CS ) + π 2π (9) where arctan() is the inverse of the tangent function and C̃S indicates the normalised structural complexity in HFSs. An HFS model is less complex when the C̃S is close to 0 and more complex when C̃S is close to 1, in terms of structural complexity. 2) Combining the structural and rule-based complexity using a generic operator: In order to combine structural complexity and rule-based complexity, it may be sufficient to use an aggregation operator that selects something between min and max values. Alternatively, it also could be something else such as a t-conorm operator. Hence, in the second challenge, several options are employed to combine the aforementioned rule-based complexity and structural complexity, such that: M C˜S (10) CHFS = CRB L where indicates the generic operator 1 . In this case, the generic operator will represent basic aggregation operators namely min, max and mean, and also another operator strategy that is a t-conorm operator, specifically one of probabilistic sum, bounded sum and Einstein sum. These alternative aggregation and t-conorm operators are as follows: (i) Aggregation operator: min CHFS = MIN(CRB , C˜S ) (ii) Aggregation operator: max CHFS = MAX(CRB , C˜S ) (iii) Aggregation operator: mean CHFS = MEAN(CRB , C˜S ) (iv) T-conorm operator: probabilistic sum CHFS = CRB + C˜S − CRB · C˜S (v) T-conorm operator: bounded sum CHFS = MIN(CRB + C˜S , 1) (vi) T-conorm operator: Einstein sum CRB + C˜S 1 + CRB · C˜S An HFS model is expected to be less complex when the CHFS is close to 0 and expected to be more complex when the CHFS is close to 1. CHFS = IV. E XPERIMENTS AND R ESULTS In this experiment, the example of a seesaw control application is used which tackles the problem of balancing a seesaw using an FLS [36]. The involved parameters of the seesaw are the distance of the cart (x1 ), the angle that the wedge makes with the vertical line (x2 ), the height of the wedge (x3 ), and the centre of mass of the wedge (x4 ). The topology and complete rules set for this FLS (81 rules) can be seen in Fig. 2 and Table I, respectively. This experiment aims to explore the features of the proposed complexity measurement by using a seesaw control as an example. The experiment process consists of three main steps: 1) produce a Parallel L-HFS and a Serial L-HFS for the Seesaw Control; 2) measure the complexity of the Parallel L-HFS and the Serial L-HFS by using the proposed measure; 3) measurement of the overall complexity of the Parallel and Serial L-HFS. 1 In this paper, we consider that both qualities (C ˜ RB and CS ) are equally important (weight). Clearly, it may be worth considering different weighting for both qualities, and we will consider it in the future date. Fig. 2. Seesaw control Application: FLS TABLE I T HE RULES OF FLAT FLS x3 Fig. 3. Seesaw control Application: Parallel L-HFS x1 x4 nb ze x2 TABLE III T HE RULES OF PARALLEL L-HFS: FLS1 pb x2 x2 nb ze pb nb ze pb nb ze pb nb nb ze pb nb nb nm nb nm ns nm ns ze nb nm ns nm ns ze ns ze ps nm ns ze ns ze ps ze ps pm ze nb ze pb nb nm ns nm ns ze ns ze ps nm ns ze ns ze ps ze ps pm ns ze ps ze ps pm ps pm pb pb nb ze pb nm ns ze ns ze ps ze ps pm ns ze ps ze ps pm ps pm pb ze ps pm ps pm pb pm pb pb A. Produce a Parallel L-HFS and a Serial L-HFS for the Seesaw Control First, this experiment begins by producing two types of HFS, namely a parallel version and a serial version. In this case, the L-HFS algorithm is used to decompose the FLS from the seesaw system to reproduce two types of HFS models, also known as Parallel L-HFS and Serial L-HFS. The topologies of these HFSs can be seen in Figs. 3 and 4, respectively. The complete rules for these HFSs are shown in Tables III–VIII. A summary of both HFSs is presented in Table II. As can be seen in Table II, the decomposition process from FLS to HFSs (Parallel L-HFS and Serial L-HFS) has effectively reduced the number of rules by almost 50% from the original FLS. Intuitively and perhaps naively, one may say that both the Parallel L-HFS and Serial L-HFS have reduced approximately 50% of the complexity from the original FLS, x1 x2 nb ze pb TABLE IV T HE RULES OF PARALLEL L-HFS: FLS2 x3 x4 nb ze pb A B C B C D C D E nb ze pb nb ze pb F G H G H I H I J if only considering the total number of rules as an indicator to measure HFS complexity. However, the uncertainty of the structure of HFSs, namely due to multiple subsystems, layers, and different topologies, may also affect the complexity of HFSs. Therefore, in the following step, the proposed measure, CHF S , is used to measure complexity for both the Parallel L-HFS and Serial L-HFS. B. Measuring Complexity of the Parallel L-HFS and Serial L-HFS Secondly, the proposed measure, CHFS is utilised to measure the complexity in the Parallel L-HFS and the Serial L-HFS. The measurement is described in detail in the following subsections. Step 1: Calculating the rule-based complexity of the Parallel and Serial L-HFS: To calculate the rule-based complexity, equation (7) is applied to measure the complexity in Parallel L-HFS and Serial L-HFS. Table IX shows a summary of the two systems’ rule-based complexity. TABLE II D ESCRIPTION OF THE FLS, PARALLEL L-HFS AND S ERIAL L-HFS FOR S EESAW C ONTROL A PPLICATIONS Descriptions Number of inputs Number of layers Number of subsystems Number of rules in FLS1 Number of rules in FLS2 Number of rules in FLS3 Total number of rules FLS Parallel L-HFS Serial L-HFS 4 1 1 81 81 4 2 3 9 9 25 43 4 3 3 9 15 21 45 Fig. 4. Seesaw control Application: Serial L-HFS TABLE V T HE RULES OF PARALLEL L-HFS: FLS3 TABLE X T HE S TRUCTURAL C OMPLEXITY BETWEEN THE FLS, PARALLEL L-HFS AND THE S ERIAL L-HFS OF S EESAW C ONTROL A PPLICATION TABLE VI T HE RULES OF S ERIAL L-HFS: FLS1 y1 y2 A F G H I J B nb nb nm ns ze C nb nm ns ze ps nm ns ze ps pm D E ns ze ps pm pb ze ps pm pb pb nb ze pb nb ze pb nb ze pb A B C B C D C D E TABLE VIII T HE RULES OF S ERIAL L-HFS: FLS3 y1 A B C D E F G H G H I H I J I J K J K L FLS Parallel L-HFS Serial L-HFS l n s CS C˜S 1 2 3 4 4 4 1 3 3 -2 4 5 0.250 0.844 0.874 TABLE XI S UMMARY OF OVERALL MEASUREMENT COMPLEXITY FOR FLS, PARALLEL L-HFS AND S ERIAL L-HFS WITH VARIOUS ALTERNATIVES TABLE VII T HE RULES OF S ERIAL L-HFS: FLS2 x3 Structural Complexity Seesaw Systems x1 x2 Alternative combination strategies y2 x4 nb ze pb COMBINATION STRATEGIES F G H I J K L nb nb nm nb nm ns nm ns ze ns ze ps ze ps pm ps pm pb pm pb pb As seen in Table IX, the computed rule-based complexity, CRB , produces a higher complexity value of 0.768 for the Parallel L-HFS, compared to 0.756 for the Serial L-HFS. This suggests that the Parallel L-HFS may be more complex than the Serial L-HFS in terms of rule-based complexity. Step 2: Calculating the structural complexity of the Parallel and Serial L-HFS: For the structural complexity, Equation (8) is used to measure the structural complexity, CS , in the two HFSs. Table X presents an overview of the summary of this structural complexity, CS , measured for the Parallel L-HFS and Serial L-HFS. Equation (9) is then used to normalise the computed structural complexity of Parallel L-HFS and Serial L-HFS to be within the range [0,1], as shown in the last column of Table X. Table X shows the pattern of the computed structural complexity that produces a higher value for the Serial L-HFS (CS = 5) in comparison to the Parallel L-HFS (CS = 4). Aggregation - min Aggregation - max Aggregation - mean T-conorm - probabilistic sum T-conorm - bounded sum T-conorm - einstein sum FLS Parallel L-HFS Serial L-HFS CRB C˜S CHF S CRB C˜S CHF S CRB C˜S CHF S 0.978 0.978 0.978 0.978 0.978 0.978 0.250 0.250 0.250 0.250 0.250 0.250 0.250 0.978 0.614 0.984 1.000 0.987 0.768 0.768 0.768 0.768 0.768 0.768 0.844 0.844 0.844 0.844 0.844 0.844 0.768 0.844 0.806 0.964 1.000 0.978 0.756 0.756 0.756 0.756 0.756 0.756 0.874 0.874 0.874 0.874 0.874 0.874 0.756 0.874 0.815 0.969 1.000 0.981 This suggests that the Serial L-HFS is more complex than the Parallel L-HFS and also the FLS in terms of structural complexity. Step 3: Overall measurement of the complexity of the Parallel and Serial L-HFS: The structural complexity has been normalised to be the same scale as the rule-based complexity in the previous step as shown in Table X. Therefore, both rule-based complexity and structural complexity for Parallel L-HFS and Serial L-HFS are comparable. Next, to obtain the overall complexity of Parallel L-HFS and Serial L-HFS, the proposed CHFS as shown in (10) is used to combine the rulebased complexity and structural complexity by using several combination options, namely aggregation and t-conorm operators. Table XI provides a summary of the overall measurement complexity in FLS, Parallel L-HFS and Serial L-HFS using various combination strategies. In addition, the results of the overall measurement complexity in FLS, Parallel L-HFS and Serial L-HFS can also be viewed graphically in Fig. 5. V. D ISCUSSION TABLE IX T HE RULE - BASED C OMPLEXITY BETWEEN THE FLS, PARALLEL L-HFS AND THE S ERIAL L-HFS OF S EESAW C ONTROL A PPLICATION Rule-based Complexity Seesaw Systems FLS NC NP Comp C (1- Comp) 7 324 0.022 0.978 Parallel L-HFS - FLS1 - FLS2 - FLS3 CRB 5 5 7 18 18 50 0.278 0.278 0.140 0.722 0.722 0.860 0.768 Serial L-HFS - FLS1 - FLS2 - FLS3 CRB 5 7 7 18 30 42 0.278 0.233 0.167 0.722 0.767 0.833 0.756 This paper has presented a new approach to measuring complexity in HFSs, namely the CHFS approach, which focusses on combining rule-based complexity and structural complexity. An experiment was conducted and observed the proposed measure using a real-world example; in this case, the Seesaw systems problem. Note that we are focusing on control example in this paper because it is a convenient example of application and available in the literature. However, in general, we are interested in modelling example context. First, an FLS was decomposed into two HFS models, namely a Parallel L-HFS and a Serial L-HFS. By doing so, both Parallel L-HFS and Serial L-HFS produced approximately 50% fewer rules than the original FLS of the Seesaw System. The rule-based complexity reveals that the FLS is more complex than the Parallel L-HFS and Serial L-HFS, as shown in Table IX. Intuitively, it may be assumed that this Fig. 5. Summary of the overall complexity (CHF S ) for FLS, Parallel L-HFS and Serial L-HFS, using different aggregation functions is because the reduction of the number of rules reduces the complexity of both models. However, it is not appropriate to consider only the number of rules as an indicator when measuring complexity in HFSs, without taking into account its structure, namely the existence of multiple subsystems, layers and different topologies. Furthermore, measuring the complexity of both the Parallel L-HFS and the Serial L-HFS was implemented using CHFS , consisting of three main steps. For the first step, the rule-based complexity for the FLS, Parallel L-HFS and Serial L-HFS was measured using the proposed CRB as shown in (7). In summary, the results show that FLS is more complex than both HFSs in term or rule-based complexity, i.e. the number of rules in FLS is higher than both HFSs. Moreover, the findings also reveal that the Parallel L-HFS produced a higher value of CRB than the Serial L-HFS. This would indicate that the Parallel L-HFS was more complex than the Serial L-HFS in terms of rule-based complexity. Interestingly, the total number of rules in the Parallel L-HFS was slightly less than in the Serial LHFS. This result of CRB gives insight into the complexity of HFSs, nevertheless, it is not appropriate to consider only the number of rules when measuring complexity in HFSs. Next, for the second step, the structural complexity for the FLS, Parallel L-HFS and Serial L-HFS was measured using the proposed CS as shown in (8). In contrast, for this case, the results showed that the Serial L-HFS produces a higher value of CS than the Parallel L-HFS, which would indicate that the Serial L-HFS is more complex than the Parallel LHFS. This finding was expected and suggests that in HFSs, the complexity increases exponentially with the number of HFS layers in terms of the structural complexity. Further, the results confirm the expectation that the structural complexity of FLS is less complex than both HFSs as its structure is simpler than the HFSs, possessing just a single layer and subsystem. For the third step, the overall complexity of the plain FLS, the Parallel L-HFS and the Serial L-HFS was calculated using the proposed CHF S as shown in (10). This aimed to combine the results of the rule-based complexity and structural complexity calculations from Steps 1 and 2 respectively, using several combination alternatives. The results from this step indicate that the combination from operators max, mean, probabilistic sum and Einstein sum presented a similar pattern, namely that the overall complexity value of CHF S for the Serial L-HFS is higher than the Parallel L-HFS, indicating that the Serial L-HFS is more complex than the Parallel LHFS. However, the operator min showed a different pattern, namely that the overall complexity value of CHF S for the Parallel L-HFS is higher than that of the Serial L-HFS, indicating that the Parallel L-HFS is more complex than the Serial L-HFS. However, the operator bounded sum produced an equal complexity value for both Parallel L-HFS and Serial L-HFS, which could indicate that they are equally complex. Overall, the results generated for the proposed measure using most operators follow intuition in the sense that one expects the Serial L-HFS to be more complex than the Parallel LHFS. For the case of the plain FLS, both aggregation and t-conorm operators produced a variety of complexity results. For instance, the max, probabilistic sum and Einstein sum operators all show the FLS is more complex than both HFSs. In contrast, the min and mean operators indicated that the FLS is less complex than both HFSs. There are several possible explanations for this result: (i) the overall complexity of HFSs is more affected by the structural complexity than rule-based complexity; (ii) complexity in HFSs intuitively increases exponentially with the number of layers; and (iii) rule-based complexity in HFSs does not entirely depend on the total number of rules, but also depends on the aggregation process of individual rule-based complexities in each HFS subsystem. VI. C ONCLUSION In conclusion, we have contributed to the field a new approach to measuring complexity in HFSs, known as CHFS . The new approach allows the combination of rule-based complexity and structural complexity as a way to obtain a more comprehensive measurement of complexity, particularly in HFSs. Although the combination strategies explored in this paper are not exhaustive and some different approaches may be appropriate, based on the current evidence, the proposed measure (CHFS ) shows promise for measuring the complexity in HFSs and it behaves in an intuitive manner. Also, the proposed measure (CHFS ) appears to be a better approach for measuring complexity in HFSs because it considers both rulebased complexity and structural complexity. In future research, we will focus on exploring other aspects of complexity, including the semantic complexity of fuzzy sets, intermediate output and the logical complexity of the rules. For other future work, we will focus on investigating the combination of the proposed measure with interpretability measures that take into account the semantics of fuzzy sets and fuzzy rules, and also the different weighing aggregation. In doing so, we would hope to gain further insight into a deeper understanding of the overall interpretability of HFSs. R EFERENCES [1] A. Bárdossy, A. Blinowska, W. Kuzmicz, J. Ollitrault, M. Lewandowski, A. Przybylski, and Z. Jaworski, “Fuzzy logic-based diagnostic algorithm for implantable cardioverter defibrillators.” Artif. Intell. Med, vol. 60, no. 2, pp. 113–21, feb 2014. [2] A. Gad and M. Farooq, “Application of Fuzzy Logic in Engineering problems,” in Proc. IECON, vol. 3. IEEE, 2001, pp. 2044–2049. [3] J. Harris, Fuzzy logic applications in engineering science. Springer Science & Business Media, 2005, vol. 29. [4] L. A. Zadeh, “Outline of a New Approach to the Analysis of Complex Systems and Decision Processes,” IEEE Trans. Syst., Man, and Cybern., vol. SMC-3, no. 1, pp. 28–44, 1973. [5] A. Gegov, D. Sanders, and B. Vatchova, “Complexity management methodology for fuzzy systems with feedforward rule bases,” Int. J. of Knowledge-based and Intell. Engineering Syst., vol. 19, no. 2, pp. 83–95, jan 2015. [6] S.-M. Zhou, J. Garibaldi, R. John, and F. Chiclana, “On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection,” IEEE Trans. Fuzzy Syst., vol. 17, no. 3, pp. 654–667, jun 2009. [7] Y. Jin, “Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement,” IEEE Trans. Fuzzy Syst., vol. 8, no. 2, pp. 212–221, apr 2000. [8] H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, “Selecting fuzzy if-then rules for classification problems using genetic algorithms,” IEEE Trans. Fuzzy Syst., vol. 3, no. 3, pp. 260–270, 1995. [9] O. Cordón, M. J. Del Jesus, F. Herrera, L. Magdalena, and P. Villar, “A Multiobjective Genetic Learning Process for joint Feature Selection and Granularity and Contexts Learning in Fuzzy Rule-Based Classification Systems,” in Interpretability issues in fuzzy modeling. Springer Berlin Heidelberg, 2003, pp. 79–99. [10] L. Koczy and K. Hirota, “Size reduction by interpolation in fuzzy rule bases,” IEEE Trans. Syst. Man Cybern. - Part B, vol. 27, no. 1, pp. 14–25, 1997. [11] Q. Liang and J. M. Mendel, “Designing interval type-2 fuzzy logic systems using an SVD-QR method: Rule reduction,” Int. J. Intell. Syst., vol. 15, no. 10, pp. 939–957, oct 2000. [12] Y. Jin, “Decentralized adaptive fuzzy control of robot manipulators,” IEEE Trans. Syst. Man Cybern. - Part B, vol. 28, no. 1, pp. 47–57, 1998. [13] S.-M. Zhou and J. Gan, “Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling,” Fuzzy Sets Syst., vol. 159, no. 23, pp. 3091–3131, dec 2008. [14] G. V. S. Raju, J. Zhou, and R. Kisner, “Hierarchical fuzzy control,” Int. J. Contr., vol. 54, no. 5, pp. 1201–1216, nov 1991. [15] T. R. Razak, J. M. Garibaldi, C. Wagner, A. Pourabdollah, and D. Soria, “Interpretability indices for hierarchical fuzzy systems,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE). IEEE, jul 2017, pp. 1–6. [16] ——, “Towards a Framework for Capturing Interpretability of Hierarchical Fuzzy Systems - A Participatory Design Approach,” IEEE Transactions on Fuzzy Systems, pp. 1–1, 2020. [17] ——, “Interpretability and Complexity of Design in the Creation of Fuzzy Logic Systems — A User Study,” in 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2018, pp. 420–426. [18] H. Ishibuchi and Y. Nojima, “Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning,” Int. J. Approx. Reas., vol. 44, no. 1, pp. 4–31, jan 2007. [19] M. Delgado, F. Von Zuben, and F. Gomide, “Multi-objective decision making: towards improvement of accuracy, interpretability and design autonomy in hierarchical genetic fuzzy systems,” in Proc. Int. Conf. FUZZ-IEEE, vol. 2. IEEE, 2002, pp. 1222–1227. [20] X. Zhu, J. Li, D. Wu, H. Wang, and C. Liang, “Balancing accuracy, complexity and interpretability in consumer credit decision making: A C-TOPSIS classification approach,” Knowledge-Based Syst., vol. 52, pp. 258–267, nov 2013. [21] T. R. Razak, J. M. Garibaldi, and C. Wagner, “A Measure of Structural Complexity of Hierarchical Fuzzy Systems Adapted from Software Engineering,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), 2019, pp. 1–7. [22] J. Casillas, O. Cordón, F. Herrera, and L. Magdalena, “Interpretability Improvements to Find the Balance Interpretability-Accuracy in Fuzzy Modeling: An Overview,” in Interpretability issues in fuzzy modeling. Springer Berlin Heidelberg, 2003, pp. 3–22. [23] M. J. Gacto, R. Alcalá, and F. Herrera, “Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures,” Inf. Sci., vol. 181, no. 20, pp. 4340–4360, 2011. [24] C. Mencar, C. Castiello, R. Cannone, and A. M. Fanelli, “Interpretability assessment of fuzzy knowledge bases: A cointension based approach,” Int. J. Approx. Reas., vol. 52, pp. 501–518, 2011. [25] O. Cordón, “A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems,” Int. J. Approx. Reas., vol. 52, no. 6, pp. 894–913, sep 2011. [26] M. Galende-Hernández, G. I. Sainz-Palmero, and M. J. Fuente-Aparicio, “Complexity reduction and interpretability improvement for fuzzy rule systems based on simple interpretability measures and indices by biobjective evolutionary rule selection,” Soft Comput., vol. 16, no. 3, pp. 451–470, mar 2012. [27] H. Ishibuchi, Y. Kaisho, and Y. Nojima, “Complexity, interpretability and explanation capability of fuzzy rule-based classifiers,” in Proc. Int. Conf. Fuzzy Systems. IEEE, aug 2009, pp. 1730–1735. [28] D. Nauck, “Measuring interpretability in rule-based classification systems,” in Proc. 12th IEEE Int. Conf. Fuzzy Syst. IEEE, 2003, pp. 196–201. [29] J. M. Alonso, L. Magdalena, and G. González-Rodrı́guez, “Looking for a good fuzzy system interpretability index: An experimental approach,” Int. J. Approx. Reas., vol. 51, no. 1, pp. 115–134, dec 2009. [30] E. Mones, L. Vicsek, and T. Vicsek, “Hierarchy Measure for Complex Networks,” PLoS ONE, vol. 7, no. 3, p. e33799, mar 2012. [31] W. G. Bail and M. V. Zelkowitz, “Program complexity using Hierarchical Abstract Computers,” Computer Languages, vol. 13, no. 3-4, pp. 109–123, jan 1988. [32] A. Gegov, Fuzzy Networks for Complex Systems, ser. Studies in Fuzziness and Soft Computing. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, vol. 259. [33] T. McCabe, “A Complexity Measure,” IEEE Trans. on Software Engineering, vol. SE-2, no. 4, pp. 308–320, dec 1976. [34] G. Raju and J. Zhou, “Adaptive Hierarchical Fuzzy Controller,” IEEE Trans. Syst., Man, and Cybern., vol. 23, no. 4, pp. 973–980, 1993. [35] L. Magdalena, “Semantic interpretability in hierarchical fuzzy systems: Creating semantically decouplable hierarchies,” Inf. Sci., vol. 496, pp. 109–123, sep 2019. [36] M.-L. Lee, H.-Y. Chung, and F.-M. Yu, “Modeling of hierarchical fuzzy systems,” Fuzzy Sets Syst., vol. 138, no. 2, pp. 343–361, sep 2003. [37] H. Hagras, “A Hierarchical Type-2 Fuzzy Logic Control Architecture for Autonomous Mobile Robots,” IEEE Trans. Fuzzy Syst., vol. 12, no. 4, pp. 524–539, aug 2004. [38] J. M. Alonso, S. Guillaume, and L. Magdalena, “A hierarchical fuzzy system for assessing interpretability of linguistic knowledge bases in classification problems,” in Proc. PIMU, 2006, pp. 348–355.