Hierarchical Fuzzy Systems: Interpretability and Complexity

Indonesian Journal of Electrical Engineering and Informatics (IJEEI) Vol. 9, No. 2, June 2021, pp. 478~489 ISSN: 2089-3272, DOI: 10.52549/ijeei.v9i2.2813  478 Hierarchical Fuzzy Systems: Interpretability and Complexity Tajul Rosli Razak1, Shukor Sanim Mohd Fauzi2, Ray Adderley JM Gining3, Mohammad Hafiz Ismail4, Ruhaila Maskat5 1,2,3,4Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Cawangan Perlis, Kampus Arau, Perlis, Malaysia. 5Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Kampus Selangor, Selangor, Malaysia. Article Info ABSTRACT Article history: Rule explosion related to an increase in the number of input variables has been recognised as a key issue in fuzzy logic systems (FLSs), and hierarchical fuzzy systems (HFSs) have been proposed as a viable solution. The typical FLS subsystem system is transformed into a low-dimensional FLS subsystem network in HFS. Furthermore, because the number of input variables in each subsystem is reduced, the rules in HFS generally contain antecedents with fewer variables than the rules in regular FLS with similar functions. As a result, HFSs can reduce rule explosion, reducing model complexity and improving model interpretability. Nonetheless, the concerns concerning the issue of “Does reducing the complexity of HFSs with various subsystems, layers, and diverse topologies actually increase their interpretability?” remain unclear. In this study, we compare two HFS topologies: parallel and serial, concentrating on interpretability and complexity. For both topologies, a full measurement of the interpretability and complexity with various configurations is presented. The goal of this comparative study is to see if there is a correlation between HFS interpretability and complexity. Received Oct 21, 2020 Revised Jun 1, 2021 Accepted Jun 8, 2021 Keywords: Hierarchical fuzzy systems Complexity Interpretability Serial Parallel Copyright © 2019 Institute of Advanced Engineering and Science. All rights reserved. Corresponding Author: Tajul Rosli Razak, Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Cawangan Perlis, Kampus Arau, 02600 Arau, Perlis, Malaysia. Email: [email protected] 1. INTRODUCTION The In a range of applications, interpretability is recognised as one of the most desirable features of fuzzy systems [1], especially in those with considerable human involvement, where it is a must [2]. A fuzzy set is used to represent the domain values of a variable in the fuzzy system. Linguistic words, like human cognition, may be employed for this purpose. This property distinguishes the fuzzy system from other modelling systems in that, despite handling some intuitive requirements related to the collection of fuzzy sets, they may be simply read, making them understandable even to non-experts [3]. Fuzzy logic systems (FLSs) have been utilised successfully in a variety of disciplines, including research, industry, manufacturing, and business. They're also used in the medical industry for decision-making, particularly when dealing with ambiguity and inaccurate data [4]–[7]. The curse of dimensionality, on the other hand, is a fundamental drawback of traditional fuzzy systems: the number of needed rules rises exponentially with the number of input variables [8]. This problem, also known as rule explosion, has the potential to reduce the openness and interpretability of FLSs [9]. Raju et al. [10] play a critical role in the development of hierarchical fuzzy systems (HFSs) to address this issue. Rather than generating a single high-dimensional rule base for an FLS, this HFS technique will divide the input variables into a set of low-dimensional fuzzy subsystems. Each fuzzy low-dimensional subsystem is connected in a hierarchical manner. The rule explosion problem may be avoided, and the model's interpretability can be enhanced, by using this strategy. Journal homepage: http://section.iaesonline.com/index.php/IJEEI/index IJEEI ISSN: 2089-3272  479 Since interpretability is inherently subjective, complexity is frequently utilised in FLS as an indirect assessment of interpretability. Indeed, some researchers claim that reducing the complexity of a fuzzy system might improve the system's interpretability [11]–[13]. However, it is unclear if this holds for HFSs as well. This is due to the HFSs' structure, which includes many subsystems, layers, topologies (parallel and serial HFS), and subsystem interactions, as well as each subsystem's regulations. The number of rules, variables, and fuzzy terms, especially in FLSs, are always indicators of complexity. However, no one knows how to assess complexity in a hierarchical system, which includes various subsystems, levels, and topologies. In addition, the relationship between HFS interpretability and complexity is yet unknown and must be researched. In this paper, we propose a new method to systematically compare the parallel and serial topologies of HFSs, focusing on interpretability and complexity. This method investigates the correlation between the interpretability and complexity of HFSs. The method consists of six keys steps: (i) generate parallel and serial using an L-HFS algorithm; (ii) generate synthetic input data; (iii) perform the input-output mapping data for parallel and serial HFS; (iv) examine the equivalence between parallel and serial HFS using the KolmogorovSmirnov test; (v) measure the interpretability and complexity of parallel and serial HFSs; and (vi) compare and analyse the results. intact. 2. BACKGROUND 2.1. Hierarchical Fuzzy Systems HFSs are identified by grouping input variables into subsystems, which are low-dimensional FLS subsystems [10], [14]. Furthermore, as illustrated in Fig. 2, HFS may be represented as a cascade structure in which each layer's output is recognised as input to subsequent levels. HFS may also be thought of as a decomposition of FLS' function [15]. From a functional standpoint, FLS and HFS, as shown in Figs. 1 and 2, respectively, can be done as follows: 𝑦 = 𝐹(𝑥1 , 𝑥2 , 𝑥3 ) ≡ 𝑦 = 𝑓2 (𝑥3 , 𝑓1 (𝑥2 , 𝑥1 )) A system that moves from one layer to two layers has fewer rules than one-in-one layers, as shown in Figs. 1 and 2, respectively. The most extreme rule reduction is if the HFS structure has two input variables for each low-dimensional FLS and has (𝑛 − 1) layers [10]. Suppose we define 𝑚 fuzzy sets for each input variable, including an intermediate output variable 𝑦1 , … , 𝑦𝑛−2 ,. In that case, the total of rules (𝑅) is a linear function [16] of the total of the input variables 𝑛 and can be represented as: 𝑅 = (𝑛 − 1)𝑚2 (1) Figure 1. Fuzzy Logic Systems Figure 2. Hierarchical Fuzzy Systems Despite its success in dealing with rule explosion, the HFS has some concerns with its intermediate output that must be solved before HFSs may be designed. The output of FLs in the previous layer, known as intermediate output, is utilised as the input linguistic variable of the following layer in the traditional HFS. This Hierarchical Fuzzy Systems: Interpretability and Complexity (Tajul Rosli Razak et al) 480  ISSN: 2089-3272 intermediate output, on the other hand, has no physical meaning. As a result, when the output is used as an input variable for the following layer, the fuzzy rule involved loses its physical meaning and becomes more complex to design [17]. Furthermore, as HFS expands, this issue will become increasingly critical. Several researchers [18]–[24] have proposed novel techniques to creating HFSs in order to address this problem. Among these approaches, only limpid-hierarchical fuzzy systems will be discussed in detail in the following section. 2.2. Limpid-Hierarchical Fuzzy Systems (L-HFS) Lee et al. suggested a new approach for dealing with the problem of an intermediate output in the middle layer that has no physical meaning [19]. To obtain the HFS rule base, they implemented a new mapping rule base called L-HFS. To illustrate this method, an HFS, as in Fig. 2, is used that consists of three input variables: 𝑥1 , 𝑥2 and 𝑥3 . FLSA and FLSB are subsystems for this HFS at layer 1 and layer 2, respectively. By defining each variable using three linguistic terms, where N is negative, Z is zero and P is positive, the complete rules for an FLS consist of 27 (3 × 3 × 3 = 27) rules. From these complete rules, Algorithm 1 (as can be seen in [19]) is then applied to the mapping rule for subsystems FLSA and FLSB of the HFS. 2.3. Interpretability In recent years, researchers have started to pay attention to interpretable fuzzy models. However, because of its contextual existence and various contributing influences, the selection of acceptable interpretability measures remains a topic of controversy. Significant research on the suggested interpretability index for FLS has been performed [25]–[30] of the proposed interpretability index for FLS. Among them, only the Fuzzy index and the H framework will be discussed in detail in the following sections. In [31], Razak et al. introduced the H (for HFS interpretability) framework, which incorporates aggregate interpretability at each subsystem into a single overall metric of HFS interpretability. It can be described in the following way: 𝑞 𝑠𝑗 𝑗=1 𝑘=1 𝐻mean = ∑ (𝑙𝑗 ∑ 𝐸𝑗𝑘 /𝑠𝑗 ) (2) where 𝐸𝑗𝑘 is the underlying (standard) FLS index associated with the subsystem 𝑘 at layer 𝑗, for example, the Fuzzy (F) index, 𝑙𝑗 is the weight connected with layer 𝑗 of the HFS, 𝑠𝑗 is the number of subsystems located in layer 𝑗, 𝑠 is the total number of subsystems and 𝑞 is the number of layers of the HFS. 2.4. Complexity In general, and particularly in FLSs, complexity is defined by several rules, variables, and fuzzy terms. An FLS with less rules, for example, is easier to customise and uses less memory and execution time than one with more rules. As a result, the majority of studies have used the index, or the number of rules, to determine the complexity of FLSs [11], [32]–[34]. Razak et al. suggested a method for evaluating the complexity of HFSs that takes into account the complexity of its structure, which requires several subsystems, layers, and a dynamic topology [35], [36]. Also, the approach seems to be better has its combined structure complexity and rule-based complexity. It can be computed as follows: 𝐶𝐻𝐹𝑆 = 𝐶𝑅𝐵 ⊕ 𝐶𝑆 (3) where 𝐶𝑅𝐵 is rule-based complexity, 𝐶𝑆 is structural complexity and ⊕ indicates the generic aggregation operator such as min, max and mean. In this paper, we will use mean in (3) to measure the complexity of HFSs. Further information for this (3) can be seen in [36]. 3. METHODOLOGY We propose a method to compare parallel and serial HFSs focusing on interpretability and complexity. Figure 3 shows the steps involved in developing this method: IJEEI, Vol.9, No. 2, June 2021: 478 - 489 IJEEI ISSN: 2089-3272  481 Figure 3. Stepwise process for the proposed methodnovelty. 3.1. Step 1: Produce parallel (P-LHFS) and serial (S-LHFS) HFSs using L-HFS algorithm The method begins by producing two types of HFSs, i.e., parallel and serial, using an L-HFS algorithm. These HFS models are named P-LHFS and S-LHFS, respectively. 3.2. Step 2: Generate synthetic input data Then, the input data of 500 samples is synthetically produced. This is achieved by using a simple random function (Rand()) as in Microsoft Excel. 3.3. Step 3: Perform the input-output mapping data for P-LHFS and S-LHFS Next, the input-output mapping is conducted for P-LHFS and S-LHFS by using the synthetic input data as in Step 2. The result of this step is stored for both P-LHFS and S-LHFS and will be used in the next step. 3.4. Step 4: Examine the equivalence between P-HFS and S-HFS using the Kolmogorov-Smirnov test For this step, a statistical test was conducted using the Kolmogorov-Smirnov (KS) test [37] to compare the equivalence of P-LHFS and S-LHFS. The K-S test statistic measures the maximum distance (D) between the empirical cumulative distribution function (ECDF) P-LHFS and S-LHFS, measured in a vertical direction. It is computed as: 𝐷 = sup (𝑥) − 𝐹𝑆−𝐿𝐻𝐹𝑆 (𝑥)| |𝐹 𝑥 𝑃−𝐿𝐻𝐹𝑆 (4) Hierarchical Fuzzy Systems: Interpretability and Complexity (Tajul Rosli Razak et al) 482  ISSN: 2089-3272 where, 𝐹𝑃−𝐿𝐻𝐹𝑆 (𝑥) indicates the cumulative distribution of output mapping of P-LHFS (as in Step 3), 𝐹𝑆−𝐿𝐻𝐹𝑆 (𝑥) indicates the cumulative distribution of output mapping of S-LHFS (as in Step 3). The D statistic is then compared with the critical value (𝐷𝛼 ) to find the equivalence between P-LHFS and S-LHFS. The D is greater than (𝐷𝛼 ), indicating that both data are distributed differently. Meanwhile, D is less than (𝐷𝛼 ), indicating that both data have the same distribution. The critical values can be found at the K-S test critical values in [37] and computed as follows: 𝐷𝛼 = 𝑐(𝛼) (5) √𝑛 where n is a sample size, 𝛼 is level of significance that will indicate the value of 𝑐(𝛼) and both values can be referred at K-S Test Critical values in [37]. In this case, we use the value of 𝛼 = 0.05 and therefore, the value of 𝑐(𝛼) = 1.36. 3.5. Step 5: Measuring the interpretability and complexity of P-LHFS and S-LHFS In this step, the interpretability and complexity for each P-LHFS and S-LHFS are measured. For the interpretability, (2) is used to measure the interpretability for both P-LHFS and S-LHFS. Then the results are compared. Also, in this step, to measure the complexity of P-LHFS and S-LHFS, we use (3) to calculate the overall complexity of HFSs that include rule-based and structural complexity. 3.6. Step 6: Comparing the relationship interpretability and complexity of P-LHFS and S-LHFS Finally, the results from Step 5 are analysed and compared. This step can provide significant insight into the relationship between the interpretability and complexity of HFSs, respectively. 4. EXPERIMENT AND RESULTS Provide As a demonstration, we have used the example of a seesaw control application (as in [19]), which solved the problem of balancing a seesaw using an FLS. The involved parameters of the seesaw are (𝑥 = 𝑥1 ), (𝜃 = 𝑥2 ), (𝑟1 = 𝑥3 ), and (𝑟2 = 𝑥4 ), as shown in Fig. 4. The topology and complete rules set for this FLS can be seen in Figure 5 and Table 2, respectively. Figure 4. Seesaw Control Application. Adapted from [19]. The experiment aims to investigate the relationship between the interpretability and complexity of different HFSs, i.e., parallel and serial, using the proposed method. The method consists of six main steps, as shown in Figure 3. For step 1, we used the L-HFS algorithm to produce two types of HFSs (parallel and serial) for seesaw systems, termed as P-LHFS and S-LHFS. The topologies of these HFSs can be seen in Figs. 6 and 7, respectively. Also, the complete rules for these HFSs were shown in Tables 3 – 8. The summary of both HFSs of the seesaw system, including FLS, is presented in Table 1. For step 2, the input data of 500 samples for the seesaw system are synthetically produced using a simple random function (Rand()) in Microsoft Excel. In Step 3, we conducted the input-output mapping for P-LHFS and S-LHFS by using the synthetic input data from Step 2. For the remaining steps (Step 4 – Step 6), a detailed explanation will be provided in the following subsections. IJEEI, Vol.9, No. 2, June 2021: 478 - 489 IJEEI  ISSN: 2089-3272 483 Table 1. Description of the three Seesaw systems, namely FLS, P-LHS and S-LHFS. Note that NOR is the number of rules. Description Number of inputs Number of subsystems NOR in FLS1 NOR in FLS2 NOR in FLS3 Total NOR FLS 4 1 81 81 P-LHFS 4 3 9 9 25 43 S-LHFS 4 3 9 15 21 45 Table 2. The complete rules of FLS 𝑥3 nb ze pb 𝑥4 nb ze pb nb ze pb nb ze pb nb nb nb nm nb nm ns nm ns ze nb 𝑥2 ze nb nm ns nm ns ze ns ze ps pb nm ns ze ns ze ps ze ps pm nb nb nm ns nm ns ze ns ze ps 𝑥1 ze 𝑥2 ze nm ns ze ns ze ps ze ps pm pb ns ze ps ze ps pm ps pm pb nb nm ns ze ns ze ps ze ps pm pb 𝑥2 ze ns ze ps ze ps pm ps pm pb pb ze ps pm ps pm pb pm pb pb Figure 5. FLS topology Figure 6. Parallel HFS (P-LHFS) topology Figure 7. Serial HFS (S-LHFS) topology Table 3. The rules of subsystem FLS1 in P-LHFS Table 4. The rules of subsystem FLS2 in P-LHFS Hierarchical Fuzzy Systems: Interpretability and Complexity (Tajul Rosli Razak et al)  484 ISSN: 2089-3272 𝑥2 nb ze pb nb A B C ze B C D 𝑥1 pb C D E nb ze pb Table 5. The rules of subsystem FLS3 in P-LHFS F G H I J 𝑦2 A nb nb nm ns ze B nb nm ns ze ps 𝑦1 C nm ns ze ps pm D ns ze ps pm pb E ze ps pm pb pb Table 7. The rules of subsystem FLS2 in S-LHFS nb ze pb 𝑥3 A F G H B G H I C H I J 𝑦1 D I J K E J K L 𝑥4 nb F G H 𝑥3 ze G H I pb H I J Table 6. The rules of subsystem FLS1 in S-LHFS nb ze pb 𝑥2 nb A B C ze B C D 𝑥1 pb C D E Table 8. The rules of subsystem FLS3 in S-LHFS 𝑥4 nb ze pb F nb nb nm G nb nm ns H nm ns ze 𝑦2 I ns ze ps J ze ps pm K ps pm pb L pm pb pb 4.1. Step 4: Comparing the equivalence of P-LHFS and S-LHFS In this step, the input-output mapping data for both P-LHFS and S-LHFS of the seesaw system from Step 3, are statistically compared using the Kolmogorov-Smirnov (KS) test. The aim is to find the equivalence between both HFSs and with FLS. It is crucial to make sure that they are equivalent before we can proceed to the next steps. For example, if the result showed that they were not equivalent, there would be is no point in doing a detailed comparison and analysis in terms of interpretability and complexity because they would be two different systems. Table 9. The Kolmogorov-Smirnov Test between FLS and HFSs (P-LHFS and S-LHFS) of Seesaw Control Application. Input sample (n) 500 FLS vs P-LHFS 𝐷 𝑝-value 0.042 FLS vs S-LHFS 𝐷 𝑝-value 0.7699 0.040 0.8186 P-LHFS vs S-LHFS 𝐷 𝑝-value 0.048 0.6121 Critical values 𝑐(𝛼) 𝐷𝛼 = √𝑛 0.061 Table 9 presents the result of the K-S test that examined the equivalence between FLS and P-LHFS, FLS and S-LHFS, and, most importantly, between P-LHFS and S-LHFS of the seesaw system. From Table 9, the K-S test showed that three seesaw systems – namely, FLS, P-LHFS and S-LHFS were equivalent when tested with the 500 input samples. The K-S test statistic, which measured the maximum distance (𝐷) between three seesaw systems, produced less than (𝐷𝛼 ), indicating they have the same distribution. These results can also be viewed graphically in Figs. 8, 9 and 10, respectively. Figure 8. FLS vs P-LHFS IJEEI, Vol.9, No. 2, June 2021: 478 - 489 IJEEI ISSN: 2089-3272  485 Figure 9. FLS vs S-LHFS Figure 10. P-LHFS vs S-LHFS 4.2. Step 5: Measuring the interpretability and complexity of P-LHFS and S-LHFS This step discusses measuring the interpretability and complexity of these Seesaw systems, namely FLS, P-LHFS and S-LHFS. For the interpretability, we use the Hmean as in (2) to measure the interpretability of for FLS, P-LHFS and S-LHFS of the seesaw system. Table 10 shows a summary of the interpretability measured for FLS, P-LHFS and S-LHFS. Table 10. The interpretability of Seesaw systems measured by Hmean Seesaw systems FLS P-LHFS S-LHFS Interpretability Index (Hmean) 0.241 0.456 0.477 For the complexity, we use the C framework as in (6) to measure the complexity for FLS, P-LHFS and S-LHFS of the seesaw system. Table 11 shows a summary of the interpretability measured for FLS, PLHFS and S-LHFS. Generally, the results of the computed C index of S-LHFS higher than P-LHFS and FLS, indicating more complex. Table 11. The interpretability of Seesaw systems measured by C framework Seesaw systems FLS P-LHFS S-LHFS Complexity Index (CHFS) 0.250 0.768 0.756 4.3. Step 6: Comparing the relationship between interpretability and complexity of HFSs Hierarchical Fuzzy Systems: Interpretability and Complexity (Tajul Rosli Razak et al) 486  ISSN: 2089-3272 The Table 12 shows the result of comparison between the interpretability and complexity for P-LHFS and S-LHFS of the seesaw system. As can be seen from Table 12, the result indicates S-LHFS is having high interpretability and less complexity for the seesaw system. Meanwhile, the result showing that P-LHFS have less interpretability and high complexity for the seesaw system. Table 12. Comparing the interpretability and complexity of P-LHFS and S-LHFS Seesaw systems P-LHFS S-LHFS Interpretability Less High Complexity High Less 5. DISCUSSION We studied the newly proposed approach to systematically compared the Parallel and Serial of HFSs in term of interpretability and complexity. The approach was achieved using the Seesaw systems application example consisting of six key steps. However, the discussion only considers the on the Steps 4- 6. In Step 4, a statistical comparison of an equivalent between FLS, P-LHFS and S-LHFS was performed. This is accomplished by using a statistical test that is the Kolmogorov-Smirnov (KS) test. The result reveals that Both P-LHFS and S-LHFS are equivalent to an FLS as Kolmogorov-Smirnov Test showed 𝐷 < 𝐷𝛼 . The detailed result can be seen in Table 9. The results also found that S-LHFS are closer to FLS than PLHFS. Also, the result of the 𝐷 in SLHFS is less than P-LHFS. However, at this stage, we cannot make any assumptions on which HFSs (parallel or serial) is better as this step aims to see the equivalent between them. Meanwhile, in Step 5, a computation measurement study between FLS, P-LHFS and S-LHFS was made based on the interpretability and complexity. For the case of interpretability, the computed values of Hmean showed that the S-LHFS is higher than P-LHFS and FLS. This could indicate that S-LHFS is more interpretable than P-LHFS and FLS. Interestingly because the S-LHFS has 45 rules and the P-LHFS has 43 rules. Also, the S-LHFS has three layers, and a P-LHFS has two layers. It seems possible that these results are due to the S-LHFS has more antecedent than the P-LHFS, particularly at subsystem 2 (FLS2), which influence the overall interpretability of HFSs. Contrary, for the case of complexity, the computed values of CHFS showed that the P-LHFS is higher than S-LHFS and FLS. This could mean that P-LHFS is more complex than P-LHFS and FLS. However, the number of rules in P-LHFS is lower than S-LHFS, as shown in Table 1. This could also reveal that the number of rules does not only influence the complexity of HFS. Nonetheless, it may also influence by its structure, namely the number of layers, multiple subsystems, and varied topologies. Specifically, the possible explanation of these results includes; (i) intermediate output variables produced by using L-HFS algorithm are different; (ii) Five new mapping variables (A, B, C, D and E) are obtained for intermediate output 𝑦1 at subsystem (FLS1) in both P-LHFS and S-LHFS, as can be seen in Tables 4 and 7, respectively; and (iii) Five new mapping variables (F, G, H, I and J) are obtained for intermediate output 𝑦2 at subsystem (FLS2) in P-LHFS as shown in Table 5. Meanwhile, seven new mapping variables (F, G, H, I, J, K and L) are obtained for intermediate output 𝑦2 at subsystem (FLS2) in S-LHFS as shown in Table 8. In Step 6, Table 12 shows a comparison of interpretability and complexity of P-LHFS and S-LHFS for Seesaw systems example. In general, the result exposes that there is a trade-off relationship between interpretability and complexity of HFSs, as also discussed in [38]. That is, the less interpretability is, the higher complexity in the system. Also, the higher interpretability is less complexity in the system. In this comparison, the result tells that P-LHFS is less interpretability and high complexity. Meanwhile, the result exhibits that SLHFS is high interpretability and less complexity. Lastly, this systematic comparison is essential for assisting as a guideline to choose the best model or system between varied topologies in HFSs. Thus, this may provide an insight into building the interpretable hierarchical fuzzy system for the real-world example. 6. CONCLUSION In conclusion, we have proposed a method for systematically comparing the HFS topologies (parallel and serial). We would like to look at the relationship between interpretability and complexity in HFSs, which are made up of six main stages, using the Seesaw method as an example. Based on the available data, the SLHFS tends to be superior to the P-LHFS for the Seesaw method. This is since, in the case of the Seesaw system, the result shows that the S-LHFS is more interpretable and less complex than the P-LHFS. Note that this systematic comparison is essential for assisting as a guideline to choose the best model or system between varied topologies in HFSs. Thus, this may provide an insight into building the interpretable hierarchical fuzzy system for the real-world example. IJEEI, Vol.9, No. 2, June 2021: 478 - 489 IJEEI ISSN: 2089-3272  487 While the comparison approach is promising, there is still room for improvement. Therefore, for future work, we will improve the approach by incorporating the investigation with the other criteria such as accuracy, semantic interpretability. ACKNOWLEDGMENTS This work is supported by the YTR research grant, Universiti Teknologi MARA, Malaysia (Reference No.: 600-RMC/YTR/5/3 (018/2020)). REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] U. Bodenhofer and P. Bauer, “A Formal Model of Interpretability of Linguistic Variables,” in Interpretability issues in fuzzy modeling, Springer Berlin Heidelberg, 2003, pp. 524–545. J. M. Alonso and L. Magdalena, “Special issue on interpretable fuzzy systems,” Inf. Sci. (Ny)., vol. 181, pp. 4331–4339, 2011, doi: 10.1016/j.ins.2011.07.001. K. Balazs and L. T. Koczy, “New parameterizable search space narrowing technique for adjusting between accuracy and interpretability in fuzzy systems,” in Proc. IEEE 13th Int. Symp. Comput. Intell. and Inform. (CINTI), 2012, pp. 323–328, doi: 10.1109/CINTI.2012.6496783. A. Bárdossy et al., “Fuzzy logic-based diagnostic algorithm for implantable cardioverter defibrillators.,” Artif. Intell. Med, vol. 60, no. 2, pp. 113–21, Feb. 2014, doi: 10.1016/j.artmed.2013.12.004. A. Gad and M. Farooq, “Application of Fuzzy Logic in Engineering problems,” in Proc. IECON, 2001, vol. 3, pp. 2044–2049, doi: 10.1109/IECON.2001.975606. O. W. Samuel, M. O. Omisore, and B. A. Ojokoh, “A web based decision support system driven by fuzzy logic for the diagnosis of typhoid fever,” Expert Syst. Appl., vol. 40, no. 10, pp. 4164–4171, Aug. 2013, doi: 10.1016/j.eswa.2013.01.030. J. Harris, Fuzzy logic applications in engineering science, vol. 29. Springer Science & Business Media, 2005. S.-M. Zhou, J. M. Garibaldi, R. I. John, and F. Chiclana, “On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection,” IEEE Trans. Fuzzy Syst., vol. 17, no. 3, pp. 654–667, Jun. 2009, doi: 10.1109/TFUZZ.2008.928597. S. Jin and J. Peng, “Towards hierarchical fuzzy rule interpolation,” in Proc. IEEE 14th Int. Conf. Cognitive Inform. & Cognitive Comput. (ICCI*CC), 2015, pp. 267–274, doi: 10.1109/ICCI-CC.2015.7259396. G. V. S. Raju, J. Zhou, and R. A. Kisner, “Hierarchical fuzzy control,” Int. J. Contr., vol. 54, no. 5, pp. 1201– 1216, Nov. 1991, doi: 10.1080/00207179108934205. Y. Jin, “Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement,” IEEE Trans. Fuzzy Syst., vol. 8, no. 2, pp. 212–221, Apr. 2000, doi: 10.1109/91.842154. M. Setnes and R. Babuska, “Rule base reduction: some comments on the use of orthogonal transforms,” IEEE Trans. Syst., Man, Cybern. - Part C, vol. 31, no. 2, pp. 199–206, May 2001, doi: 10.1109/5326.941843. J. Yen and L. Wang, “Simplifying fuzzy rule-based models using orthogonal transformation methods,” IEEE Trans. Syst., Man, Cybern. - Part B, vol. 29, no. 1, pp. 13–24, 1999, doi: 10.1109/3477.740162. G. V. S. Raju and J. Zhou, “Adaptive Hierarchical Fuzzy Controller,” IEEE Trans. Syst., Man, Cybern., vol. 23, no. 4, pp. 973–980, 1993, doi: 10.1109/21.247882. L. Magdalena, F. Monasterio, and C. Rivero, “Hierarchical decomposition of fuzzy controllers based on metaknowledge,” Mathw. soft Comput., vol. 5, pp. 291–304, 1998. L.-X. Wang, “Universal approximation by hierarchical fuzzy systems,” Fuzzy Sets Syst., vol. 93, no. 2, pp. 223– 230, Jan. 1998, doi: 10.1016/S0165-0114(96)00197-2. T. R. Razak, C. Chen, J. Garibaldi, and C. Wagner, “Designing the Hierarchical Fuzzy Systems Via FuzzyR Toolbox,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), 2021, pp. 1–6. M. G. Joo and J. S. Lee, “Hierarchical fuzzy control scheme using structured Takagi-Sugeno type fuzzy inference,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), 1999, vol. 1, pp. 78–83, doi: 10.1109/FUZZY.1999.793210. M.-L. Lee, H.-Y. Chung, and F.-M. Yu, “Modeling of hierarchical fuzzy systems,” Fuzzy Sets Syst., vol. 138, no. 2, pp. 343–361, Sep. 2003, doi: 10.1016/S0165-0114(02)00517-1. N. K. Masmoudi, C. Rekik, M. Djemel, and N. Derbel, “Optimal Control for Discrete Large Scale Nonlinear Systems Using Hierarchical Fuzzy Systems,” in 2010 Second Int. Conf. on Mach. Learning and Comput., 2010, pp. 91–95, doi: 10.1109/ICMLC.2010.32. A. Ge, Y. Wang, Z. Liu, and P. Jiang, “Hierarchical fuzzy control of multivariable systems via semi-tensor product method,” in Proc. of the 33rd Chinese Control Conf., 2014, pp. 4551–4556, doi: 10.1109/ChiCC.2014.6895705. V. Cross and T. Sudkamp, “Patterns of fuzzy rule-based inference,” Int. J. Approx. Reas., vol. 11, no. 3, pp. 235–255, Oct. 1994, doi: 10.1016/0888-613X(94)90032-9. M. G. Joo and T. Sudkamp, “A Method of Converting a Fuzzy System to a Two-Layered Hierarchical Fuzzy System and Its Run-Time Efficiency,” IEEE Trans. Fuzzy Syst., vol. 17, no. 1, pp. 93–103, Feb. 2009, doi: 10.1109/TFUZZ.2008.2007570. R. Holve, “Rule generation for hierarchical fuzzy systems,” in Proc. Annu.Conf. North American Fuzzy Information Processing, 1997, pp. 444–449, doi: 10.1109/NAFIPS.1997.624082. Hierarchical Fuzzy Systems: Interpretability and Complexity (Tajul Rosli Razak et al) 488  [25] S.-M. Zhou and J. Q. Gan, “Low-level interpretability and high-level interpretability: a unified view of datadriven interpretable fuzzy system modelling,” Fuzzy Sets Syst., vol. 159, no. 23, pp. 3091–3131, Dec. 2008, doi: 10.1016/j.fss.2008.05.016. M. J. Gacto, R. Alcalá, and F. Herrera, “Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures,” Inf. Sci., vol. 181, no. 20, pp. 4340–4360, 2011, doi: 10.1016/j.ins.2011.02.021. D. D. Nauck, “Measuring interpretability in rule-based classification systems,” in Proc. 12th IEEE Int. Conf. Fuzzy Syst., 2003, pp. 196–201, doi: 10.1109/FUZZ.2003.1209361. J. M. Alonso, L. Magdalena, and G. González-Rodríguez, “Looking for a good fuzzy system interpretability index: An experimental approach,” Int. J. Approx. Reas., vol. 51, no. 1, pp. 115–134, Dec. 2009, doi: 10.1016/j.ijar.2009.09.004. C. Mencar, C. Castiello, R. Cannone, and A. M. Fanelli, “Interpretability assessment of fuzzy knowledge bases: A cointension based approach,” Int. J. Approx. Reas., vol. 52, pp. 501–518, 2011, doi: 10.1016/j.ijar.2010.11.007. T. R. Razak, J. M. Garibaldi, C. Wagner, A. Pourabdollah, and D. Soria, “Interpretability indices for hierarchical fuzzy systems,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), 2017, pp. 1–6, doi: 10.1109/FUZZIEEE.2017.8015616. T. R. Razak, J. M. Garibaldi, C. Wagner, A. Pourabdollah, and D. Soria, “Towards a Framework for Capturing Interpretability of Hierarchical Fuzzy Systems - A Participatory Design Approach,” IEEE Trans. Fuzzy Syst., vol. 29, no. 5, pp. 1160–1172, 2021, doi: 10.1109/TFUZZ.2020.2969901. H. Ishibuchi and Y. Nojima, “Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning,” Int. J. Approx. Reas., vol. 44, no. 1, pp. 4–31, Jan. 2007, doi: 10.1016/j.ijar.2006.01.004. M. R. Delgado, F. Von Zuben, and F. Gomide, “Multi-objective decision making: towards improvement of accuracy, interpretability and design autonomy in hierarchical genetic fuzzy systems,” in Proc. Int. Conf. FUZZIEEE, 2002, vol. 2, pp. 1222–1227, doi: 10.1109/FUZZ.2002.1006678. X. Zhu, J. Li, D. Wu, H. Wang, and C. Liang, “Balancing accuracy, complexity and interpretability in consumer credit decision making: A C-TOPSIS classification approach,” Knowledge-Based Syst., vol. 52, pp. 258–267, Nov. 2013, doi: 10.1016/J.KNOSYS.2013.08.004. T. R. Razak, J. M. Garibaldi, and C. Wagner, “A Measure of Structural Complexity of Hierarchical Fuzzy Systems Adapted from Software Engineering,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), 2019, pp. 1–7. T. R. Razak, J. M. Garibaldi, and C. Wagner, “An Improved Complexity Measure in Hierarchical Fuzzy Systems,” in Proc. IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), 2020, pp. 1–8. F. J. Massey and Jr., “The Kolmogorov-Smirnov Test for Goodness of Fit,” J. Am. Stat. Assoc., vol. 46, no. 253, pp. 68–78, Mar. 1951, doi: 10.2307/2280095. T. R. Razak, J. M. Garibaldi, C. Wagner, A. Pourabdollah, and D. Soria, “Interpretability and Complexity of Design in the Creation of Fuzzy Logic Systems — A User Study,” in 2018 IEEE Symposium Series on Computational Intelligence (SSCI), 2018, pp. 420–426, doi: 10.1109/SSCI.2018.8628924. [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] ISSN: 2089-3272 BIOGRAPHY OF AUTHORS Tajul Rosli Razak is currently working at Universiti Teknologi MARA, Perlis Branch, Malaysia as a member of Faculty of Computer and Mathematical Sciences. He received Bachelor of Information Technology (Hons) specializing in Artificial Intelligent from the Universiti Utara Malaysia, and Master of Science (Intelligent Systems) from the Universiti Utara Malaysia. He then obtained my PhD in Computer Science from the University of Nottingham, United Kingdom. While pursuing his PhD, He was a member of Intelligent Modelling & Analysis (IMA) and Laboratory for Uncertainty in Data and Decision Making (LUCID) research groups. He is currently a member of the IEEE-CIS Task Force on Explainable Fuzzy Systems. My research interests centre around the field of Intelligence Systems, Fuzzy Logic Systems and Hierarchical Fuzzy Systems, Decision Support Systems, and Interpretability. Associate Professor Ts. Dr Shukor Sanim Mohd Fauzi is a Deputy Rector (Research and Industrial Linkages) at UiTM Perlis Branch, and a faculty member of Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Perlis Branch, Malaysia. He received Master of Science (Computer Science - Real-Time Software Engineering) from the Centre for Advanced Software Engineering, Universiti Teknologi Malaysia. He then obtained his PhD in Software Engineering from the University of New South Wales (UNSW). IJEEI, Vol.9, No. 2, June 2021: 478 - 489 IJEEI ISSN: 2089-3272  489 R.A.J.M. Gining is now a lecturer at the Universiti Teknologi MARA, Perlis Branch, Malaysia's Faculty of Computer and Mathematical Sciences. He holds a Diploma of Computer Science and a Bachelor of Science (Hons) in Information Systems Engineering from Universiti Teknologi MARA, as well as a Master in Science in Cloud Computing from Newcastle University in the United Kingdom. System development, IoT, and machine learning are among his research interests. He's also a member of the Applied Computing and Technology Study Group as a researcher. M.H Ismail is a lecturer and researcher from Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Perlis Branch, Malaysia.He obtained his first degree in Data Communication and Network and his master’s degree in information technology. His primary research interest is Mobile and Pervasive Computing and is actively involved in mobile technology solution in his community. R. Maskat received her BSc. in IT from the Universiti Utara Malaysia, her MSc. In Software Engineering from the Universiti Putra Malaysia and her PhD. in Computer Science from the University of Manchester, United Kingdom. Prior to her academic life, she was in the computing industry. She is now a senior lecture in the Universiti Teknologi MARA, Malaysia. Her research interests are in Text Analytics, Database, Data Mining and Dataspaces. She has been awarded with the Malaysian government’s research grant for text mining using Natural Language Processing of social media. Hierarchical Fuzzy Systems: Interpretability and Complexity (Tajul Rosli Razak et al)