Application of hidden Markov models for signature verification

Pattern Recognition, Vol. 28, No. 2, pp. 161 170, 1995 Elsevier Science Ltd Copyright ~) 1995 Pattern Recognition Society Printed in Great Britain. All rights reserved 0031 3203/95 $9.50+.00 Pergamon 0031-3203(94)00092-1 APPLICATION OF HIDDEN MARKOV MODELS FOR SIGNATURE VERIFICATION L. YANG, B. K. WIDJAJA and R. PRASAD t Telecommunications and Traffic-Control Systems Group, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands (Received 22 June 1993; in revised form 27 July 1994; received for publication 5 August 1994) Abstract--This paper describesa technique for on-line signature verificationusing Hidden Markov Models (HMMs). Signatures are captured and digitized in real-time using a graphic tablet. For each signature a HMM is constructed using a set of sample signatures described by the normalized directional angle function of the distance along the signature trajectory. The Baum-Welch algorithm is used for both training and classification.Experimental results based on 496 signatures from 31 subjects are presented which show that HMM technique is very potential for signature verification. HMMs Baum-Welch algorithm Signature verification Forward probability 1. INTRODUCTION Multimedia has now become one of the most attractive applied research subjects in the field of telecommunications. Many applications have already started to affect the real life, such as facsimile, videophone and tele-writing systems. I1~Some of the present operations both in business and daily life will experience dramatic change and tremendous improvement in efficiency and other aspects by the introduction of tele-writing systems. For instance, tele-working and tele-banking will make it possible to have business done while the people involved are in other locations. These are expected to be realized in the near future with the continuous improvement on user-machine interfaces and the enhancement of machine intelligence. Writing tablet introduced for improving man and machine communications plays a very important role in the field. One of a few available commercial prototypes is pencomputers which are designed to enable the input of handwritten script for computer systems. One very interesting and potential application is to build tele-banking systems using writing tablets as the man-machine interfaces of the automatic systems. With customers' information stored in a central database accessible by all authorized terminals of the systems, bank operation is expected to be more efficient and customer friendly. This kind of system requires highly reliable ability of machine interpretation and verification of handwriting, apart from sophisticated hardware and telecommunication facility. Reliable automatic signature verification would be of great use in many other application areas in law +Author to whom all correspondence should be addressed. Backward probability enforcement, industry security control and so on. Lately, a lot of effort has been focused on investigation of automatic signature verification methods. In general, automatic signature verification can be done in two fashions: off-line and on-line. The two methods differ in the form by which the input data are captured, tn the off-line situation, signatures prewritten on paper are considered. Because it is difficult to extract individual features from static images or to detect imitations, off-line signature verification is fisually more difficult than on-line verification.121 Most of the systems proposed for signature verification are on-line systems which extract signature features based on position, velocity, acceleration and pressure signals of the pen tip, among others. The comparison algorithms used in those systems are mostly dynamic time warping and regional correlation. Those systems have shown a various degree of success in signature verification.12'3) Other techniques like Fourier transform have also been applied for signature verification with some success/4) Recently, neural network has also been applied for signature verification,tS) In addition to signature verification, signature recognition has also been investigated by several researchers, e.g. Lorette. t6~ In this paper an approach for on-line signature verification is proposed. We attempt to apply the well known speech recognition technique, Hidden Markov Models (HMMs), to the problems of on-line signature verification. HMMs have been successfully used for speech recognition.~7~The application of the method to on-line character recognition was also investigated and the experimental results have shown that the method is very promising/s'9) The advantage of modelling signatures with HMMs is that it is possible to accept variability in signing and at the same time capture the individual features of the signatures. In the proposed 161 162 L. YANG et al. method, signatures are described by the normalized directional angle function of the distance along the signature trajectory. Lorette t6~ used initial angle and total cumulated angle of a signature jointly with other measurements to form a feature vector describing a signature for signature recognition. By applying data analysis and clustering techniques, the feature space was partitioned into clusters corresponding to signature classes. Whereas, we try to incorporate dynamic sequence information of signing by extracting normalized angles (features) along signature trajectory and model the generation of this sequence by HMMs. For each class of signatures, a HMM is constructed using a set of sample signatures and stored as the reference of that class. We have used the Baum-Welch algorithm for both training and classification. To verify a signature, the probability that an unknown signature was generated by a particular HMM is computed. Based on a threshold a decision on whether to accept the signature as authentic or to reject as forgery is made. Our primary research has shown that the HMM approach is very potential for signature verification, t~°~ This paper includes further research on the topic. The paper mainly focused on the issues on processing of signatures in order to apply HMMs. Section 2 presents a brief discussion on the general aspects of HMM and issues of model training and application for classification. We discuss the extraction of features for modelling signatures with HMM, the signature verification process and threshold problem in Section 3. Experimental results are described in detail in Section 4. Finally conclusions are given in Section 5. 2. H I D D E N MARKOV MODELS The HMM models a doubly stochastic process governed by an underlying Markov chain with a finite number of states and a set of random functions each of which is associated with one state. At discrete instants of time, the process is in one of the states and generates an observation symbol according to the random function corresponding to the current state. The model is hidden in the sense that all that can be seen is a sequence of observations. The underlying state obeexvations \T/ oblervatione \TI which generated each symbol is hidden. The HMMs applied here are based on one proposed for speech recognition by Levinson et al. ~ ~ We restrict ourselves to the consideration of processes whose observation sequence are drawn from a discrete finite alphabet according to discrete probability distribution functions associated with the states. A HMM may have different structure. It is possible to constrain a HMM such that only certain desired state transitions are allowed. An example of left-to-right HMM is shown in Fig. 1, where the five circles represent the states of the model and states are not permitted to transit back to the previous states. At a discrete time instant t, the model stays in one of the states and generates an observation. At instant t + 1, the model either remains in the same state or moves to a new state according to the state transition probabilities. This continues until a final terminating state is reached at time T. The model can generate any observation symbol of a finite alphabet from each state governed by observation probabilities. The model is initialized by initial probabilities of occupying states att=l. Quantitatively, a HMM is described as following using the same notation as used by Levinson et al. ~ ~ • set of N states {q~,q2,...,qN}; • a state transition matrix A = {ao}, where aij is the transition probability from state ql to state qj: aij=Pr(q~att+llq~att), l<i,j<N; • set of M discrete symbols {v~,v2..... Vu}; • an observation probability matrix B = ( b j k } , where b~k is the probability of generating symbol Vk from state qj, and • the initial probability distribution for the states H = {rtj},j = 1,2 ..... N; rtj = Pr(qj at t = 1). In general, to use HMMs for pattern recognition, there are two problems: (i) classification--compute the probability that a HMM generated a test observation sequence representing an unknown pattern; (ii) trainin9 of the models--estimate the model parameters based on a training set of observation sequences of each pattern. observations observations \l; \TI Fig. l. Left-to-right HMM with five states. observations \T; Application of Hidden Markov Models 163 2.1. Classification problem ~. Given an observation sequence O = {O1,02 . . . . . Or} representing an u n k n o w n pattern from some vocabulary W = {wl, w: ..... wv}, where each O t is some Vke { U I , U 2 . . . . . / ) M } ' and a set of V models M 1 , M 2 . . . . . M v each of which for one vocabulary word, the classification of an u n k n o w n pattern requires the computation of Pr~(OIMO for 1 _< i_< V. An u n k n o w n pattern is classified as w~ iff Pri >-Pr~ for 1 < j _< V. An efficient method to calculate Pr(O]M) is to use forward and backward probability, known as the forward-backward algorithm3 ~~ Denote 'forward' and 'backward' probability a s o~t(i ) and f,(i), respectively. They are given by: ~'+'(J)=[ ~ ct'(i)ao] l < t <_ T - - 1 (1) oq(j)ft(j) -bjk ~ t~Ot = vk r ~,(J)f,(J) t=l ffi . . . . . . (5) 1 Pr(OIM) ct, (i)f I (i). A detailed description of the algorithm and discussions on implementation issues can be found in literature." 1) For left-to-right models, the re-estimation is based on a set of Q observation sequences which all start at state q l(nl = 1). The re-estimation procedure is then modified to handle multiple observation sequences and aimed to adjust the model parameters such that the following is maximized: t2 (2 [I Pr(O(k)lM)= 1-I Prt" N f , ( i ) = ~ a~jbj(O,+Of,+l(j) j k=l T - l - > t ->1, (2) (7) k=l 1 setting ~1(i) = nibi(O 0 for all i, and fir(J) = 1 for allj. Equations (1) and (2) can be used to calculate Pr(OI M) which is given by: The modified re-estimation formulae are given by: t2 T - l E2 k=l ~:tk(t)aij bj(O tk+ 1)fl, + 1(J) t-1 (8) aij = N Q (3) k=l ~,U)f,U) for 1 ~ < t ~ < T - 1 . bjk = k= a ,~o, = ~ Q T 2.2. Training problem The training process can be generally described as the following steps: (i) make an initial guess of M; (ii) use some re-estimation algorithm and O to derive a new model M' with the property that Pr(OIM') -> Pr(OIM); (iii) replace M by M' and repeat the re-estimation. The re-estimation process iterates until the increase in Pr (OIM) is small enough. Here, we consider the B a u m Welch algorithm which guarantees to increase Pr(OIM) with the re-estimated A, B and n until the optimal point is reached. (~ 1) In this algorithm, the forward and backward probabilities are used to solve the problem of training by parameter estimation. Given some estimate of M and an observation sequence O, a new estimate ofa~j is computed as the ratio of the expected number of transitions from state q~ to qj, to the expected n u m b e r of transitions out of state qi, conditioned on O. A new value of bjk is estimated as the ration of frequency of occurrence of vk in state qj to the frequency of occurrence of any symbol in state qj. In terms of forward-backward probabilities, the parameters are computed as: T t=l O i=1j=1 Z T- 1 N ~ ~t,(i)a#bj(O,÷l)f,+,(j) Pr(OIM)= E 1 u,(i)aijbj(O,+ Oft+ ,(J) t=l aij = (6) (4) T-1 ct,(i)ft(i) t=l k=l 3. S I G N A T U R E (9) t=l VERIFICATION APPLYING HMM 3.1. Description of signatures In order to apply H M M s for signature verification, it is very important to find appropriate description for signatures which are independent of translation, rotation and scaling of a signature. Here, a signature is captured in real time using a writing tablet. The absolute angular direction of a signature as a function of the distance s along the signature trajectory, denoted as O(s), is used to represent a signature. In the digitized form, a signature is encoded as a sequence of absolute angles: 01,02,..., 0 i. An example is shown in Fig. 2. The rotation invariance is realized by subtracting the starting angle from 0 i. Then, we get a sequence of angles (0i - 01) which is independent of rotation. The effect of size variance of signatures is eliminated by normalization. Each signature is uniformly divided into K number of segments. The length of the segments are slightly different from each other due to the length difference in tablet quantization vectors. The n u m b e r K is the observation length of signatures for use in HMMs. The normalized angle of a segment, denoted as q~(k), k = 1, 2 . . . . . K, is derived as follows. Assume segment k consists of n samples provided by the tablet and the distance between the samples is st(I = i + 1..... i + n). L. YANGet al. 164 "~__0_I÷1 Fig. 2. Encoding of a signature. The angle ~b(k)of this segment is given by: (k)=arctan| -- _--|. L,Y+, s,,,,ooso,,,,_j (10) The normalized angle is then quantized into 16 levels as: q~*(k) = Q[q~(k)], k = 1,2 .... K. (11) The simple quantization scheme has been justified by Veltman and Prasad. (9) Each of the 16 levels is represented by a symbol called a pen-down symbol. The pen-ups within a signature are eliminated by linear interpolation. Those pen-ups are detected using a threshold. When the distance between two consecutive points exceeds the threshold, pen-up is assumed. The threshold is selected as the possible largest distance between any two consecutive sample points captured by the tablet. To distinguish the interpolated trace from the ordinal trace, another 16 symbols have been introduced, called pen-up symbols. The pen-up symbols have exactly the same values as the pen-down symbols, but with different token. Thus, a signature is described by a sequence of observation (~b*(k)) of length K, each of which is one of the 32 symbols. Figure 3 presents some examples of quantized signatures applying different number of symbols. In the case of using merely pen-down symbols, the interpolated portion of the trace between pen-ups is treated as ordinary part of the signature and, therefore, a signature becomes one continuous trajectory. However, when pen-down symbols and pen-up symbols are used combined, the pen-up part is recorded by the pen-up symbols. Figure 3(a) is the original signature captured by a tablet. In Fig. 3(b)-(d), the signatures described using 16 pen-down and 16 pen-up symbols, 8 pen-doWn and pen-up symbols and 4 pen-down and 4 pen-up symbols are presented, respectively. Figure 3(e)-(g) show the quantized signatures applying 32 pen-down symbols, 16 pen-down symbols and 8 pendown symbols, respectively. It can be seen that the signature is greatly distorted when too few number of symbols are used. The distortion is in an accumulative fashion and it will directly influence the performance of the method. It is also seen that the distortion is more severe in the case of using pen-down plus pen-up symbols than pen-down symbols only for the same total number of symbols is used. This is obvious because the former situation actually uses half of the total symbols in quantization. In our primary research, we have observed that using pen-up symbols can improve the performance of the method in certain circumstance. It means that the introduction of pen-up symbols compensates the effect of distortion to certain extent. More discussion on this issue can be found in Section 4. The increase of number of symbols will greatly increase the time needed for model training. If the number is chosen too large, the time used in model training may become impractical. Therefore, a certain compromise between the degree of distortion allowed and the time cost for model training needs to be made in deciding the number of symbols to use. 3.2. Signature verification The proposed signature verification system works as follows. The signers sign their names with a pen on a tablet that captures the information of signatures. Based on a number of sample signatures of a signer, a H M M is trained applying equations (4)-(6) or (8)-(9). The derived model is used as the important and unique feature of the signatures and it is stored as the reference of that signer. The average probability (or mean) of the Application of Hidden Markov Models 165 (a) (b) (e) (e) (0 i i.a--1 i I --..., ---, (d) (g) Fig. 3. (a) Original signature; the signature quantized using: (b) 16, (c) 8, and (d) 4 pen-down and pen-up symbols. (e) 32, (f) 16, and (g) 8 pen-down symbols. training signatures being generated by the model, denoted a s / 5 is also stored. Whenever someone claims to be a particular person and writes his signature on the tablet, the system retrieves the reference of the claimed signer and calculate the probability (P,) that the signature is generated by the particular model. A decision is made on whether the signature is authentic or forged using a threshold do. If P, is below the threshold the signature is rejected as a forgery. There are generally two types of error which may occur in the signature verification process: false rejection (FR), the error which arises when an authentic signature is rejected; false acceptance (FA), the error for accepting a forgery. The level of the two error rates are dependent on the threshold chosen. In this paper, d o is chosen to be dependent on /5 obtained using the training signatures as: do . . . . . . . . , = 6. fi, ........ ~, (12) where the factor 6 is determined in experiments. The value of 6 is dependent on the deviation of P, from its mean/5. For each signer, 6 can be determined accor- ding to the probability deviation behaviour. Figure 4 plots the mean and deviation of - l o g ( P , ) of both genuine and forgery signatures of the training signatures of 31 signers in the database used in our experiments (only simple forgery was considered, see Section 4 for more information on the experiment considerations). In Fig. 4, the points marked with • and [] are the mean and standard deviation of genuine signatures of all the signers, respectively, while the points marked with • and O are the mean and standard deviation of forgeries, respectively. It can be observed that for most of the signers, the differences between the mean probabilities of genuine signatures and forgeries are fairly large. On the other hand, the standard deviations of probabilities of genuine signatures of all signers are relatively very small and more or less consistent. Based on these observations, we decided to use an uniformed 6 in our system in order to simplify the problem. According to our experimental results, a 6-state left-to-right H M M shows a relation between 6 and error rates as in Fig. 5. It is reasonable to chose 6 to be 0.85 where FR and FA equals. But, it 166 L. YANG et al. ~-., 14oo • . *, I, "6 l=oo C ", i >e looo ,' o' 2 ~..<~ '10 C ¢g t= ¢U i-., ', / ID 1=o ,, ," ..9".~ i ' ta i .~. .,¢' .~ '.~, ~,. . . .~.. ~ ~..~ . ~ ~t~,; \ ' t i . . . . IL~-i' ' ; ~ <;' t (1(30 ti , , oli : e. 6 .:, -i : " . ~ ~- ° .. o • .: . ... ~ ! l-."J,= ~ I ",..7 <c. %' t7 i • :E 4OO • i ' 2 i 4 ' i 6 ' ] 8 i \ r I 10 ' I 12 ' I 14 ' I 16 ' I 18 ' I 20 ' I 22 ' I 24 ' I 26 ' I ' 28 F ' 30 Signer No. Fig. 4. Mean (11) and standard deviation ([]) of -1og(Pr) of genuine signatures of each signer; mean ( e ) and standard deviation (O) of - log(Pr) of forgeries. 100 ~(~) gO 70 "\ File R i g O r / eO 50 40 30 20 10 0 0.~ ci.4 o.5 o.e 0.7 0.8 0.9 1 1.1 12 1.3 Fig. 5. Error rates vs threshold of a 6-state left-to-right HMM. is expected that if more elaborate 6 is selected for each signer, the performance should be improved, to a certain extent. Further, we observed in the experiments that the mean and standard deviation of P,(thus the choice of 6) is influenced by the number of signatures used in training when the number of training signatures used is too small, say 4. If the training is based on sufficient number of signatures (above 7), the influence becomes hardly observable. 4. IMPLEMENTATION AND EXPERIMENTAL RESULTS An experimental system using H M M s for signature verification was implemented on a personal computer (486 DX2 33MHz processor) connected with a graphic tablet (DSD 703 Digitizer, electromagnetic). The tablet captures the trajectory of handwriting in spatial sampling fashion and provides the x - y coordinates of the samples. The tablet has an effective working area of 19 cm x 14cm with resolution of 2100 x 1536 points. The maximal data transmission rate to the computer is 4800 coordinate pairs per second. The recognition software system is implemented in PASCAL. The experiments were carried out based on a data base containing 496 signatures of 31 signers. Each of the signers was asked to write his/her signature 16 times of which 8 times were used for model training and the other 8 times for verification test. For each signer, a H M M was generated based on his/her training signatures and the model training was done off-line. To evaluate the performance of the system the authentic signatures of each signer and the other signatures of the remaining 30 signers which were considered as forgeries were tested over each corresponding model. The overall average error rate of false rejection and false acceptance were evaluated. First, several model structures were investigated in order to obtain the most suitable model for the problem. The models chosen are listed as following and also shown in Fig. 6: Application of Hidden Markov Models 167 (a) (d) (b) (e) (¢) Fig. 6. Different model structure: (a) LTR-1; (b) LTR-2; (c) LTR-3; (d) parallel; (e) generalized. (a) LTR- 1: only left-to-right transitions are allowed. (b) LTR-2: only left-to-right transitions and single skips of states are allowed. (c) LTR-3: only left-to-right transitions and no skips of states are allowed. (d) Parallel: only left-to-right transitions are allowed in which some states are parallel to each other. (e) Generalized: transitions from any states to any other states wilt be possible. In the experiments, the number of states used is 6 and the observation length is 300. The overall false acceptance and false rejection error rates were measured for the test signatures. The results are shown in Table 1. It is clear that the constrained models performed better than the generalized model. Among the constrained models, it is observed that the LTR-1 model gives the best performance. There is a clear trend that the leftto-right model with less restriction gives better performance than those with more restriction. This implies that the increase of freedom in the restricted model may allow more inherent variance in the signatures of a signer and at the same time can still reasonably model the unique characteristics of a signer and, however, too much freedom in the model may increase the probability of accepting false signatures. The reason that left-to-right model gives better performance than the generalized model may be that the left-to-right model captures the dynamic characteristics of signing in a better way. Left-to-right models inherently impose a temporal order to the HMM since lower numbered states corresponds to observations occurring prior to those to higher numbered states. The progressive nature of the states sequence reflects a certain dynamic characteristics. We chose LTR-1 model structure for our further experiments. The performance under different number of states was also evaluated. With the number of states varying from 2 to 9, we obtain the results shown in Fig. 7, setting the observation length at 300. In spite of a statistic fluctuation, a slow steadily improvement of performance is observed as the number of states increases. However, the increase of the number of states does not show significant influence on the performance when certain number of states is reached. It is shown that the selection of 6 states in the model is reasonable. Table 1. Error rates of different models. Type of error rates (%) Model structure LTR-1 LTR-2 LTR-3 Parallel Generalized False acceptance 6.45 8.47 False rejection 1.18 11.69 1.41 2.54 0.78 5.24 14.11 1.87 Average 3.815 5.17 6.55 3.89 7.445 168 L. YANG et al. Increasing the number of states will greatly increase the time needed to train a model. The experimental results showed that the performance of the model is deteriorated using too small number of states, i.e. less than 4. It is expected that a greater number of states should be used with greater observation length in a model. Investigation on performance with different observation lengths was carried out by varying the observation length from 100 to 500, while the number of states was chosen as 6. The experimental results are shown in Fig. 8. It is observed that at very low observation length, say 100, the performance is very poor. This is because that too short an observation length will lead to very rough description of a signature and, consequently, the inaccuracy in the signatures resulted in a large false acceptance of signatures. As the observation length increases, more details of signatures are captured. 10 As shown in the figure, it is suitable to choose the observation length to be 300. Larger numbers of observations may slightly improve the performance. However from time consumption point of view in model training, the gain may not be worth. As the number of observation length increases, the time used for a model training is increased tremendously. It is observed that at higher observation length, the error rates may even increase. This may be due to the fact we mentioned earlier that for a greater observation length the model should incorporate with a greater number of states. However, an optimal solution to observation length is to adapt observation length according to the length of signatures and at the same time eliminate the influence of size variation. In general, it is reasonable that for short signatures a smaller observation length can be used than that for long signatures. To determine the length of observation for each signature class indi- Error rates ( % ) 9 / 8 j J t / f F ~ s e /.m:xx~tm'me / t/1 7 . \ 6 5 4 3 r~ectlon 2 1 0 6 a S ,i e 7 S 0 N u m b e r of s l a t e s Fig. 7. Error rates vs number of states. 18 Error rates ( % ) 16 14 '\ \ \ \ 12 \ \ \ \ 10 8 \ \ \ \~ False acceptance 6 \ 4 False rejection 2 0 1OO 150 200 250 300 350 400 4,50 500 Observation length Fig. 8. Error rates vs observation length. Application of Hidden Markov Models 169 Table 2. Error rates with different number of symbols. Total symbols 32 32 16 16 8 8 4 4 Number of symbols Pen-down symbols Pen-up symbols 32 16 16 8 8 4 4 2 -16 -8 -4 -2 vidually, an appropriate solution is to chose the observation length in experiments according to the performance of signature verification. To find an optimal value for each signature class requires tremendous time. In our future research, an adaptive observation length will be considered and the results will be reported in the near future. Section 3.1 has shown that the number of symbols used has great impact on the way signatures are described. It is very useful to see how it influences the performance of signature verification. The performance applying the number of symbols of 4, 8, 16 and 32 were evaluated both in the cases of using only pen-down symbols and pen-down plus pen-up symbols. When pen-down and pen-up symbols are used combined, it means that half of the indicated number of symbols are pen-down symbols and half of it are pen-up symbols. Table 2 presents the results obtained in our experiments. It is shown that the introduction of pen-up symbols indeed improves the performance when the distortion introduced is not too large. As the number of symbols decreases, the performance became deteriorated in both cases and, especially, the advantage of using pen-up symbols is diminished. It can be seen that the performance may even become worse using pen-down plus pen-up symbols than using only pen-down symbols at some point. As shown in Section 3.1, when too few number of symbols is used, signatures are severely distorted resulting in poor performance. 5. CONCLUSIONS We have presented in this paper a method for online signature verification applying HMM technique. The signature features are taken as the normalized angular direction as the function of the distance from the starting point of the signature. The experimental results show good signature verification performance against substitution forgery. The overall error rates of 1.75~o and 4.44~ofor false rejection and false acceptance, respectively, were observed. Generally speaking, leftto-right HMMs capture the information of signing better than other model structures. The selection of the number of states, number of symbols and observation length is made with compromise between performance and time cost in model training. The use of pen-up Error rates (~) False acceptance False rejection 7.55 4.44 8.1 4.26 16.01 16.43 36.02 38.91 1.21 1.75 1.04 4.03 1.21 3.23 3.23 4.44 symbols improves the performance to a certain extent. When the number of symbols chosen decreases, the improvement is diminished. At some point, the performance is even worse. The performance may be further improved using some better discrimination method instead of a simple uniform threshold. Our further attention will be focused on the system given satisfactory performance against skilled forgery by combining dynamic information of signature, i.e. pressure, velocity and/or acceleration, etc., in the verification system. REFERENCES 1. J.C. Arnbak, J. H. Bons and J. W. Vieveen, Graphical correspondence in electronic-mail networks using personal computers, IEEE Selected Areas in Commun. 7(2), 257-267 (1989). 2. R. Plamondon and G. Lorette, Automatic signature verification and writer identification--the state of the art, Pattern Recognition 22(2), 107-131 (1989). 3. M. Parizeau and R. Plamondon, A comparative analysis of regional correlation, dynamic time warping, and skeletal tree matching for signature verification,IEEE Trans. PAMI 12(7), 710-717 (1990). 4. C. F. Lam and D. Kamins, Signature verificationthrough spectral analysis,Pattern Recognition 22(1),39-44 (1989). 5. H. Chang, J. Wang and H. Suen, Dynamic handwritten Chinese signature verification, Proc. 2nt Int. Conf. on Document Analysis and Recognition pp. 258-261, Tsukuba, Japan (October 1993). 6. G. Lorette, On-line handwritten signature recognition based on data analysis and clustering,Proc. 7th Int. Conf. Pattern Recognition 2, 1284-1287 (1984). 7. S. E. Levinson, L. R. Rabiner and M. M. Sondhi, On the application of vector quantization and hidden Markov models to speaker-independent,isolated word recognition, Bell Syst. Tech. Jnl. 62, 1075-1105 (1983). 8. A. Kundu, Y. He and P. Bahl, Recognition of handwritten word: first and second order hidden Markov model based approach, Pattern Recognition 22(3), 283-297 (1989). 9. S. R. Veltman and R. Prasad, Hidden Markov models applied to on-line handwritten isolated character recognition, IEEE Trans. on Image Processing 3(3), 314 318 (1994). 10. L. Yang, B. K. Widjaja and R. prasad, On-line signature verification applying hidden Markov models, Proc. 8th Scandinavian Conf. on Image Analysis pp. 1311-1316, Tromso, Norway (1993). 11. S. E. Levinson, L.R. Rabiner and M. M. Sondhi, An introduction to the application of the theory of probabilistic Functions of a Markov process to automatic speech recognition, Bell Syst. Tech. J. 62, 1035-1074 (1983). 170 L. YANG et al. About the Author--LIPING YANG received the B.S. degree in electrical engineering from Hunan University, P.R. China, in 1982. She started graduate studies in electrical engineering at Delft University of Technology, The Netherlands, in 1990. She is currently a Research Assistant in this university and working towards her Ph.D. degree. Her current research interests include analysis and recognition of handwriting, signature verification, image processing and multi-media communications. About the Author--MICHAEL B. K. WIDJAJA was born in The Hague, The Netherlands on 19 June 1970. He received the M.Sc.E.E. degree from Delft University of Technology, The Netherlands. He has done research on signature verification applying Hidden Markov Models. His interests are in signal processing, pattern recognition and communication systems. About the Author--RAMJEE PRASAD, since February 1988, has been with the Telecommunications and Traffic Control Systems Group, Delft University of Technology, The Netherlands, where he is actively involved in the area of personal radio and multi-media communications. He has published over 150 technical papers. Prof. Prasad is listed in the U.S. Who's Who in the world. He was organizer and Interim Chairman of IEEE Vehicle Technology/Communications Society Chapter, Benelux Section. Now he is elected chairman of the joint chapter. He is also founder of the IEEE Symposium on Communications Vehicle Technology (SCVT) in the Benelux and he was the Symposium chairman of SCVT'93. He is one of the Editor-in Chief of a new journal on "Wireless Personal Communications" and also a member of the editorial board of the other international journals including IEEE Communication Magazine. He is the technical Program Chairman of PIMRC'94 International Symposium to be held in The Hague, The Netherlands during 19-23 September 1994 and also of the Third Communication Theory Mini-Conference in conjunction with GLOBECOM'94 to be held in San Francisco, California during 27-30 November 1994. He is a Fellow of IEE, a Fellow of the Institute of Electronics & Telecommunication Engineers, a Senior Member of IEEE and a Member of the New York Academy of Science and of NERG (The Netherlands Electronics and Radio Society).