Mathematical Models of Diagnostic Tactics in Psychiatry

Mathematical Models of Diagnostic Tactics in Psychiatry Josef M. Zislin, M.D., Alexander Rybalov, Ph.D.*, Rimona Durst, M.D. Kfar Shaul Mental Health Center, Jerusalem; *Jerusalem College of Technology This article was presented at the XII World Congress of Psychiatry, Yokohama Japan. August 24-29 2002. Address for reprints. Dr. J. Zislin. Kfar Shaul Mental Hospital, Givat Shaul B, Jerusalem 91060, Israel. E-mail: [email protected] Background Investigation and the formalization of diagnostic procedures are important processes in the practice of modern medicine. Diagnostic tactics need to integrate theoretical approaches and cognitive elements. Aims To identify and describe patterns and/or attitudes according to which the diagnosis of mental illness is conducted by psychiatrists. Method The PANSS scale has been applied as the initial basis of our research. This scale has been modified so that apart from giving each characteristic (symptom) its corresponding grade, the doctor is also able to analyzed the relevance scale of these characteristic in the final diagnosis of a patient. The mathematical model used in Classical Test Theory was applied to identify quantitative parameters. Results We have found dependencies between tactics that are used by physicians and the relevancy of clinical characteristics and quantified these dependencies. The following quantitative diagnostic criteria have been identified: Caution/Recklessness; Flexibility/Rigidity; Experience/Lack of experience. Conclusions Using these three pairs of characteristics (caution/recklessness, experience/inexperience, flexibility/rigidity) we can construct unique individual synchronic and diachronic characteristics for every physician and quantify relationships between tactics, diagnosis and treatment. This approach enables clinicians/researchers to build these characteristics in dynamics and thereby to see professional changes over time. Declaration of interest None. Diagnosis is the aim and final stage of the diagnostic procedure. Recently, important advances have been made in the theory of decision making in clinical practice, medical problem-solving and cognitive basis of diagnostic procedure (Bordage et al, 1991; Custers et al, 1996; Charlin, 2000; Hastie, 2001; Patel, 2001, 2002; Sadegh-Zadeh, 2000). Internationally accepted diagnostic criteria in psychiatry (such as ICD or DSM) were adopted with the purpose of unifying the diagnostic nomenclature. Whilst the DSM-R and ICD-10 are internationally accepted and widely used, the processes of reaching diagnosis and establishing criteria with a stable theory of diagnostic procedure and quantitative measurements have not been standardized yet. The terms “Diagnostic tactics” as used in our research, refers to methods of identification and grading of certain symptoms. Each symptom has its own significance and evaluation of its subjective relevance. In our approach diagnostic tactics implicitly includes two main components: 1) formal diagnostic entities and 2) cognitive rules dealing with these criteria. The aim of the present research is to identify and describe patterns according to which the diagnosis of illness is conducted by a psychiatrist. Method The principal clinical scales which have been developed, including PANSS, CGI, Hamilton Depressive Scale and the like, enable us to make the first steps towards establishing and formalizing clinical diagnosis (Kay et al, 1988; Kay, 1991; Hamilton, 1960). The principles employed in the creation of these different scales are remarkably similar. Typically, the psychiatrist grades the symptoms of a particular illness. Symptom grading is performed either qualitatively (presence or absence of the symptom) or quantitatively on a numerical scale, e.g. from 1= absent, up to 7=extreme (PANSS) (Kay et al, 1988; Kay, 1991); from 0 = absent, up to 3 = severe (Barnes Akatisia Rating Scale) (Barnes, 1989). All these scales have similar disadvantages: - Subjectivity of evaluation - All symptoms are given equal significance not corresponding to real clinical situations A simple mathematical sum of digital items indicates severity of illness. The purpose of our research was not to improve this particular scale (PANSS). Rather, proceeding from the modified and simplified version of the PANSS scale, we aimed to gain an understanding of how the scale is used by psychiatrists and on what variability evaluation is based. For our research only the first seven items (positive scale of PANSS) have been used. In addition to the standard scale each item is simultaneously evaluated by a psychiatrist on the relevance scale in a particular diagnostic situation. It should be noted that the item can be graded by the psychiatrist as 1 = absent, but can still have a relevance of 10 (maximum). We can interpret this situation as follows: the psychiatrist admits that a certain symptom does exist, but that for some reason or other it cannot be clearly defined. At the same time, the psychiatrist can assume that this particular symptom can play a very important part in the clinical picture of the disease on the whole. To make diagnostic criteria more precise, diagnostic scales have been designed according to the same principle. We have based our approach on the supposition that different diagnostic parameters used on a given scale have different weight. This assumption is based on observations that in diagnostic practice a physician who has identified only one symptom is able to build the entire diagnostic chain. The classical way of arriving at a diagnosis is through putting together several symptoms. However, what frequently occurs in clinical practice is that the physician, first postulates a diagnosis, and then subsequently identifies symptoms to verify his hypothetical diagnosis. When the diagnostic process occurs in reverse (from diagnosis to symptoms) errors are made. Signs and symptoms that don’t objectively exist acquire unreasonably large weighting whereas existing signs and symptoms are ignored The techniques described above make it possible to define the principal parameters of the diagnostic process. We succeeded in showing that even where there are few patients, the diagnostic patterns of individual psychiatrists can be defined. These patterns vary between practitioners and are revealed in different distributions of the weights of diagnostic parameters. Thus, we have to define a new measure that captures these patterns. In our terms they describe the dynamics of the changes of relative distributions of weights for different patients. The mirror problem will be to define a new variable that determines the persistency of these patterns. For illustration, consider the situation of having only two patients. In this case the problem can be restated in the following way: to what extent can the relative distribution of the weights of diagnostic parameters for one patient be explained by the distribution for another patient. In the Classical Test Theory (CTT) [(Lord et al, 1969) one of the most important concepts discussed is reliability generally understood to mean the extent to which a measure is stable or consistent. It is defined as the squared correlation coefficient between observed and true scores. Correlation ρ determines to what extent a change in one variable, x = {x1, x2,?xi,?xn}(true scores), influences the change in another, y = {y1, y2,?yi,?yn} (observed scores). where and are averages of x and y correspondingly. In most cases correlation is used to determine the relationship between variables. In this case correlation is used to define the new variable: reliability. We follow the same approach to determine the persistency of distribution of the weights of the diagnostic parameters for these two patients. In other words, we want to determine similarity between the vectors representing the weights of the diagnostic parameters of two patients. In our case the correlation coefficient ρ between these two vectors determines to what extent the distribution of these weights for one patient influences the distribution of these weights for the other. To find out the persistency of distribution of weights for all patients we need averaging of these correlation coefficients for all patients of an individual psychiatrist. Results The following diagnostic characteristics were identified: I. Experience is defined as the average of correlation coefficients ρij between the vectors of weights of the diagnostic parameters (n is the total number of patients): where n is the total number of patients and ρij is the correlation coefficient between weights wik and wjk of the diagnostic parameters k (k = 1,…, m) for patients i and j: where and are averages of the weights for patients i and j. The smaller value of Experience corresponds to the greater number of diagnostic approaches tried by the psychiatrist and, therefore, the more variation in tactics. The latter comes with experience (unfortunately, not always). Therefore, we use this parameter as a proxy for it. Experience determines dynamics of the changes of distributions of weights and therefore is regarded as a dynamic parameter. These heuristics can be revealed in the tactics used by psychiatrists. One commonly used heuristic is to make the diagnosis by assigning the highest weights to diagnostic symptoms having the highest grades. Once it is established that the patient has some severe symptoms, only these symptoms determine the diagnosis. This heuristic should be associated with caution. To find the quantifying measure of these heuristics, we have to define the parameter that captures the extent of the dependence of the weights of diagnostic symptoms from values of the symptoms themselves. Here we also use correlation. In this case the resulting measure is static: it doesn’t depend on the number of patients and can be calculated even if there is only one patient. II. Caution is defined as the normalized sum of the correlations between the grades of symptoms and their weights where ρi is the correlation coefficient between the grades and their relevancies for patient i; n is the total number of patients. If the doctor decides that the weight of a particular symptom should be high only because the corresponding grade is high, then the correlation coefficient is high. It can be interpreted that the doctor is risk-averse. If the doctor determines that the weight of any characteristic should be independent of its corresponding grade, then the correlation coefficient should be low. In this case such a doctor is likely to be risk-taking. Another heuristic to be considered is to base the diagnosis on selected symptoms and to large extent disregard others. In other words, the weights of symptoms will vary little from one patient to another. In this case the diagnosis will depend only on these selected diagnostic symptoms. The weights of these (as well as other) symptoms will vary slightly, and so the tactics based on this heuristics is as follows: if a psychiatrist changes weights of different characteristics easily as he/she moves from one patient to another then his/her tactic can be viewed as flexible. Contrastingly, if a doctor changes weights little, then his/her tactic is rigid. Therefore, the key measure will be the differences between the weights. To quantify these heuristics, we use absolute values of the differences between the weights of the symptoms of different patients. For every two patients we have to take the average of the absolute values of differences between the weights for all symptoms. To calculate the measure for all patients, we have to normalize the result. III. Flexibility is equal to the normalized average of the sum of absolute values of the differences between the weights for all patients: where wij is the weight of symptom i for patient j; abs (x) is the absolute value of x; m is the number of symptoms; n is the number of patients. Flexibility depends on the dynamics of changes of the weights for different patients and, therefore, similar to Experience, is a dynamic parameter. Discussion Given the scientific character of diagnostic judgment, it is surprising that the nature of the diagnostic process has not been scrutinized. Classic diagnostic procedure consists of three stages: verification of symptoms, verification of syndromes, and verification of the disease. However it is clear that other processes can be in the mind of a psychiatrist. Moreover, given the unique importance of diagnosis to the practice of medicine, it is even more surprising that the study of diagnosis should continue to make so small a contribution to standard medical school curricula. Traditional attitudes to diagnosis as a 'process of intuition' or an arcane 'natural art’ go a long way towards explaining this situation. The stages of the diagnostic process remain unexplored, in spite of the fact that various models have been proposed for diagnostic work in clinical practice. Researches of last decades have shown necessity of construction cognitive models of diagnostic process and have described different cognitive strategies of diagnostic process (Croskerry, 2002). It was demonstrated (Simon, 1955) that human beings have significant information-processing limitations (e.g. attention, memory, and perceptual constraints) and therefore have to use simplifying heuristics. However, these heuristics, instead of being viewed as erroneous, can be regarded as powerful and effective strategies to make many everyday decisions (Patel et al, 2002). For example in psychiatry most diagnoses are made within 30 seconds to three minutes of meeting an individual (Gauron et al, 1969). But here lies the crux of problem: these heuristics don’t fit well in Procrustean bed of Unification principles. Moreover, in diagnostic decision-making people do not use one but several heuristics simultaneously. The degree to which a particular heuristic is applied in diagnostic process plays a crucial role in decision-making. To determine this degree we need exact quantitative measures describing diagnostic process. Unfortunately, traditional attitudes toward diagnosis are responsible for the situation where discussion about such measures is outside of current research of diagnostic decision-making. In one of the first works in diagnostic strategies in psychiatry (Gauron et al, 1966) six types of diagnostic approaches were defined. These types were based upon two-dimensional structure: number of pieces of information and probability level. They include intuitive-adversary, diagnostic-by-exclusion, overinclusive-indecisive, textbook, bibliography and flexible-adaptable approaches. In this study we propose a new approach, based upon persistency of relations between clinical characteristics using in diagnosis process (such as hallucinations, delusions etc. in our clinical examples) and diagnostic tactics of psychiatrists. Our approach is twofold: 1) establishment of dependencies between tactics of a psychiatrist and relevancy of clinical characteristics; 2) quantification of these dependencies. These dependencies reveal influence of heuristics used by psychiatrists in diagnostic process. Heuristics determine persistence of patterns of relevancies of clinical characteristics. Thus, quantification of dependencies between tactics of a physician and relevancy of clinical characteristics is reduced to quantification of persistency of such patterns. To solve this problem we apply the approach similar to one used in the Classical Test Theory (CTT) where the variable determining persistency of relations is defined as correlation between corresponding parameters. This approach allows us to introduce quantitative characteristics (caution/recklessness, experience/inexperience, flexibility/rigidity) that can describe decision-making in psychiatry. The most important applications of this model are as follows: 1. Using these three pairs of characteristics (caution/recklessness, experience/inexperience, flexibility/rigidity) we can construct unique individual synchronic and diachronic characteristics for every physician. 2. We are able to quantify relation between tactics and diagnosis and treatment. Both prospect theory and cumulative prospect theory propose that in decision making, small probabilities are overweighted and large probabilities underweighted, contrary to the assumption of standard decision theory. In our model it means that grades of clinical characteristics (but not their relevancies) change to smaller extent than warranted by observation i.e. values of experience/inexperience are greater. Thus, such pattern of decision-making is associated with less experience. This approach allows to build these characteristics in dynamics and thereby to see professional changes over time. Our method is based on psychiatric practice only. However, according to our opinion this method can be applied to any medical field. Clinical implications · This method provides opportunity to construct objective professional characteristics of physicians. · Professional personnel selection based on an objective approach can now be realized. · The practical implication of this method can lead to change of self-awareness and higher professional responsibility of psychiatrists. Limitations · The minimum number of patients needed for constructing of reliable characteristics should be at least seven. · It can be difficult to convince clinical staff of the importance of the second scale (relevancies of clinical characteristics). · For each diagnostic entity relevant and verifiable diagnostic scale is required. References Barnes, T.R.E., (1989) A rating scale for drug-induced akathisia. Br J Psychiatry, 154, 672-676. Charlin, B., Tardif, J. & Boshuizen, H. (2000) Scripts and Medical Diagnostic Knowledge Theory and Applications for Clinical Reasoning Instruction and Research Acad. Medicine, 75, 182-190. Bordage, G., Lemieux, M. (1991) Semantic structures and diagnostic thinking of experts and novices. Acad Meicine, 66 (9 Suppl),S70-72. Gauron, E.F., Dickinson, J.K. (1969) The influence of seeing the patient first on diagnostic decision making in psychiatry. Am J Psychiatry, 126, 199-205. Gauron, E.F., Dickinson, J.K. (1966) Diagnostic decision making in psychiatry. II. Diagnostic styles. Arch Gen Psychiatry, 14, 233-237. Croskerry, P. (2002) Achieving Quality in Clinical Decision Making: Cognitive Strategies and Detection of Bias. Acad Emerg Med, 9, 1184-1204. Custers, E.J., Regehr, G. & Norman, G.R. (1996) Mental representations of medical diagnostic knowledge: a review. Acad Med, 71, 55-61. Hamilton, M. (1960) A rating scale for depression. J Neurol, Neurosurg Psychiatry, 23, 56-62. Hastie, R. (2001) Problems for judgment and decision making. Ann Rev Psychol, 52, 653- 683. Kay, S.R., Opler, L.A. & Lindenmayer, J.P. (1988) Reliability and validity of the Positive and Negative Syndrome Scale for Schizophrenia. Psychiatry Res, 23, 99-110. Kay, S.R. (1991) Positive and Negative Syndromes in Schizophrenia. New York: Bruner/Mazel. Lord, FM., Novick, MR. (1968) Statistical theories of mental test scores. Reading, Addison-Wesley. Patel, V.L, Arocha, J.F, Kaufman, D.R. (2001) A primer on aspects of cognition for medical informatics. J Am Med Inform Assoc, 8, 324-343. Sadeh-Zadeh K. (2000) Fundumentals of clinical methodology. 4. Diagnosis. Artificial Intelligemce in Medicine, 20, 227-241. Simon H.A. (1955) A behavioral model of rational choice. Q J Econ. 69, 99-118. Vimla, L., Patel,D., Kaufman1,& Arocha, J. (2002) Emerging Paradigms of Cognition in Medical Decision Making, Journal of Biomedical Informatics,35, 52-75. Appendix This is an example of one physician (code-named RD) who completed diagnosis for seven patients (results are in the table below). The resulting values of criteria shows that he is experienced (low value of 0.28 indicates high experience), cautious (high value of 0.74 out of maximum 1 indicates high degree of caution), and flexible (value of 1.08 is the one of highest possible and the highest among the group of 8 participants). Tabl 1. 1 1 2 2 3 3 4 4 Grade Importance Grade Importance Grade Importance Grade Importance 7 10 6 10 7 10 7 10 6 8 5 8 6 9 5 9 1 5 1 8 6 10 4 8 3 7 6 8 4 7 4 8 5 8 1 5 6 9 7 10 2 8 7 9 6 9 2 3 3 6 6 9 1 6 3 5 5 5 6 6 7 7 Grade Importance Grade Importance Grade Importance 7 10 1 4 1 8 3 8 2 3 6 9 1 7 1 4 1 9 6 9 2 2 3 6 2 7 1 3 1 5 7 9 1 3 1 6 6 8 4 5 1 3 Tabl. 2 Experience 0.2751 Caution 0.7394 Flexibility 1.0845