An appearance-based measure of surface defects

Int J Mater Form (2009) 2:83–91 DOI 10.1007/s12289-009-0393-0 ORIGINAL RESEARCH An appearance-based measure of surface defects Michael Nordström & Niklas Järvstråt Received: 24 October 2008 / Accepted: 12 January 2009 / Published online: 3 February 2009 # Springer/ESAFORM 2009 Abstract Surface defects on sheet metal panels such as local out-of-plane geometric deviations, cause unwanted deviations from the intended visual appearance. In the automobile industry, these defects are commonly graded for visual acceptability through reflection analysis on the hardware. This widely used method is unfortunately subjective and thus has low repeatability. The same method can be applied on simulated sheet metal panels by the use of ray-tracing. In this paper we present an objective, appearance-based, measure by formalising the approach of subjective reflection analysis. Hence, a natural connection between an objective measure and the appearance is obtained making it easy to establish acceptable levels. The performance of this measure and three other objective measures is discussed and demonstrated for a real panel and a panel geometry obtained through sheet forming simulations. These examples illustrate that different measures yield different information about the geometry, and the proposed measure gives a useful quantification of the visual appearance deviation. Keywords Sheet metal forming . Appearance . Surface defects . Ray-trace M. Nordström (*) Saab Automobile AB, V. Bodane 290, SE-460 64 Frändefors, Sweden e-mail: [email protected] N. Järvstråt Division of Production Engineering, University West, Trollhättan, Sweden e-mail: [email protected] Introduction This paper presents a measure for detecting and quantifying surface defects on automobile sheet metal parts. Surface defects are cosmetic defects appearing as a disturbance in the reflection of painted sheet metal panels. Since the surface defects negatively affect the appearance of a car and eliminating them on hard tools is a costly process, it is important to detect and quantify the severity of surface defects as early as possible. Prior to making hard tools, in the virtual development phase, making changes to the virtual models are not as costly and by the use of Finite Element methods it is possible to detect surface defects although the accuracy is lower than on hard tools. Surface defects is the most common name in previous literature [1–5], but other names have also been used such as “surface deflections” [6–8], “surface strains” by Yoshida [9] as well as “highs and lows” or “cosmetic defects” by Sing and Konieczny [10]. Surface defects appear during manufacturing and are caused by local out-of-plane geometrical deviations of small magnitude. These deviations typically occur on flat areas in the vicinity of geometrical features, such as door handle depressions. A number of authors have specified the geometrical range of the surface defects in terms of in-plane size and out-of-plane deviation. In-plane sizes have been specified as: over 50 mm [2]; and between 50–100 mm [3]. The out-of-plane deviation has been specified as: 0.1– 0.5 mm [2]; up to 0.5 mm [3]; and up to 0.2 mm [9]. Generally, a surface defect with a short wavelength is more severe than one with a long wavelength but with equal amplitude [9]. At the tryout stage of the panel development phase, the physical defects are commonly graded using a subjective method called reflection analysis. This method will always 84 require experienced evaluators but even then the repeatability is low; it does, however, directly address the requirement for visual appearance, and reflection analysis will consequently remain the most commonly used method for detecting and grading surface defects. During the virtual panel development phase, however, the panel is only available in digital form, through the use of Finite Element Analysis. For characterising defects at this stage, several objective methods of detecting surface defects have been presented such as Andersson [4] and Kase et al [11] who presented two different 3D curvature measures and Yoshida [9] who presented a 2D curvature measure. Dutton and Pask [3] and Sing and D’Silva [12] both made virtual reflection analysis on the FEM mesh by ray-tracing. The objective methods generally calculate a local geometric value, which more or less characterizes the unwanted shape; for some of the geometric values it is even possible to see a correlation to the appearance. It is, however, hard to establish acceptability criteria of these measures since the acceptance is due to human perception of the panels’ appearance rather than down to the actual geometry. Several hundreds of panels with different geometrical features and grades of defects need to be examined in order to be able to correlate the specific geometrical value with the appearance before an acceptability criterion can be established. It is desirable to find a measure that has direct relation to the appearance, making it easier to establish the acceptability criterion for the measure. In this paper we present an appearance based measure that has the intention of resembling the process of reflection analysis made on hard on tools as described above. The measure therefore has the potential to be as objective as possible to a subjective process. The next section describes a suggested algorithm to derive objective values from the subjective reflection analysis as well as a study on how the algorithm’s parameters are chosen. Furthermore, the measure Fig. 1 Reflection analysis showing a single white line reflected on the intended panel and on the manufactured panel with different results. The deviation is the appearance based measure for this viewpoint Int J Mater Form (2009) 2:83–91 as well as three other measures are applied on a real panel scanned from hard tools as well as a simulated panel from a virtual model. This is presented in the Result section. The result is discussed in the following section where the appearance based measure is compared to the other three. No criterion for establishing if a certain value is considered a surface defect or not are given in this paper. However, a practical implementation of the measure is also discussed which also highlights the establishment of criteria. The appearance-based measure This section presents a measure based on quantifying the deviation of the appearance as seen in traditional reflection analysis. Reflection analysis consists of looking at zebra patterned reflections on the manufactured panel from several positions around a point of interest. With the intended reflections in mind, a remark is made when a deviation in the reflected zebra pattern shows up. The basic idea of the measure presented here is simply to calculate this deviation as illustrated in Fig. 1. If a real panel were available it would be possible to measure this deviation using a ruler and by estimating the location of the intended line as this line should be smooth. Assuming a general case including both real panels and virtual models, one has to calculate the deviation by using an algorithm on digital models of the panels. In short, the algorithm assumes that a single zebra line is reflected for each point on the intended panel. To calculate the deviation, the intended panel is replaced by a manufactured one and the reflection of the same zebra line is calculated. It is thus possible to calculate the deviation by superimposing these reflections. The deviation of each point is calculated for a large number of viewing angles and the largest deviation is chosen as the measure for that point. Int J Mater Form (2009) 2:83–91 85 Algorithm This section explains how the deviation of the zebra pattern reflection is calculated for an arbitrarily chosen point on the panel. The algorithm is implemented in Matlab and it is assumed that two sets of points define the surfaces of the intended and manufactured panels. The surfaces are oriented such that for each x-y coordinate pair there is only one z-value, i.e. the usual orientation of a stamping tool. It is also assumed that the manufactured surface is globally aligned with the intended one. For simplicity, the measure is first derived for a 2D section, and then it will be developed for a 3D surface. First, the point of interest, Pi, on the intended panel is chosen. The viewing point, O is defined by Pi, the surface normal, ni, the viewing angle, β, and the distance Li between O and Pi according to Fig. 2. The screen in the shape of an infinitesimal long cylinder is defined by Pi as the centre of the cylinder, and the distance L2 as the radius of the cylinder. It is now assumed that Pi is perfectly reflecting a single zebra line. This mimics a human evaluator finding a zebra line through the point, Pi, for assessing how well the reflections agree with those of the intended panel. The point, Ri, on the screen is easily calculated using the law of reflection: R i ¼ P i þ L2 ð v i 2ni ðni
vi ÞÞ ð1Þ where vi is the viewing direction of the intended panel calculated by normalizing the vector between O and Pi. The setup of the virtual reflection analysis for the point Pi is now established. Such a setup needs to be made for all points, Pi, view angles, β, and distances L1 and L2. Zebra pattern screen Ri Next the location of the zebra line reflection on the manufactured panel is established. This is made by calculating which point of the screen is reflected on the manufactured panel until the point with the zebra line is found. The point on the manufactured panel with the same x-y coordinates as Pi on the intended panel is chosen as a ð1Þ starting point, Pm . The point Rm on the screen is calculated by solving the non-linear equation defined by the equation of the screen radius and the equation of the line between ð1Þ Pm and Rm: Rm ðPm þ srm Þ ¼ 0 ðR m Pi Þ2 L22 ¼ 0 where the scale factor s is the unknown and rm is derived using the law of reflection. The location of Ri and Rm is now known. In order to have a convenient way of comparing these two, the angle difference (with a sign) between the points as seen from Pi is calculated. If this angle is within a tolerance, in this ð1Þ is paper, 0.001 degrees is chosen, the point Pi and Pm considered to reflect the same point on the screen. If the difference is not within the tolerance, Rm is calculated for ð2Þ the next point, Pm , on the section. In this manner the section is traversed (in both directions) until the angle difference between R i and R m for the new point ðkÞ Pm changes signs which indicates that the zebra line on the screen has been passed as illustrated in Fig. 3. A simple ðintÞ linear interpolation of the point Pm that should have reflected the white line is then made. The density of ðintÞ available points will affect the accuracy of Pm but it will also affect the computation time. It is therefore important to Zebra pattern screen Zebra line Ri Zebra line L2 R (1) m ni O β L1 β Intended panel O Manufactured panel Pi (k) P(k-1) Pm z x Fig. 2 Reflection analysis setup looking from viewpoint O towards the intended panel at point Pi with normal ni at distance L1 and angle β, the zebra line at position Ri on the zebra pattern screen is reflected in point Pi at distance L2 from Ri at an angle β m (1) Pm z x Fig. 3 The same reflection analysis setup as in Fig. 2 but here the manufactured panel is first viewed at a starting point Pð1Þ m and then the panel is traversed until the zebra line defined by the intended panel is ðk 1Þ reflected at the point between PðkÞ m and Pm 86 Int J Mater Form (2009) 2:83–91 choose a density of points giving enough accuracy with a ðintÞ reasonable computation time. When the correct point Pm has been found the algorithm calculates the deviation of the zebra line as the deviation seen in the current view position as illustrated in Fig. 4. Other deviation measures could be used, e.g. the ðintÞ Euclidian distance between Pi and Pm , but the choice made here is motivated by the fact that when viewing a small deviation from some distance properly judging the ðintÞ depth deviation between Pi and Pm seems difficult. The section is traversed in both directions simultaneously as the direction the zebra line is deviating cannot be ð1Þ known since the point Pm does not contain information about position and normal of neighbouring points. The algorithm will choose the first encountered point with a zebra line and stop. The reason for this is that it is assumed that the geometric defect is a small one giving only one reflection of the zebra line, compared to e.g. wrinkles where there will probably be several reflections from the same zebra line. This means that the measure will not detect wrinkles properly, but this is not the objective and they are in any case easily detected by other measures. For a 3D surface the measure is simply calculated by cutting 2-D sections of the panel in several view directions, α, and then calculating the measure for each 2D section. For each point, we can then define all view positions by the angles α and β and the view distance, L1, similar to a spherical coordinate system. Each chosen point on the panel will thus have one value of zebra pattern deviation for each section cut. To mimic that the examiner is trying to find the worst deviation of the zebra line when going around the panel, we will choose the highest zebra pattern deviation value for each point among all section cuts. reflection from all view directions α and view angles β and distance L1 to detect the surface defect and grade the severity of it on the fixed panel. In order to have a reasonable computation time, the parameters L1 and L2 are set to 1 m and 2 m respectively describing a certain reflection analysis setup and process. Setting L1 and L2 to other values, would generate a different result, therefore it is important to set L1 and L2 at values resembling the companies’ own reflection analysis setup. In order to choose β, a simple artificial defect is designed by fitting a polynomial to some basic boundary conditions, making a rotationally symmetric, continuous curvature defect with amplitude A and wavelength W. An illustration of the reflections caused by the defect is shown in Fig. 5a. Figure 5b shows a plot of the appearance based measure using a β value of 90 degrees and the above chosen values of L1 and L2. For simplicity, α is set to 0 degrees, i.e looking only in one view direction. Figure 6 shows the calculated maximum measure of the artificial defect using the presented values, L1, L2 and α. The defect has a constant wavelength of 60 mm but different amplitude values and the maximum deviation is plotted as a function of β. As seen, the biggest deviation is reached when viewing the point perpendicular to the surface for all amplitudes. Experimental data from a real panel obtained by scanning is used to prove this further. The scanned data was aligned with the intended geometry using an algorithm based on Besl and McKay [13] which was implemented in Matlab. Smoothing is applied to the calculation of surface normals. Figure 7a and 7b show ray-traced images of a reflection analysis setup for the intended and manufactured panels, respectively. Figure 7c shows a plot of the appearance based measure using a β value of 90 degrees and as found for the simple defect an α value of 0 degrees Parameter choice The previous section described the principles of calculating the measure. This section describes how parameters, α, β, L1 and L2 are chosen. These parameters are dependent on the actual setup and evaluation process of the reflection analysis. In reflection analysis of the physical panel, the evaluator goes around the point of interest checking the  P(int) + P  i  m O 2 Pi γ Pm(int) Deviation Fig. 4 Illustration of how the deviation between two reflected zebra lines is calculated. The dashed grey line represents the intended panel and the solid grey line represents the manufactured panel. Pi is the ðintÞ reflected point on the intended panel. Pm is the reflected point on the manufactured panel. Fig. 5 a shows a raytraced image of a reflection analysis of an artificial rotation symmetric curvature continuous defect with amplitude 0.05 mm and a wavelength 60 mm; b shows the plot obtained when the appearance based measure is applied on the same defect. The reflection setup uses the values L1 =1 m, L2 =2 m, β=90 degrees and α=0 degrees Int J Mater Form (2009) 2:83–91 87 Fig. 6 The diagram shows the maximum zebra pattern deviation calculated for the artificial defect as a function of the view angle β for different defect amplitudes is used. The white area in the middle of the panel is where a depression is located. This depression, as well as geometrical features outside the outer boundary, are causing appearance problems. The depression is cutaway because it is in this case assumed not interesting, referring to many cases of depressions on automobile sheet metal panels that are covered by other parts, e.g. glass parts such as rear lights. Three areas of the panel as indicated in Fig. 7c are extracted and the measure is calculated in these areas as a function of β. The result is presented in Fig. 8 and it is obvious that the same conclusion can be drawn on a real panel. The biggest deviation is obtained by looking perpendicular, i.e. β=90 degrees, to the surface using the presented method of deviation calculation. Up until now, the view direction α has been set to 0 degrees. In order to fully reflect that the examiner is looking in several directions and choosing the maximum deviation of each point and direction, we need to choose appropriate values of α. To save calculation time, the measure is only calculated for every 15 degrees of α up until 165 degrees, which is considered to provide an adequate angular sampling. Obviously, identical results would be obtained for sample angles α+180°, dispensing of the need to sample more than the first 180°. Boundary conditions Fig. 7 a raytraced image of a reflection analysis of the intended appearance of a real panel; b raytraced image of a reflection analysis on a real panel obtained by scanning; c the plot obtained by applying the appearance-based measure on the same data. The reflection setup uses the values L1 =1 m, L2 =2 m, β=90 degrees and α=0 degrees Since only visible points are of interest, there will be cases where the algorithm encounters an outer boundary or an inner boundary. If a boundary is encountered when the section is traversed, it is assumed the boundary is reflecting the zebra line. However, before the algorithm is stopped, the opposite direction is traversed until a possible zebra line is encountered. If one is encountered, this is the one that is chosen. However, if another boundary is encountered, the algorithm will choose the point producing the smallest deviation. Comparison to other measures applied on a real panel and a simulated one The real panel as previously shown in Fig. 7 will be used for making comparisons with other objective measures. Figure 9 illustrates the intended reflections, the reflections of the real panel and the reflections of a simulated equivalent panel. The simulation was made in the finite element software Autoform. The blank draw-in of the simulation was correlated to the real panel using the friction coefficient. The appearance based measure will be applied 88 Int J Mater Form (2009) 2:83–91 Fig. 8 The maximum zebra pattern deviation on the real panel as a function of the view angle β for three areas indicated in Fig. 7 Fig. 9 Raytraces of reflection analysis setup and appearance based measure plots for the α directions 0, 45, 90 and 135 degrees going from left to right, a–d raytrace of the intended panel, e–h raytrace of the real panel, i–l raytrace of the simulated panel Int J Mater Form (2009) 2:83–91 to both the real panel and the simulated one using the setup described in section 2 with α every 15 degrees, β 90 degrees, L1 1 m and L2 2 m. The point density for both the intended and manufactured panel is one point per square mm which is estimated to give a good enough accuracy. The other measures are coordinate deviation (in z-direction) between the intended and manufactured panels, the maximum slope deviation between the intended and manufactured panels and the mean curvature in each point. For the scanned data of the real panel, smoothing is applied on surface normal, slope and curvature values. Figure 10a–d show the plots for the real panel and Fig. 11a–d show the measures for the simulated panel. Discussion A strong correlation between the measure and the appearance can be seen when comparing the appearance based measure in Fig. 10a with the raytraces in Fig. 9. The same strong correlation can be seen on the simulated panel in Fig. 11a compared to Fig. 9. Both the features around the inner boundary and the outer boundary seen in the ray-traces are captured well in the appearance based measure. It can also be noted that there are similarities between the appearance of the real panel and the simulated one, particularly around the Fig. 10 Four different measures applied on a real panel: a Appearance based measure, using the values L1 =1 m, L2 =2 m, β= 90 degrees for the reflection setup and α sampled at 15 degree intervals; b coordinate deviation between manufactured and intended panels; c maximum slope deviation between manufactured and intended panel; d mean curvature on a real panel 89 inner boundary. However, although one cannot say that the correlation between the real panel and the simulated one is good, the agreement seems much better from the raytraced images than from the geometrical measures (Figs. 10b–d and 11b–d). Indeed, only the appearance based measure provides any information about where in the panel that the simulation deviates from real panel. Looking at the coordinate deviation plots of Figs. 10b and 11b it is hard to find any correlation to the appearance in Fig. 9. The maximum slope deviation on the other hand shows some correlation and there is a similarity to the appearance based measure. This is logical since the appearance based measure uses the normal of the surface to calculate the measure. The curvature plot of the real panel in Fig. 10d is an example of how sensitive the curvature, which is a second derivative of the geometry, is to poor geometry scanning. Looking instead at Fig. 11d, it can be seen that there is some correlation between the appearance and the curvature of the simulated panel. The coordinate deviation, maximum slope deviation and curvature plot measures are all local values of the geometry or geometry difference. Thus, the measures do not take into account the whole defect. They can of course be a measure of the severity of the defect in a local point within the defect but in order to establish the extent of the defect one have to study neighbouring points in a plot for example. The appearance 90 Int J Mater Form (2009) 2:83–91 Fig. 11 Four different measures applied on a simulated panel: a Appearance based measure, using the values L1 =1 m, L2 =2 m, β=90 degrees for the reflection setup and α sampled at 15 degree intervals; b coordinate deviation between manufactured and intended panels; c maximum slope deviation between manufactured and intended panel; d mean curvature on a simulated panel based measure on the other hand gives information on the extent of the defect without having to study neighbouring points. This is already included in the measure. Regardless of algorithm used, the appearance based method is basically a method of extracting the reflected line deviation as an objective number using the same process as traditional reflection analysis which is a subjective but much used method. The measure could practically be implemented by studying the current process of traditional reflection analysis, either a general one or a company specific one, and then adapting the view distance (L1) and the reflective screen distance (L2) of the algorithm. Ideally, this would lead to a situation where it would be possible to measure a deviation on a real panel using a ruler and calculating the same measure on the scanned panel. This means that instead of giving the defect a subjective grade, an objective number is established. Since, the objective number is a number of what you see using traditional reflection analysis; it should be fairly easy to establish criteria for acceptable objective numbers by interview of the quality inspectors of the real panels, if not already established. Such a system would be very beneficial for use on virtual models where the defects are predicted using the Finite Element method as well as on real panels since the measure is independent on the subjective quality inspector. As described earlier, in traditional reflection analysis different grades are given to the defects depending on their severity, but depending on the location of the defect on the automobile different grades are considered acceptable. For instance, areas that the customer is likely to see, such as around the door handle, is more important than areas which the customer is less likely to see, such as the lower areas of the door or on secondary surfaces such as ones made visible when opening the trunk. This process is also easily resembled by dividing the points into the different zones that apply and thus giving them different criteria. The behaviour of the panels when experiencing a large global springback, such as big twisting or bending of the whole panel, has not been tested in this study. The appearance based measure is likely to work at its best if the springback of the panel is only local, i.e. we only have surface defects and low global springback. With a big global springback, the deviation of the zebra lines will probably be so big that the boundary of the panel is reached in many areas. A solution to this problem could be to make a local alignment of the CADmodel and the FEM mesh for areas or points of specific interest instead of a global alignment. Then, the zebra pattern deviation would tend to be of a more local character. Conclusion & further work An appearance based measure was presented and applied to two examples. The measure was shown to perform well on Int J Mater Form (2009) 2:83–91 both a scanned and a simulated panel. It was also shown that the curvature, at least on the simulated panel, shows promising correlation to the appearance as do the slope deviation. With the appearance based measure however, a non-local objective measure is obtained where the units of the measure is mm deviation of a reflection, i.e. for each point on the panel we can calculate how much a reflected line would deviate due to a defect much like the work of a quality inspector using traditional reflection analysis. It is equally applicable to both real panels (scanned) and simulated ones (Finite Element mesh) but the big benefit will be on the latter, especially in the virtual development phase. A selection of industrial examples should be examined to study the behaviour of the appearance based measure for different types of surface defects and geometries. It would be particularly interesting to assess the performance also for less severe defects, since the panels in this study all had several severe surface defects. Further, an acceptability criterion using the measure needs to be established according to the specific quality requirements of each automobile company. The styling and quality engineers can establish these levels by investigating their grading procedure during the reflection analysis and deciding on an acceptable zebra pattern deviation. Thus having established an acceptability criterion using the suggested measure, it would be straightforward to assess surface quality without variations due to the human factor, and perhaps even to perform fully automated surface quality acceptance tests. A variant of the presented measure that should also be investigated is to calculate the curvature of the zebra pattern on the manufactured panel. During reflection analysis, the quality inspector is also looking for disturbances of the reflected zebra pattern so instead of calculating the deviation of the zebra pattern we could calculate the amount of curvature of the reflected zebra line. Such a measure would be more sensitive to appearance deviations, but it would 91 probably not be as amenable for establishing acceptability criteria. References 1. Mattiasson K (2000) On finite element simulation of sheet metal forming processes in industry. In: Eccomas 2000, Barcelona 2. Kolodziejski J, Assempoor A, Liu S (1994) An approach for designing out surface defects in metal stampings. SAE Technical Pap Ser 940750 3. Dutton T, Pask E (1998) Simulation of surface defects in sheet metal panels. In: Simul of Mater Process: Theory, Methods and Applications, Huétink & Baaijens, Rotterdam, pp 869–874 4. Andersson A (2005) Evaluation and visualization of surface defects—a numerical and experimental study on sheet metal parts. In: Numisheet 2005 Conference proceedings, AIP, pp 113–118 5. Qiu Y, Konieczny A, Sing B (1997) An integration of One-step FEM global analysis with incremental FEM local analysis of surface defects of auto body panels. In: IBEC ’97 Adv Body Concept and Dev, pp 7–11 6. Sawada T, Liu L (1998) Proposal for analysis of surface deflections of sheet metal in embossing. In: Simul of Mater Process: Theory, Methods and Applications, Huétink & Baaijens, Rotterdam, pp 863–868 7. Hayashi H (1996) The control of shape fixation and springback of aluminium alloy sheet in model forming. SAE Technical Pap Ser 960819 8. Liu L, Sawada T, Sakamoto M (2000) Evaluation of the surface deflections in pressed automobile panels by an optical reflection method. J Mater Process Technology 103:280–287 9. Yoshida K (1993) Handbook of ease or difficulty in press formings. National Center for Manufacturing Sciences, Michigan, USA 10. Sing B, Konieczny A (1997) Changing of stamping process design and practice due to integrated design approach to car body quality enhancement. SAE Technical Pap Ser 970983 11. Kase K, Makinouchi A, Nakagawa T, Suzuki H, Kimura F (1999) Shape error evaluation of free-form surfaces. Comp Aided Des 31:495–505 12. Singh B, D’Silva D (1998) A virtual environment to develop car body quality. SAE Technical Pap Ser 982369 13. Besl PJ, McKay ND (1992) A Method for Registration of 3D Shapes. IEEE Trans on Pattern Analysis and Machine Intell 14.2:239–256