Foot Sole Scanning by Laser Metrology and Computer Algorithms

In recent years, the foot sole anatomy has been used in applications such as footwear manufacturing, person identification, foot deformation and health care (Xiong S., et. al 2010). Typically, the foot sole shape is used to make profitable shoes (Lee Au E. Y., et. al 2007). Nowadays, several researches have been developed in computer vision to detect the foot sole. Methods such as multiple camera, photogrametry, laser trinagulation, fringe projection habe been used to determine the foot sole. The ligthings methods, compute the surface of foot sole based on the calibration via perspective projection. Thus, the calibration provides the vision parameters to compute the object surface. These lighting methods provide a smoothed surface of the foot sole and some details are not profiled (Chen M. J. L. et. al, 2003). Furthermore, these systems do not provide the fitting of the shoe sole mould to the foot sole. The laser line technique can be used to improve these matters. In this technique, the three-dimensional surface is computed based on laser triangulation and the calibration via perspective projection model (Zhou F., et. al, 2005). The calibration includes the measurement of the distances of the setup geometry, focal distance, camera orientation, pixel scale and image centre. Typically, the calibration is performed by an external procedure to the vision system. This procedure is carried out by the recognition of the calibrated references. Typically, the setup geometry is modified online. In this case, a recalibration is necessary to compute the three-dimensional surface (Wang G. et. al, 2004). But, the calibration references do not exist online and the calibration should be repeated. In this case, the perspective projection does not provide online re-calibration.


Introduction
In recent years, the foot sole anatomy has been used in applications such as footwear manufacturing, person identification, foot deformation and health care (Xiong S., et. al 2010). Typically, the foot sole shape is used to make profitable shoes (Lee Au E. Y., et. al 2007). Nowadays, several researches have been developed in computer vision to detect the foot sole. Methods such as multiple camera, photogrametry, laser trinagulation, fringe projection habe been used to determine the foot sole. The ligthings methods, compute the surface of foot sole based on the calibration via perspective projection. Thus, the calibration provides the vision parameters to compute the object surface. These lighting methods provide a smoothed surface of the foot sole and some details are not profiled (Chen M. J. L. et. al, 2003). Furthermore, these systems do not provide the fitting of the shoe sole mould to the foot sole. The laser line technique can be used to improve these matters. In this technique, the three-dimensional surface is computed based on laser triangulation and the calibration via perspective projection model (Zhou F., et. al, 2005). The calibration includes the measurement of the distances of the setup geometry, focal distance, camera orientation, pixel scale and image centre. Typically, the calibration is performed by an external procedure to the vision system. This procedure is carried out by the recognition of the calibrated references. Typically, the setup geometry is modified online. In this case, a recalibration is necessary to compute the three-dimensional surface (Wang G. et. al, 2004). But, the calibration references do not exist online and the calibration should be repeated. In this case, the perspective projection does not provide online re-calibration.
In the proposed chapter, a review of our methods for three-vision are applied to determine the foot sole. The proposed vision system moved in x-axis to perform the scanning. Here, an automatic re-calibration is performed to overcome limitations caused by online geometric modifications and physical measurements. This technique, retrieves the foot sole by laser line metrology and Bezier networks. Also, the modifications on the setup geometry are recalibrated based on the data provided by the network. Thus, the accuracy and the performance are improved. The vision system is implemented based on laser line to scann the foot sole. Here, the three-dimensional model is performed by approximation networks based on behavior of the line position (Muñoz Rodríguez J.A, et. al, 2007). Also, the extrinsic and intrinsic parameters are computed based on the data provided by the network. Thus, setup modifications are re-calibrated during the visión task and external procedures are avoided. In this manner, limitations caused by online modification are avoided. Thus, a contribution is achieved in the field calibration of vision parameters for foot sole scanning. This technique is evaluated respect to the perspective projection model, which is used in laser triangulation for foot sole profiling. The error analysis is based on the root mean squared of error using references of a contact method. This technique is applied to fitting the shoe last to the foot sole. This procedure is carried out by Bezier curves using the foot sole data, which are outside of the contour of the shoe last. Also, foot length, foot width, heel width, mid foot width, arch length and arch angle are measured to achieve the fitting of the shoe last. In this manner, a profitable shoe last is achieved for footwear manufacturing. Thus, a contribution is achieved by the vision system on this field. Also, the time processing is short to obtain the foot sole topography.

Basic description of vision system
The proposed vision system includes an electromechanical device, a glass window, a CCD camera, a laser line projector and a computer. Fig. 1 shows the experimental setup to perform the foot sole scanning. In this setup, the laser line is projected perpendicularly to the glass window. Here, the laser line pass by the glass window and then it is projected on the foot sole. Then, the CCD camera captures the projected line on the foot sole. The camera and the laser diode are mounted on a linear slide, which can be moved in x-axis, y-axis and z-axis. To perform the scanning, the laser and the camera are moved in x-axis. From this scanning, a Bezier network computes the foot sole topography based on the position of laser line. Also, the three-dimensional modeling and the calibration are performed by means of the network. This modeling is performed based on the setup geometry shown in Fig.2. In this geometry, the x-axis and y-axis are located on the glass window, which correspond to the reference plane. The z-axis is located perpendicularly to the reference plane. The focal length f is the distance between the lens and the image plane. The center of image plane is indicated by x c in the x-axis. The distance between the laser line and the optical axis is indicated by  a . In the z-axis, the foot sole depth is indicated by h(x, y). The distance between the lens and the foot surface is defined by z i =D+h(x,y). In the proposed modeling, the surface depth is computed based on the position of laser line (Muñoz Rodríguez J.A, et. al, 2003). When the laser line is projected on the point A of the reference plane, the line position in the image plane is x A . When the laser line is projected on the point B of the foot sole, the line position is shifted to the position x B . This line shifting is described by next expression Based on the camera position, the shifting s i is directly proportional to the foot sole depth h(x, y). This relationship is performed by a Bezier network to compute the surface depth. The network to compute the foot sole depth is described in section 3. To compute the shifting, the line position x B and x A are detected in the image. This procedure is performed by measuring the intensity maximum in each row of the image. To carry it out, the pixels are approximated to a continuous function. Then, first and second derivative are computed to obtain the maximum. In this case, the pixels are represented by (x 0 , I 0 ), (x 1 , I 1 ),......, (x n , I n ), where x i is the pixel position, I i is the pixel intensity and n is the pixel number. Bezier curves are used to fit a continuous function from the pixel intensity. The Bezier curves (Mortenson M. E., 1997) are is described by By applying the definition of Eq.
(2), two equations are obtained, one for x and one for I: Eq.
(3) represents the pixel position and Eq.(4) represents the pixel intensity. Based on these equations, a continuous function is fitted from the pixels a laser line image. To carry it out, the pixel position x 0 , x 1 , x 2 ,......,x n , are substituted into Eq. (3) and the pixel intensity I 0 , I 1 , I 2 ,....,I n , are substituted into Eq.(4). These two equations are evaluated in the interval 0 u1 and the result of this curve is a concave function. Therefore, the second derivative I''(u) is positive and the peak is a maximum global. In this manner, the maximum of I(u) is www.intechopen.com Laser Scanner Technology 4 computed based on the first derivative I'(u)=0 (Frederick H., et. al, 1982). To find the derivative I'(u) = 0, the bisection method is applied. The Bezier function is defined in the interval 0u1. Therefore, the initial values are u i =0 and u f =1. Then, the middle point is computed by u*=(u i +u f )/2 to find u that converges to the I´(u)=0. Then, the first derivative The next middle point u* is obtained from the last pair of values u i and u f . These steps are repeated until to find I´(u)=0 based on a tolerance value. The value u=u* where I´(u) = 0 is substituted into Eq.
(3) to determine the position of the intensity maximum x(u).

Bezier network of a laser line
The network to perform the three-dimensional modeling includes an input vector, two parametric inputs, a hidden layer and an output layer. Each layer of this network is performed as follow. The input includes: the surface depth h 0 , h 1 , h 2 ,...,h n , the line shifting s i , and parametric values (u ,v). The value u is proportional to the shifting s i according to the camera position in x-axis. The line shifting s i is obtained by the procedure described in section 2. This shifting is represented by a value u, which is obtained by the next expresion. 01 , ua a s   where a 0 and a 1 are constant to be determined. By means of two values s and its respective value u, Eq.(5) is determined. The Bezier curves are defined in the interval 0  u  1. Therefore, u=0 for the first line shifting and u=1 for the last line shifting s n . Substituting these values in Eq.(5), two equation with two unknown constants are obtained. Solving these equations, a 0 and a 1 are determined. Thus, for each shifting s i , a value u is computed via Eq.(5). The value v is proportional to the coordinate y in y-axis. This relationship is described by the next expression where b 0 and b 1 are the unknown constants. Using two values y and its respective v from Bezier curves, Eq.(6) is determined. Bezier curves are defined in the interval 0v1. Therefore, v=0 for the first y and v=1 for the last y. Substituting these two values y and its respective v in Eq.(6), two equation with two unknown constants are obtained. Solving these equations, b 0 and b 1 are determined. Thus, for each coordinate y, a value v is computed via Eq.(6). The hidden layer is built by a Bezier basis function, which is described by The output layer is obtained by the summation of the neurons, which are multiplied by a weight. Thus, the output response is the surface dimension given by next expression where w ij are the weights, h i is the known object surface, B i (u) and B j (v) are the Bezier basis function represented by Eq. (7). To construct the complete network Eq. (8), the suitable weights w ij should be determined. To carry it out, the network is being forced to produce the correct surface depth h(x,y) by means of an adjustment mechanism. This procedure is performed based on the shifting s i produced by the known depth h i . The coordinate y is the pixel number in the image of the vertical laser line. Then, the shifting s i is converted to a value u via Eq. (5)  Based on this output, a linear system is obtained and Eq.(9) is solved. In this manner, the weights w ij are determined. Thus, the Bezier network H (u, v) has been completed. The result of this network is a model that computes the foot sole depth h(x,y) based on a line shifting s i . Fig.  3(a) shows a projected laser line on a foot sole mould. Here, the network is applied to obtain a profile of the mould of foot sole. To carry it out, the shifting s i is detected in each image row, which corresponds to the coordinate y. Then, the shifting s i is converted to a value u and the coordinate y is converted to the value v. Then, these values are substituted in the network Eq.(9) to compute the surface depth shown in Fig. 3(b). To know the accuracy, the data provided by the network are compared with the data provided by a contact method. The accuracy is computed based on a root means squared error (rms) (Masters T., 1993) where ho i is the data provided by a coordinate measure machine (CMM), hc i is the calculated data by the network and n is the number of data. For the data shown in Fig. 3(b), the error is a rms=0.118 mm. The depth resolution is provided by the network based on the minimum line shifting. In this setup, the magnitude of s i is based on the camera position. Therefore, the depth resolution is defined as the minimum s i based on the distance  a . In this case, the network is built using the minimum s i and the maximum s i at distance  a of the camera position. For this configuration, a shifting s i =0.38 pixels is detected from the reference plane. To compute the surface depth, this s i is substituted in the Eq.(8) and the result is h =0.28 mm.

Calibration of vision parameters
In the lighting methods, the object surface is computed based on the calibration of the vision parameters. This procedure is performed based on the perspective projection model (Mclvor A. M., 2002). Thus, the distances of the setup geometry, focal distance, camera orientation, pixel scale and image centre are determined. The calibration is carried out by means of a transformation matrix and calibrated references. This calibration is performed by the transformation P w =(x w , y w , z w ) to the camera coordinates P c =(x c , y c , z c ) by P c = RPw+t. Where R is a rotation matrix and t is a translation vector. Here, the transformation of P c to the image coordinates (X u ,Y u ) is given by X u =fx c /z c and Y u =fy c /z c . Considering radial distortion, the image coordinates are represented by X d +D x =X u and Y d +D y =Y u . Where D x =X d ( 1 r 2 + 2 r 4 +…), D y =Y d ( 1 r 2 + 2 r 4 +…) and r=(X d 2 +Y d 2 ) 1/2 . In these expressions, X d and Y d are the distorted image coordinates. Also, these image coordinates in pixels should be converted to real coordinates by means a scale factor. Thus, the parameters to be calibrated are the matrix R, the vector t, the focal distance f, the distortion coefficient  i , the image center (c x , c y ) and the scale factor. This procedure is carried out by using calibrated references such as rectangles or circles (Zhou F., et. al, 2005 ). In this step, coordinates of the calibrated references are detected. Then, the vision parameters are passed to the vision system to compute the object surface. The setup modifications play an important role to get good sensibility. Due to these modifications, a re-calibration is necessary to compute the three-dimensional shape (Wang G., et. al, 2004). But, the calibration references do not exist for online measurements. Therefore, an online re-calibration is not possible. In this case, the calibration is repeated again via calibrated references. Therefore, an automatic re-calibration is necessary to overcome limitations caused by online geometric modifications.
In the proposed setup, camera is placed around the central axis of the foot to perform the scanning. Here, the electromechanical device provides distance modification between the surface and the optical devices. When the vision system is moved in x-axis or y-axis, the setup geometry is not modified. But, when the vision system is moved in z-axis, the setup geometry is modified. Here, a re-calibration is performed based on the data provided by the network (Muñoz Rodríguez J. A., 2011). The camera orientation is described based on the geometry shown in Fig. 4. In this geometry, the shifting s 1 , s 2 ,…, s n correspond to the depth h 1 , h 2 ,… h n , respectively. The distance between the lens and the surface depth is indicated by z i =D +h i . In this case, the optical axis is perpendicular to the reference plane. Thus, the depth h i has a projection k i in the reference plane. In this manner, the relation (h Therefore, the projection k i Eq.(11) is rewritten by The camera orientation in y-axis is performed based on the geometry of Fig. 5. In this case, a surface pattern in the laser line is moved in steps y i according to the depth h i in the y-axis. Thus, the shifting pattern is computed by t i =(y A -y i )=(y A -y c )-(y i -y c ). Based on this geometry, the next expression is obtained by triangulation Considering radial distortion, Y A = y A +  A and Y i = y i +  i , where y is the distorted image coordinate and  is the distortion. In this case,  For the camera orientation in x-axis, k i is computed via Eq.(11) using the surface h i provided by the network based on s i . Due to the distortion, the derivative dk/ds is not exactly a constant. But, this derivative is the more similar to a constant. For the camera orientation in y-axis, k i is computed via Eq.(13). Also, the derivative dq/dt is not exactly a constant. But, this derivative is the more similar to a constant. Therefore, the camera orientation perpendicular to reference plane is defined when the dk/ds and dq/dt are very similar to constant according to tolerance. To compute dku/ds=constant, the derivative of first projection k 1 is used. Thus, dku i /ds = dk 1 /ds = constant for i=1,2,…n and ku i =i*k 1 . The first derivative of k i is selected because the distortion is observed in the last pixels of the image. Thus, using the terms from Eq.(11) and Eq.(12), the next expression is obtained From Eq.(15), the distortion  B can be computed by the next expression From the terms In this manner, the orientation and distortion have been deduced. Based on a perpendicular optical axis, the vision parameters are deduced. This procedure is carried out based on S i = s i +( A - B ) and the setup geometry Fig.4, which is described by The values T 1 , T 2 , Y 1 , Y 2 and Y 3 are collected from the camera orientation in y-axis. These values are substituted in Eq.(20) to solve the equation system and the value y c is determined. In this manner, the vision parameters are determined based on the data provided by network and image processing.
Typically, occlusions appear in the scanning of the foot sole. These occlusions are shown in the reported results by lighting methods (Cortizo E., et. al 2003). To avoid the occlusions, the camera position should be moved away from the surface in z-axis or toward laser line in x-axis. When an occlusion appears, the electromechanical device moves the camera away from the surface in z-axis. In this case, initial geometry has been modified and the displacement magnitude is different from the initial configuration. Therefore, the displacement should be converted to a displacement of the initial configuration. This procedure is carried out by the calibration of vision parameters via Eq. (19). Thus, the vision system provides online geometric modifications to overcome occlusions. The accuracy provided by the network and by the perspective projection is shown in section 5.

Results of foot sole scanning
The proposed foot sole scanning is performed based on Bezier networks without physical measurements. In the setup, the scanning is carried out a 15 mW laser line in x-axis in steps of 1.69 mm. The laser line is captured and digitized in 256 gray levels. From each image, the network computes the surface depth based on the line shifting s i . The depth resolution provided by the network is around of 0.28 mm. Thus, the foot sole is reconstructed by the network. The foot sole to be profiled by the vision system is the shown in Fig.6(a). To carry it out, the foot sole is scanned by a laser line. In this step, a set of images is captured by the CCD camera. From each image, the network computes the surface depth. This procedure is performed detecting line shifting via Eq.(1). Then, the shifting s i is converted to a value u via Eq.(5). Also, the coordinate y of the laser line is converted to a value v via Eq.(6). Then, the values (u, v) are substituted in Eq.(9) to obtain the depth H (u,v). Thus, the network produces a transverse section of the foot sole. Then, all transverse sections are stored in array memory to obtain the shape of the foot sole shown in Fig. 6(b). The scale of this figure is in mm. One hundred and forty six images are processed to obtain the whole foot sole. In this case, the number of data to compute the accuracy was n=490 and the error is a rms=0.137 mm. The morphological parameters that provide the model of a foot sole are the foot length, foot width, heel width, mid foot width, arch height and the arch angle (San-Tsung B. Y., Zhang M., Fan Y. B., Boone D. A., 2003). Therefore, these parameters should be computed from the foot sole to perform the fitting to the shoe mould. To compute these parameters, the contour of the foot sole shown in Fig. 7(a) is extracted via edge detection. The foot length is obtained by detecting the distance between the heel point to the maximum value of the contour in the direction of x-axis. Thus, the foot length is computed by the distance = x e -x b =247.8136 mm. The foot width is the breadth computed along the foot sole in y-axis. This distance is computed by detecting the maximum and minimum value in of the contour in y-axis. Thus, the foot width = wy b -wy e = 102.4685 mm. The mid foot width is the breadth of the foot sole at ½ of the length foot in x-axis. Therefore, the mid foot width is computed by detecting the coordinates of the two points of the contour in y-axis. Thus, mid foot width = my bmy e = 67.3522 mm. The heel width is the breadth of the foot sole at 1/6 of the foot length in x-axis. Therefore, in this position, the heel width is computed by detecting the coordinates of the two points of the contour in y-axis. Thus, heel width = hy bhy e =66.0853 mm. The arch height is the distance from the reference plane to the apical arch in z-axis. Therefore, arch height is computed by detecting the maximum distance in z-axis. Thus, the arch height =Az e -Az b =14.1472 mm, which is the shown in Fig.7(b). The arch angle is the angle formed between the apical arch and the reference plane in z-axis. This angle is computed based on geometry shown in Fig.7(b). Thus, arch angle is computed as  = tan -1 [(Az e -Az b )/(Ay e -Ay b )] = 14.7816.
The accuracy of the morphological parameters of the foot sole is around 0.1 mm. Typically, the accuracy required in foot sole measurement is 1 mm (Cortizo E., et. al, 2003). The accuracy provided by the Bezier network for the three-dimensional vision of the foot sole is around of a rms= 0.130 mm. This means an error minor than 1%. This result is good according to the reported results in the recent years (Lee Au E. Y., et. al, 2007).
The approach of this technique is to fitting a shoe last to the foot sole. This procedure is carried out by adjusting the shoe last to the plantar contour. The shoe last to be fitted is shown in Fig. 8(a). To carry it out, the shoe last is scanned in the vision system and a set of images is captured. The network computes the shoe last depth from each image. Thus, the network produces a transverse section of the shoe sole from each line image. Then, all transverse sections are stored in array memory to obtain the shoe sole shown in Fig. 8(b). The scale of this figure is in mm. The accuracy computed based on the error for the shoe sole mould is a rms=0.121 mm. Secondly, the morphologic parameters of the foot sole are compared with the parameters of the contour of shoe last. In this case, the parameters of the shoe mould are different to parameters of the foot sole. Therefore, the contour of the shoe last should be modified. To carry it out, the contours of the foot sole and the shoe last are overlapped. The procedure of this overlapping is shown in Fig.9(a). In this figure, the dot line corresponds to the foot sole contour and the continuous line corresponds to the shoe last. Thirdly, Bezier curves Eq.(3) and Eq.(4) are applied to perform the contour fitting. This procedure is carried out using the foot sole points, which are outside of the shoe mould contour. In this case, x(u) corresponds to the coordinate in x-axis and I(u) = y(u) corresponds to the coordinate in y-axis. Thus, the coordinates x are substituted into Eq.(3) and the coordinates y are substituted into Eq.(4).

www.intechopen.com
Foot Sole Scanning by Laser Metrology and Computer Algorithms

13
These two equations are evaluated in the interval 0 u1 to fit the shoe sole mould. The result of the fitted contour of the shoe sole mould is shown in Fig. 9 (b). In this manner, the fitting of the shoe sole mould has been completed. The employed computer in this vision system is a PC to 1.8 GHz. The capture velocity of the camera is 34 fps. Also, the electromechanical device is moved at 34 steps per second. Each image of laser line is processed by the network in 0.010 sec. The shape of the foot sole is profiled in 5.75 sec. The time processing of the proposed system provides a fast method to perform the three-dimensional vision of the foot sole. In this procedure, distances of the geometry of the setup are not used to compute the surface depth. Therefore, the proposed reconstruction of the foot sole is easier than the reported techniques. The reported are based on the laser triangulation and perspective projection techniques and the vision parameters are collected by an external procedure. In this manner, the proposed technique is performed automatically by Bezier networks and the measurements on optical setup are avoided.

Conclusions
A technique for three-dimensional vision of the foot sole based on Bezier networks has been presented. The described technique here provides a valuable tool for the footwear manufacturing. In this procedure, a Bezier network provides the measurement of the foot sole and the vision parameters. This system performs the fitting of the shoe sole mould to the foot sole via Bezier curves. The automatic technique avoids the physical measurements on the setup. Also the proposed network provides the data to compute the geometric modifications due to the mobile setup. This procedure improves the performance and the accuracy of the results. It is because measurement errors are not passed to the vision system. The ability to detect the laser line with a sub-pixel resolution has been achieved by Bezier curves. This procedure is achieved with few operations. In this computational-optical setup, a good repeatability is achieved in each reconstructed surface. Therefore, this technique is performed in good manner.