http://ijece.iaescore.com A method to convert non-numeric characters into numerical values in dynamic time warping for string

2020 Dynamic time warping (DTW) is one of the well-known algorithms for measuring similarity between two temporal sequences, and it can be used for character matching. It uses a distance of two character strings. However, since the characters are non-numeric, it must be assigned to numerical values to calculate a distance between two character strings. Therefore, in this paper, we propose a method to convert non-numeric characters into numerical values in dynamic time warping for string matching. The proposed method uses normalized correlation coefficient, and it makes DTW gives more accurate results. Experimental results show that the proposed method gives excellent


INTRODUCTION
Recent advances in deep learning are raising interest in how to solve problems in various fields using computer vision. As a result, a variety of applications is being created in artificial intelligence, robotics, factory automation, security, and so on [1][2][3][4][5]. Among them, one of the most widely used fields of computer vision is character recognition [6][7][8]. With the development of the character recognition technology using the convolutional neural network (CNN) [9,10], the performance of the character recognition is much better than before. Printed documents or text can be recognized with high accuracy using image processing technology and convolutional neural network. However, if there is a lot of noise in the image, there is a character that is damaged enough to be not recognized, if the character is not hidden, it is still difficult to accurately recognize the character even with the latest technology. There may be partially misrecognized or unrecognized characters, and some characters may be added or dropped, so pure character recognition results are not available.
Therefore, in these cases post-processing is required. Various methods [11][12][13] can be used here. If there is a list of character strings to be recognized, the recognition result can be greatly improved. This is because misrecognized characters can be corrected by comparing the recognized result with a list to see if there are similar items. To do this, you need a method to compare list items and character strings to check similarity and find the most similar items. Methods of comparing the similarity of character strings include Jaccard similarity [14], Hamming distance [15], Jaro-Winkler distance [16], Levenshtein distance [17], and dynamic time warping [18].
In this paper, we match the list with the recognized results using dynamic time warping (DTW). DTW is one of the well-known algorithms for measuring similarity between two temporal sequences. DTW has been used to solve various problems such as online signature recognition [19], shape matching [20], speech recognition [21], speaker recognition [22], and so on. The DTW algorithm calculates the similarity using the distance of two data. Therefore, in order to find the distance of data, the data must be represented as a numerical value. But what we want to match is a character string, which is not numeric data. Therefore, in order to use DTW, a method of converting nonnumeric characters to numeric values is required. Using the converted numeric values, the similarity is calculated with each item in the list and the most similar item can be found. This method has a great influence on the matching accuracy.
In this paper, we propose a method of converting nonnumeric characters into numerical data to use DTW. This method makes it possible to properly calculate the distance between characters by expressing the characters with appropriate numerical values using the shape differences of the characters. Therefore, by using the proposed method, DTW performance can be improved by converting non-numerical data into numerical data properly, and as a result, the final recognition result of incorrect character string recognized by computer vision technique can be improved.

DYNAMIC TIME WARPING
DTW is a method to calculate an optimal match between two given sequences using certain restrictions and rules [23]. The length of the character string recognized by the image processing may be different even for the same character string. However, DTW can match without considering the length of the two sequences, so you can use DTW to match the list. The DTW distance DTW(Q, C) of two sequences Q={q1, …, qn} and C={c1, …, cn} of length n is defined recursively as in (1) [24].
In general, DTW is implemented using dynamic programming technique. It calculates the warping matrix γ(1…n, 1…n) as in (2) to get the DTW distance.
Therefore, to calculate the warping matrix, it has to know the distance between two sequences (dist(qi, ci)). If the sequence is a numeric data, there is no problem to calculate the distance, but if the sequence is a non-numeric data, it has to be converted into numeric data.

METHODS TO CONVERT CHARACTERS INTO NUMERIC DATA
In this paper, DTW is used to improve the accuracy of the material code recognized using CNN in the noisy image as shown in Figure 1. There are 41 characters used in the material code, 0 ~ 9, A ~ Z,-, _, #, [,]. The accuracy improvement depends on how these characters are converted to numeric data.

Method using ASCII code
The simplest method to convert a character to a numeric value is to use the American Standard Code for Information Interchange (ASCII) code. ASCII is one of the character codes, where every single bit represents a unique character. Table 1 shows ASCII code for some characters.  48 49 50 51 52 53 54 55 56 57 65 66 67 68 69 70 71 72 73 74 75 Char 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 35 45 137 133 135 Usually, computer represents characters using ASCII. Therefore, the distance between two characters can be simply calculated by ASCII difference. For example, the distance between the characters 'A' and 'Z' can be calculated as | 65-90 |.

Manual method
The ASCII method is simplest and fastest because there is no need to convert characters to numeric values. However, for character recognition, we should think about whether it is reasonable that the distance between 'A' and 'Z' is 25 and the distance between 'A' and 'B' is 1. When matching with lists using DTW, there is a significant difference between matching digitized strings and matching strings recognized by the CNN. There is no error in the digitized string. However, in character recognition using image processing, there may be characters added or missing due to misrecognition, unrecognition, and incorrect recognition. In particular, similarly shaped characters such as 'B' and '8', 'I' and 'l', 'S' and '5' are often incorrectly recognized.
For example, the recognition result of the string "[3033] N133S-13S-U" in the image using CNN, "[3033] N133S-135-U" was obtained. '5' came from misrecognition of 'S'. To improve the recognition result, DTW using ASCII was used to match incorrect strings and lists. As a result, the most similar item came out as "[3033] N143P-1B-U" (DTW distance = 19) even though there were "[3033] N133S-13S-U" in the list. 'S' and '5' can be misrecognized each other in CNN because the shape is similar. However, the distance using ASCII is 18 (| 53-35 |), so the other item is matched. If the distance difference between 'S' and '5' was small, DTW would have given the correct item.
Therefore, in order to obtain accurate results when matching strings recognized by CNN using DTW, the distance difference between characters that can be misrecognized should be small. It means that characters with similar shapes must have similar values. On the other hand, the distance difference between characters that do not easily misrecognized each other should be large. It means that the characters with different shapes should have a large difference. Table 2 shows the numeric values manually set for each character. 'B' and '8', 'I' and 'l', 'S' and '5', which are similar in shape, are set to similar values to reduce the distance difference, but the characters which have different shape such as '1' and '_' are set to different values to increase the distance.

Character shape based method
In this paper, we propose a method to convert non-numeric character string to numerical values. Now we know that there should be a dependency on the character shapes and the numeric values, so the proposed method uses the shape of the characters to calculate the values. First, to calculate the numerical values based on the shapes of the characters, each character is created as an image as shown in Figure 2. Since the shape of the characters depends on the font, the result is calculated differently depending on the font. In this paper, the image is generated using Console font.
The proposed method uses the normalized correlation coefficient (NCC), which is widely used in template matching [25], to find the differences in character shapes. NCC is one of the methods for calculating the similarity between two images of the same size. It can be calculated using (3).
where n is the number of pixels, f and g are the images, f(x, y) and g(x, y) are the pixels of the (x, y) coordinates of the image, ̅ and ̅ are the average of the image pixels, and σf and σg are the standard deviation of the image f and g. Table 3 shows the calculated NCC values. The high NCC value indicates that the two characters have a more similar shape. The most similar to '0' is 'O', the most similar to '1' is 'I', '5' is 'S', '8' is 'B', and 'E' is 'F'. It shows that the similarity is appropriately represented. 41 × 41 NCC matrix should be reduced to one dimension. In this paper, the average value of NCC is calculated and numerical value is set for each character. Table 4 shows the calculated values.

EXPERIMENTAL RESULTS
The experimental environment for evaluating the proposed method is as follows. We used laptop computer with Intel i7-7500U 2.70 GHz CPU and 8.0 GB RAM, and the proposed method is implemented in Visual Studio 2017 with MFC-based C ++ language. OpenCV 3.4.2 was used as an image processing library. In the experiment, a total of 1.309 images including the material code shown in Figure 1 were used, and the resolution was 1920×1080.
Since there is a lot of noise in the image, the recognition rate is about 30% when all the characters are perfectly recognized. To improve this, we used DTW to match the material code list. Therefore, the recognition success in this paper is defined as follows. The character string recognized in the image is matched with the material code list using the DTW. If the top five items with a small DTW distance contain the correct material code, the recognition is successful. Figure 3 shows the results of DTW matching using the proposed method. The characters on the left are strings recognized using CNN, and the DTW distance and matching results are shown on the right. Table 5 shows the accuracy according to the method of converting characters to numerical value.
The DTW using ASCII successfully matched character strings in 975 of 1,309 images. It improved the pure recognition rate, which was about 30%, to 74.48%. It can be seen that DTW effectively matches strings. The DTW using the manual method correctly matched 1,238 images and showed a 94.58% accuracy. It means that the DTW matching results are improved when the characters with similar shapes are set to similar values as analyzed in this paper. Finally, the proposed method was experimented, and it showed a 95.49% success rate by accurately matching 1,250 images. This means that the normalized correlation coefficient used to compare the similarity of character shapes properly represented the similarity of characters, and it gives better DTW matching results than other methods.

CONCLUSION
In this paper, we proposed a method of converting non-numeric characters into numerical values when using DTW to improve the recognition rate of incorrectly recognized character strings in images. As a result of analyzing the DTW result for string matching, it was found that the matching accuracy is improved by assigning similar values to characters with similar shapes. Therefore, the proposed method calculates the shape similarity between characters using normalized correlation coefficient, and sets numerical values for each character using calculated values. Experimental results showed that the proposed method gives better results (95.49%) than simply converting characters to ASCII (74.48%) or manually converting letters to numbers (94.58%). The proposed method improved the pure recognition rate, which was about 30%, to 95.49%. Therefore, we can confirm that DTW matching using the proposed method is very effective for string matching. However, in the proposed method, since the average is simply used in the process of converting the similarity calculated using NCC into one dimension, it may not accurately reflect the similarity difference of each character. Although the proposed method shows good results, but it is not optimal, so further study on this is needed.