Inspiration

The objective of writing this article is to use a very crude approach of doing image classification, in this case, images of handwritten text. While we use Convolutional Neural Network models from scratch or use pre-trained models on MNIST data sets, it's more preconditioned for this job. We use transfer learning, and in that process as a student myself, I miss out on the very basics.

It's like I am driving an automatic car, wherein I know what the drivetrain, the clutch, the accelerator, and brakes do, but I don't know anything beyond these. We attempt to break down the problem of handwritten alphabet image recognition into a simple process rather than using heavy packages. This is an attempt to create the data and then build a model using Support Vector Machines for Classification.

Preparing the Data

We are going to manually prepare the data rather than download it from the web. This would allow us to understand our data from an initial stage. We are going to write a few alphabets manually on a white sheet and take images on our phone's camera. We will then port it to our hard drive.

Since it's an experiment, I don't want to spend a lot of time in the initial run. I would create data for maybe two or three distinct alphabets for demonstration. Its recommended that you try this mechanism for all the alphabets and check the efficiency.

You may need to modify your code when you add more classes of alphabets, but that's where the learning part will start. We are, right now. just in the training phase.

Structure of Data Storage

We can either write the alphabets on white paper and then extract it using a camera phone or directly use a graphics tool like paint to write using the pen tool. I have created two folders — train and test. In the train folder, we can save folders with the alphabet name, while in the test folder, we can dump the images for which we want to finally classify the instances.

The training subfolders have been kept with an intention to have the subfolder name as the training tags. The Testing folder has not been kept in the same form, as we intend to do the classification here.

Figure 1: File Schema of Hand Written Images 

Alternatively, if you want to download the data I have used, right-click on this download data link and open it in a new tab or window. Unzip the folders and you should be able to see the same structure and data as above in your downloads folder. Later, you should create your own data and do the whole process again. This will show you the complete cycle. 

Download the Dependency Packages in RStudio

We will be using the jpeg package in R for Image handling and the Support Vector Machine implementation from the kernlab package.

This would be a one-time install. Also, we have made sure the image data has dimensions of 200 x 200 pixels, with a horizontal and vertical resolution of 120dpi. You can try variations in the color channels and resolution later once you have implemented it in the current form.

R1

   install package"jpeg"

2

   install.packages("jpeg", dependencies = TRUE)

3

4

   # install the "kernlab" package for building the model using support vector machines

5

   install.packages("kernlab", dependencies= TRUE)

Loading the Training Data Set

R

1

   # load the "jpeg" package for reading the JPEG format files

2

   library(jpeg)

3

4

   # set the working directory for reading the training image data set

5

   setwd("C:/Users/mohan/Desktop/alphabet_folder/Train")

6

7

   # extract the directory names for using as image labels

8

   f_train<-list.files()

9

10

   # Create an empty data frame to store the image data labels and the extracted new features in training environment

11

   df_train<- data.frame(matrix(nrow=0,ncol=5))

Feature Engineering

Since our intention is to not use a typical CNN approach, we are going to use the white, grey, and black pixel values for new feature creation.

We intend to use the summation of all the pixel values of an image instance and save it in a feature called "sum", the count of all pixels evaluating to zero as "zero", the count of all pixels evaluating to "ones", and the sum of all pixels evaluating to values other than zeroes and ones as "in_between". The "label" feature is extracted from the folder name.

R

1

   # names the columns of the data frame as per the feature name schema

2

   names(df_train) <- c("sum","zero","one","in_between","label")

3

4

   # loop to compute as per the logic

5

   counter <-1

6

7

   for(i in 1:length(f_train))

8

  {

9

       setwd(paste("C:/Users/mohan/Desktop/alphabet_folder/Train/",f_train[i],sep=""))

10

11

       data_list <-list.files()

12

13

       for(j in 1:length(data_list))

14

      {

15

           temp<- readJPEG(data_list[j])

16

           df_train[counter,1] <- sum(temp)

17

           df_train[counter,2] <- sum(temp==0)

18

           df_train[counter,3] <- sum(temp==1)

19

           df_train[counter,4] <- sum(temp > 0 & temp < 1)

20

           df_train[counter,5] <- f_train[i]

21

           counter=counter+1

22

      }

23

  }

Build the Support Vector Machine Model

R

1

   # load the "kernlab" package for accessing the support vector machine implementation

2

   library(kernlab)

3

4

   # build the model using the training data

5

   image_classifier <- ksvm(label~.,data=df_train)

Loading the Testing Data Set

R

1

   # set the working directory for reading the testing image data set

2

   setwd("C:/Users/mohan/Desktop/alphabet_folder/Test")

3

4

   # extract the directory names for using as image labels

5

   f_test <- list.files()

6

7

   # Create an empty data frame to store the image data labels and the extracted new features in training environment

8

   df_test<- data.frame(matrix(nrow=0,ncol=5))

9

10

   # Repeat of feature extraction in test data

11

12

   names(df_test)<- c("sum","zero","one","in_between","label")

13

14

   # loop to compute as per the logic

15

   for(i in 1:length(f_test))

16

  {

17

       temp<- readJPEG(f_test[i])

18

       df_test[i,1]<- sum(temp)

19

       df_test[i,2]<- sum(temp==0)

20

       df_test[i,4]<- sum(temp > 0 & temp < 1)

21

       df_test[i,5]<- strsplit(x = f_test[i],split = "[.]")[[1]][1]

22

  }

23

Implement for Prediction and Check Metrics

R

1

   # Use the classifier named "image_classifier" built in train environment to predict the outcomes on features in Test environment

2

   df_test$label_predicted<- predict(image_classifier,df_test)

3

   # Cross Tab for Classification Metric evaluation

4

   table(Actual_data=df_test$label,Predicted_data=df_test$label_predicted)

I would encourage you to the learn concepts of Support Vector Machines that couldn’t be explored completely in this article by clicking on this link for my free Data Science and Machine Learning video courses.

While we have used the kernlab package and created the classifier, there is a lot of math involved, starting from vector space to kernel tricks. We have worked on the implementation of the classifier, but you should certainly learn the conceptual part of Support Vector Machines and other interesting algorithms.