Brain Tumor Classification Using Tensorflow and Transfer Learning

Source: Freepik.com

I used machine-learning techniques to classify four different types of brain tumors; No Tumor, Glioma Tumor, Meningioma Tumor, and Pituitary Tumor. So, I thought of writing an article about the same.

I will divide this article into two different parts, In part 1 we will build a machine-learning model to classify the brain tumor (the one you’re reading), and in part 2 I will talk about how to put your machine-learning model into production.

PART 2: Machine Learning in Production

The points we are going to cover today:

• What are Brain Tumor and their types
• Data preparation
• What is Transfer Learning?
• Definition of different layers
• Training the model
• Making predictions through trained model
• Evaluating the model
• Saving the model for future use.

So, without any further ado, let’s get started.

What is Brain Tumor?

A Brain Tumor is nothing but a mass or growth of abnormal cells in your brain. It is considered one of the most aggressive diseases, among children and adults. The best technique to detect brain tumors is Magnetic Resonance Imaging (MRI).

A huge amount of image data is generated through the scans. These images are examined by the radiologist. A manual examination can be error-prone due to the level of complexities involved in brain tumors and their properties. As a Machine learning enthusiast, the best way I could think of to reduce human intervention and human error is to automate this classification process using machine-learning techniques.

Enough of talking, let's dive into the actual work!

We are going to use the dataset provided by Kaggle. You can download it or use the Kaggle notebook for performing classification. I have used the Kaggle notebook and will share it with you on GitHub.

As the title suggests we are going to use the Tensorflow library for classification tasks beside that, for data analysis and visualization we are going to use some other libraries such as pandas, numpy, seaborn, matplotlilb, and Scikit-learn. Now, let's import all the libraries.

import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
import cv2
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tqdm import tqdm
import os
from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split
from tensorflow.keras.applications import EfficientNetB0
from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, TensorBoard, ModelCheckpoint
from sklearn.metrics import classification_report,confusion_matrix
import ipywidgets as widgets
import io
from PIL import Image
from IPython.display import display,clear_output
from warnings import filterwarnings
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

let’s define some colors which will help us with visualization.

colors_dark = ["#1F1F1F", "#313131", '#636363', '#AEAEAE', '#DADADA']
colors_red = ["#331313", "#582626", '#9E1717', '#D35151', '#E9B4B4']
colors_green = ['#01411C','#4B6F44','#4F7942','#74C365','#D0F0C0']

sns.palplot(colors_dark)
sns.palplot(colors_green)
sns.palplot(colors_red)

Data Preparation

We are going to detect four different types of Brain Tumors. So, declare a variable called label like below.

labels = ['glioma_tumor','no_tumor','meningioma_tumor','pituitary_tumor']

We will split the training set for creating a classification model. The testing set provided by Kaggle is for testing the ML model into production. If you don't understand this now, don't worry go on reading the article and it will all make sense :)

X_train = []
y_train = []
image_size = 150
for i in labels:
    folderPath = os.path.join('../input/brain-tumor-classification-mri','Training',i)
    for j in tqdm(os.listdir(folderPath)):
        img = cv2.imread(os.path.join(folderPath,j))
        img = cv2.resize(img,(image_size, image_size))
        X_train.append(img)
        y_train.append(i)
        
for i in labels:
    folderPath = os.path.join('../input/brain-tumor-classification-mri','Testing',i)
    for j in tqdm(os.listdir(folderPath)):
        img = cv2.imread(os.path.join(folderPath,j))
        img = cv2.resize(img,(image_size,image_size))
        X_train.append(img)
        y_train.append(i)
        
X_train = np.array(X_train)
y_train = np.array(y_train)

In the above code, we are walking through the training and testing directories, and appending to the python list after resizing the image. Note that we are resizing the image to 150x150.

Now, let’s plot a few images for each label.

k=0
fig, ax = plt.subplots(1,4,figsize=(20,20))
fig.text(s='Sample Image From Each Label',size=18,fontweight='bold',
             fontname='monospace',color=colors_dark[1],y=0.62,x=0.4,alpha=0.8)
for i in labels:
    j=0
    while True :
        if y_train[j]==i:
            ax[k].imshow(X_train[j])
            ax[k].set_title(y_train[j])
            ax[k].axis('off')
            k+=1
            break
        j+=1

We will shuffle the x_train and y_train that we created using the shuffle method of python and will look at the shape.

X_train, y_train = shuffle(X_train,y_train, random_state=101)
X_train.shape

Further, we will split our dataset into Training and Testing sets. Test set being the 10% of the Training set.

X_train,X_test,y_train,y_test = train_test_split(X_train,y_train, test_size=0.1,random_state=101)

We will make use of the sci-kit learn train_test_split function for splitting our dataset into two. The test_size parameter helps us to set the size of the testing set (0.1 = 10%).

We all know that Computers only understand numerical values. Our labels which we have declared before is a string of list. Somehow, we have to convert these labels into numerical values so that we can input the labels into a machine-learning model. The best solution to convert your labels into numerical values is to use a famous technique called One Hot Encoding.

y_train_new = []
for i in y_train:
    y_train_new.append(labels.index(i))
y_train = y_train_new
y_train = tf.keras.utils.to_categorical(y_train)


y_test_new = []
for i in y_test:
    y_test_new.append(labels.index(i))
y_test = y_test_new
y_test = tf.keras.utils.to_categorical(y_test)

Explaining One Hot Encoding is beyond the scope of this article but what it does is that it assigns the binary value to each label. The above code helps us to do the same.

Phew! 😮‍💨

Lots of work 😵

But now, we have to do actual work: building a classification model using transfer learning but

What is Transfer Learning?

Deep Convolution Neural Network models may take days or even months to train on very large datasets. A simple way to short-cut this process is to re-use the model weights from pre-trained models that were developed for standard computer vision benchmark datasets, such as the ImageNet image recognition tasks. In simple words, Transfer Learning is the reuse of a pre-trained model on a new problem. It also recommended using transfer learning when we have a small dataset.

Top-performing models can be downloaded and used, or integrated into a new model for your computer vision problems. We are going to use the EfficientNetB0 model which will use the weights from the ImageNet dataset.

effnet = EfficientNetB0(weights='imagenet',include_top=False,input_shape=(image_size,image_size,3))

The above line of code will help us to reuse the weights. The include_top parameter is set to False so that the network doesn’t include the top layer/ output layer from the pre-built model which allows us to add our output layer depending upon our use case!

Now, we are going to append our custom layers to the EfficientNetB0 model.

model = effnet.output
model = tf.keras.layers.GlobalAveragePooling2D()(model)
model = tf.keras.layers.Dropout(rate=0.5)(model)
model = tf.keras.layers.Dense(4,activation='softmax')(model)
model = tf.keras.models.Model(inputs=effnet.input, outputs = model)

We have used a few layers and explanations of every layer are given below:

GlobalAveragePooling2D: This layer acts similarly to the Max Pooling layer in CNNs, the only difference being that it uses the Average values instead of the Max value while pooling. This will helps us in decreasing the computational load on the machine while training.

Dropout: Dropout is a technique where randomly selected neurons are ignored during training. They are “dropped out” randomly. It helps in avoiding overfitting.

Dense: A simple Neural Network layer and also our output layer which classifies the image into 1 of the 4 possible classes. Note that in the above code we have 4 as the first parameter in the Dense layer which means we have to output 4 different possibilities (classes).

We have also used the Softmax function in the Dense layer which outputs the probabilities of the prediction values.

model.compile(loss='categorical_crossentropy',optimizer = 'Adam', metrics= ['accuracy'])

We finally compile our model.

checkpoint = ModelCheckpoint("effnet.h5",monitor="val_accuracy",save_best_only=True,mode="auto",verbose=1)
reduce_lr = ReduceLROnPlateau(monitor = 'val_accuracy', factor = 0.3, patience = 2, min_delta = 0.001, mode='auto',verbose=1)

Callbacks: Callbacks can help you fix bugs more quickly, and can help you build better models. They can help you visualize how your model’s training is going, and can even help prevent overfitting by implementing early stopping or customizing the learning rate on each iteration.

Training Model

history = model.fit(X_train,y_train,validation_split=0.1, epochs =12, verbose=1, batch_size=32, callbacks=[tensorboard,checkpoint,reduce_lr])

The above code will help you to train the model. If you’re using CPU to train the model then it will take almost one or two hours to train the model but if you’re using GPU then it will take only a few minutes.

filterwarnings('ignore')

epochs = [i for i in range(12)]
fig, ax = plt.subplots(1,2,figsize=(14,7))
train_acc = history.history['accuracy']
train_loss = history.history['loss']
val_acc = history.history['val_accuracy']
val_loss = history.history['val_loss']

fig.text(s='Epochs vs. Training and Validation Accuracy/Loss',size=18,fontweight='bold',
             fontname='monospace',color=colors_dark[1],y=1,x=0.28,alpha=0.8)

sns.despine()
ax[0].plot(epochs, train_acc, marker='o',markerfacecolor=colors_green[2],color=colors_green[3],
           label = 'Training Accuracy')
ax[0].plot(epochs, val_acc, marker='o',markerfacecolor=colors_red[2],color=colors_red[3],
           label = 'Validation Accuracy')
ax[0].legend(frameon=False)
ax[0].set_xlabel('Epochs')
ax[0].set_ylabel('Accuracy')

sns.despine()
ax[1].plot(epochs, train_loss, marker='o',markerfacecolor=colors_green[2],color=colors_green[3],
           label ='Training Loss')
ax[1].plot(epochs, val_loss, marker='o',markerfacecolor=colors_red[2],color=colors_red[3],
           label = 'Validation Loss')
ax[1].legend(frameon=False)
ax[1].set_xlabel('Epochs')
ax[1].set_ylabel('Training & Validation Loss')

fig.show()

After the model has been trained, it is good to visualize your model performance on every epoch. It helps us to identify how our model performed on each epoch and when we can stop our training, etc. The best way to measure the performance of your model is to use a line graph.

The above code plots the line graph of training and validation accuracy/loss.

Prediction

pred = model.predict(X_test)
pred = np.argmax(pred,axis=1)
y_test_new = np.argmax(y_test,axis=1)

We will use the argmax function as each row from the prediction array contains four values for the respective labels. The maximum value which is in each row depicts the predicted output out of the 4 possible outcomes.

So with argmax, we’ll be able to find out the index associated with the predicted outcome.

Evaluation

After training the model and doing predictions most of us (mostly newbies) directly use the model in production because most of us only care about the accuracy which is one way to measure the performance of our model but there are other different ways to measure the model performance as well.

By evaluating our model we can get an insight into the performance of our model. The evaluation of the model helps us to understand different matrices (performance measures).

print(classification_report(y_test_new,pred))

The classification_report method of sci-kit learn will help us to find out different performance measures such as precision, recall, f1-score, support, accuracy, macro avg, etc.

We don’t understand the numbers and text easily but visualization of the same helps us to understand better. The above code will give you a report with different numbers.

Let’s visualize it.

fig,ax=plt.subplots(1,1,figsize=(14,7))
sns.heatmap(confusion_matrix(y_test_new,pred),ax=ax,xticklabels=labels,yticklabels=labels,annot=True,
           cmap=colors_green[::-1],alpha=0.7,linewidths=2,linecolor=colors_dark[3])
fig.text(s='Heatmap of the Confusion Matrix',size=18,fontweight='bold',
             fontname='monospace',color=colors_dark[1],y=0.92,x=0.28,alpha=0.8)

plt.show()

We will plot a heatmap of the confusion matrix which in turn helps us to summarize the model performance visually.

Saving our model

model.save('modelv1.h5')

There are different ways to save the Tensflow model but I save the model in .h5 format. It’s because it's the easiest way to save the model and loading the same is also easier.

Conclusion

We learned about transfer learning and we also applied the same to a real-world project. The above code will help you to get around 98% of accuracy. We also learn about Brain Tumors.

In the upcoming article which is part 2 of this article, we will use this model to build and web app and learn how to use our ML model in production.

Till then I hope you liked my article. You can find the necessary links in this article and down here’s my Github code for this project. I hope I helped you a bit.

I got so much love from my previous articles. If you have any doubts, recommendations, issues, or anything you can always contact me on LinkedIn, Twitter, and Instagram.

A very small request, if you have any suggestions or anything please fill out the below form which will help me to write on the technology and topics which you wish me to write. Thank you :)

Happy Coding 😉

https://forms.gle/1zB9GiKAdejyNbkh6

GitHub - thejitenpatel/brainTumorDetection

Jiten Patel - Medium