Simple MNIST ConvNet (convolutional neural network)

In this article, we will be creating a basic custom neural network model designed for MNIST dataset using Deep Learning Studio.

Deep Learning Studio

Instead of coding up the entire network, we will be using the GUI based drag and drop software Deep Learning Studio (DLS) to build the custom neural network. DLS will also be used to train and test the network on the dataset provided.

The software can be downloaded from deepcognition.ai by creating a free account. For this example, Windows v3.0.2 will be used.

Dataset

In this example, we will be using the MNIST dataset which is available on DLS. DLS provides MNIST dataset by default in public datasets. MNIST Dataset consists of 70000 grayscale (28x28) images of handwritten digits (0-9). This project uses convolutional layers to classify MNIST images.

Custom Neural Network Model

Creating Project in DLS

We proceed to make our custom neural network model. Click on the Projects tab. To create a new project, click on the Create Project button. A pop up asking for the project name, project type and description shows up. The project type for our model is Custom Neural Network. You can set the name and description as per your preference. Click on the newly created project and you will see a screen with multiple tabs as shown below:

Setting up the Dataset in DLS

In the Data tab (shown above), select the MNIST public dataset that was uploaded to DLS. We will use a 90% - 5% - 5% shuffled train/validation/test split for our dataset i.e. we will train on 70,000 images and using 3,500 images for our validation. The testing set will also have 63,000 images. The input (InputPort0) is the column of Images. The data type for all input will be Image. The output (OutputPort0) will be the Class column and it will be categorical data type. Here is a screenshot of how it should look:

Making Custom Neural Network Model

In the Model tab, we will be creating the neural network for our dataset. The neural network would only consist of fully connected NN layers (or Dense layers) and the Dropout layers. We will also use Convolutional layer on the input test sets. Also with MaxPooling layer In DLS, you will need to drag and drop the layers from the available layers on the left menu. You then connect them together in order. The layers which require additional parameters, like the number of units or the activation function in a Dense layer, needs to be entered in the respective options by clicking on the layer. Following is how the model will look like in DLS along with parameters for each layer (the code for this model is presented after the hyperparameters are selected in the next step):

  • Input_1

    • Shape: (28, 28, 1)

  • Conv2D_1

    • Filters: 32

    • Kernel_size: 3

    • Activation: relu

  • Conv2D_2

    • Filters: 32

    • Kernel_size: 3

    • Activation: relu

  • MaxPolling2D_1

    • pool size: (2,2)

  • Dropout_1

    • Rate: 0.25

  • Flatten_1

  • Dense_1

    • Units: 128

    • Activation: relu

  • Dropout_2

    • Rate: 0.5

  • Dense_2

    • Units: 10

    • Activation: softmax

  • Output_1

    • Shape: (None, 10)

Setting up Hyperparameters

On the Hyperparameters tab, we will tune the hyperparameters using Show Advanced Hyperparameters. The following will be the hyperparameters used:

Training the Custom Neural Network Model

We will now train the model from the Training tab. Select the device using which you want to compute the model parameters and click on Start Training. An ETA will be shown in the top left function along with other statistics shown on the tab. Following is a screenshot of DLS during model training:

Training Results

Once the model has completed its training, we now go to the Results tab to check our result. On the Graphs option, select the RunX from the left and you will see the graphs for our training run. As you can see on the screenshot below, the training loss, training accuracy, validation accuracy, validation loss.

On the Configuration option, you can see all the layers and hyperparameters that were used for the given training run.

Inference Testing Data

On the Inference/Deploy tab, we will now test the model using the testing set of 63,000 images. Select the Dataset Inference tab and select Testing as the Dataset Source. While the DLS software shows the model result and the actual result, we can download it to check for additional metrics, if needed. Following is the screenshot:

Model Code

import tensorflow.keras
from tensorflow.python.keras.layers.convolutional import Conv2D
from tensorflow.python.keras.layers.pooling import MaxPooling2D
from tensorflow.python.keras.layers.core import Dropout
from tensorflow.python.keras.layers.core import Flatten
from tensorflow.python.keras.layers.core import Dense
from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model
from tensorflow.keras.regularizers import *


def get_model():
	aliases = {}
	Input_1 = Input(shape=(28, 28, 1), name='Input_1')
	Conv2D_1 = Conv2D(name='Conv2D_1',filters= 32,kernel_size= 3,activation= 'relu' )(Input_1)
	Conv2D_2 = Conv2D(name='Conv2D_2',filters= 32,kernel_size= 3,activation= 'relu' )(Conv2D_1)
	MaxPooling2D_1 = MaxPooling2D(name='MaxPooling2D_1',pool_size= (2,2))(Conv2D_2)
	Dropout_1 = Dropout(name='Dropout_1',rate= 0.25)(MaxPooling2D_1)
	Flatten_1 = Flatten(name='Flatten_1')(Dropout_1)
	Dense_1 = Dense(name='Dense_1',units= 128,activation= 'relu' )(Flatten_1)
	Dropout_2 = Dropout(name='Dropout_2',rate= 0.5)(Dense_1)
	Dense_2 = Dense(name='Dense_2',units= 10,activation= 'softmax' )(Dropout_2)

	model = Model([Input_1],[Dense_2])
	return aliases, model


from tensorflow.keras.optimizers import *

def get_optimizer():
	return Adadelta()

def is_custom_loss_function():
	return False

def get_loss_function():
	return 'categorical_crossentropy'

def get_batch_size():
	return 128

def get_num_epoch():
	return 12

def get_data_config():
	return '{"mapping": {"Digit Label": {"type": "Categorical", "port": "OutputPort0", "shape": "", "options": {}}, "Image": {"type": "Image", "port": "InputPort0", "shape": "", "options": {"pretrained": "None", "Augmentation": false, "rotation_range": 0, "width_shift_range": 0, "height_shift_range": 0, "shear_range": 0, "horizontal_flip": false, "vertical_flip": false, "Scaling": 1, "Normalization": false, "Resize": false, "Width": 28, "Height": 28}}}, "numPorts": 1, "samples": {"training": 56000, "validation": 7000, "test": 7000, "split": 3}, "dataset": {"name": "mnist", "samples": 70000, "type": "public"}, "datasetLoadOption": "full", "shuffle": false, "kfold": 1}'

Last updated