Convolution made easy

In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, where The name “convolutional neural network” indicates that the network employs a mathematical operation called convolution.

A convolutional neural network consists of an input layer, hidden layers, and an output layer. In any feed-forward neural network, any middle layers are called hidden because their inputs and outputs are masked by the activation function and final convolution.

In a convolutional neural network, the hidden layers include layers that perform convolutions. Typically this includes a layer that does element-wise multiplication or dot product. 1D Convolutions are simplified versions of the 2D Convolution. Unlike 2D Convolutions, where we slide the kernel in two directions, for 1D Convolutions, we only slide the kernel in a single direction as shown in the below diagram. In a CNN, the input is a tensor with a shape: (number of inputs) x (input height) x (input width) x (input channels). After passing through a convolutional layer, the image becomes abstracted to a feature map, also called an activation map, with shape: (number of inputs) x (feature map height) x (feature map width) x (feature map channels).

Heart Sound Classification

A convolutional layer within a CNN generally has the following attributes: Convolutional filters/kernels defined by a width and height (hyper-parameters). The number of input channels and output channels (hyper-parameters). One layer’s input channels must equal the number of output channels (also called depth) of its input. Additional hyperparameters of the convolution operation, such as padding, stride, dilation.

Fig1:1D Convolution with a stride=1

Example: 1D Convolution with default padding where padding field is set to=’Valid

1D Convolution without padding with stride=1

Element-wise multiplication and summing up the result or apply dot product on an input sequence with convolving filter/kernel.

Example: 1D Convolution with padding where padding field is set to=’Same’(padding with constant ‘zero’ known as Zero paddings.)

Following the above method to get an output sequence length same as that of the input sequence length only after padding.

Wondering how many zeros to be padded with???

Padding (P)=(f-1)/2 i.e append zero on the front and rear end of an input sequence). Here,f is the length of the filter.

1D Convolution with padding with stride = 1

Below code is been designed taking into account traversing through multiple layers of filter(s) and extending the code to work for Conv 2D, Conv 3D.

Convolution Class takes 3 parameters namely: filter_len(Filter length),n_f(Number of filters),b(bias)

import numpy as np

# Function for 1DConvolution which will convolve one and only input layer passed by Conv function as a parameter.

 
def conv1d(input,filter,stride,padding):
print(“Filter”,filter)
input=input.reshape(-1)
n_f=filter.shape[0]
print(“Input”,input)
if (len(input.shape)>3):
m=input.shape[3]
n=filter.shape[3]
elif (len(input.shape)>2 and len(filter.shape)>3):
m=input.shape[2]
n=filter.shape[3]
elif (len(input.shape)>2 and len(filter.shape)>2):
m=input.shape[2]
n=filter.shape[2]
elif (len(input.shape)<2 and len(filter.shape)==2):
m=input.shape[0]
n=filter.shape[1]
else:
m=input.size
n=filter.shape[2]
padding_strings = {‘same’, ‘valid’}
if padding not in padding_strings:
raise ValueError(“Invalid padding string {!r}, should be one of {}”.format(padding,padding_strings))
if padding == ‘valid’:
output_size=int((m-n)/stride)+1
result=np.zeros((n_f,1,output_size))

####Implementation of Convolution which can be modified on the need basis depending on 1D,2D or 3D convolution

###Convolution Class takes 3 parameters namely: filter_len(Filter length),n_f(Number of filters),b(bias)

class Convolution:
def __init__(self,filter_len,n_f,b):
np.random.seed(1)
self.filter=np.random.randint(5,size=(n_f,1,filter_len))
self.filter_len=filter_len
self.n_f=n_f
self.bias=np.random.randint(b,size=1)
print(“Bias:”,self.bias)

“1-D Convolution” function file to perform the convolution operation, given the input sequence and one filter sequence, such that the output convolution length is given by m-n+1, where ‘m’ and ’n’ are the length of input and the filter sequence respectively.

Output convolution with length m-n+1 implies that its padding field is set to ‘valid’ which also means that Convolution without padding.

#Create Convolution Layer

“1-D Convolution” function file to perform the convolution operation given the input sequence and one filter sequence, such that the output convolution length is the same as that of the length of the input sequence.

Given: Input sequence, Number of filters=1 In order to make the Output convolution length the same as Input sequence length, set the padding field=’same’, which means zero is padded to the original input sequence and then it is convolved with the filter.

np.random.seed(1)
input=np.random.randint(20,size=(5))
f1=1
Total_params=0
print(“Input,Input shape:”,input,input.shape)
layer1=Convolution(3,1,1)
layer1.conv(input,f1,1,’same’)
print(“Output feature map,it’s shape:”,layer1.feature_map,layer1.feature_map.shape)

Perform the convolution operation given the input sequence and ‘F1’ number of filter sequences, followed by the convolution operation with ‘F2’ number of filter sequences, where F2>F1.

Let us take F1=1 and F2=2 where F1 is the number of filters used in Convolution Layer 1 and F2 is the number of filters used in Convolution layer 2. The output of Convolution Layer 1 = Input to Convolution Layer 2

First Convolution Layer

np.random.seed(1)
input=np.random.randint(20,size=(8))
f1=1
print(“F1 Filter”)
Total_params=0
print(“Input,Input shape:”,input,input.shape)
layer1=Convolution(3,1,1)
layer1.conv(input,f1,1,’valid’)
print(“Output feature map,it’s shape:”,layer1.feature_map,layer1.feature_map.shape)
if (len(input.shape)>1):
n=input.shape[0]
else:
n=1
Learnable_Params=layer1.summary(layer1.feature_map.shape[0],n)
print(“Learnable Prameters:”,Learnable_Params)
Total_params+=Learnable_Params
print(“Total Number of Learnable Prameters:”,Total_params)

Second Convolution Layer

input=layer1.feature_map
print(“Input”,input,input.shape)
f2=2
print(“F2 filter”)
layer2=Convolution(3,2,1)
layer2.conv(input,f2,1,’valid’)
print(“Output feature map:”,layer2.feature_map,layer2.feature_map.shape[0])

Please find the code built from scratch in my Github Repo

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store