Devlog #002

Today I continued working on my neural network project. After implementing a simple XOR model, I wanted to move to something closer to a real machine learning problem, so I started working with the MNIST dataset.

The goal was to make a neural network capable of recognizing handwritten digits from images. I used TensorFlow only to load the dataset, while the entire neural network implementation was written from scratch using only NumPy.

I started by preprocessing the data, converting the 28x28 pixel images into 784 input values and normalizing the pixel values between 0 and 1. I also implemented one-hot encoding for the labels, so the network could classify the 10 possible digits.

For the model, I created a simple Multi-Layer Perceptron (MLP) with:

• an input layer of 784 neurons
• a hidden layer with 32 neurons using the ReLU activation function
• an output layer with 10 neurons using Softmax for classification

I implemented the forward propagation, cross-entropy loss function, backpropagation, and gradient descent completely manually with NumPy.

The most interesting part was seeing how the formulas I studied previously actually translate into code: matrix multiplications for the layers, derivatives for backpropagation, and gradient updates for learning the weights.

The model is still very simple, but it can already reach around 78-80% accuracy on MNIST, which was a great result considering that everything was implemented from scratch.

In the next devlogs, I want to improve the project by adding:

• mini-batch training for better and faster learning
• the Adam optimizer instead of simple gradient descent
• larger and deeper networks
• dropout and batch normalization
• UI to represent how the network works.

Thank you for reading!