Machine Learning from Scratch

This repository documents my journey into the core mechanics of machine learning and deep learning by implementing fundamental algorithms and neural network components from scratch using only NumPy. The goal is to move beyond the black-box abstractions of high-level frameworks and build a deep, first-principles understanding of how these systems learn.

Project Philosophy

Modern deep learning frameworks are powerful tools for productivity. However, a true understanding of their inner workings—gradient flow, numerical stability, and the implementation of optimization algorithms—is essential for debugging complex models, optimizing for performance, and developing novel architectures. This repository is a deliberate effort to build that foundational knowledge.

Implemented Algorithms & Components

Module	Component	Status	Description
Classical ML	K-Means Clustering	✅ Done	Unsupervised clustering using iterative centroid updates.
	Logistic Regression	✅ Done	Binary classification using gradient descent on the log-loss function.
	Principal Component Analysis (PCA)	✅ Done	Dimensionality reduction via eigenvector decomposition of the covariance matrix.
Neural Network	DenseLayer	✅ Done	A fully connected layer with forward and backward passes.
	ReLU Activation	✅ Done	Rectified Linear Unit activation with its derivative.
	Softmax & CrossEntropyLoss	✅ Done	Combined, numerically stable final layer and loss function for classification.
	Full MLP Classifier	✅ Done	A complete Multi-Layer Perceptron trained on the MNIST dataset.
Convolutional NN	Conv2D Layer	✅ Done	2D convolutional layer for feature extraction from images.
	MaxPool2D Layer	✅ Done	Max pooling layer for downsampling feature maps.
Sequence Models	RNN/LSTM Layer	⏳ Planned	Recurrent layers for processing sequential data.

Installation (Needed libraries)

pip install -r requirements.txt

Usage

Each implementation is designed to be self-contained and easily understandable. To run an example, navigate to the relevant directory and execute the main script.

# Example: Running the K-Means implementation
cd k_means
python plot_data.py ```