Penguin Species Classification using Backpropagation Neural Network : An Empirical Study of Learning Rate and Mini-Batch Stochastic Gradient Descent

Abstract

This paper presents an empirical study of a fullyconnected
feedforward neural network, trained with the classical
backpropagation algorithm, applied to the task of classifying
penguin species from morphological measurements. Using the
Palmer Archipelago (Antarctica) penguin dataset, a compact
4–4–3 network is implemented from first principles in an
object-oriented MATLAB framework and trained with minibatch
stochastic gradient descent (SGD). Rather than restating
the results of the original design report, the present work
re-implements the network and reproduces the experiments
independently in Python on the same underlying data, so that
the reported numbers are directly verifiable. Two questions are
investigated in detail: (i) how the learning rate η affects test
accuracy over 10 and 50 training epochs, and (ii) whether
mini-batch SGD offers a measurable advantage over full-batch
gradient descent on a dataset of this size. The results show that
classification accuracy exceeds 98% for a broad range of learning
rates (1 ≤ η ≤ 20), that excessively small or large learning
rates cause underfitting or divergence respectively, and that minibatch
SGD yields a statistically clearer and more consistent
convergence at low epoch counts than full-batch descent, although
the two methods converge to comparable accuracy once training
is allowed to proceed for 100 epochs. These findings are discussed
in the context of the theory of stochastic approximation and standard
practical recommendations for training small feedforward
networks.