TL;DR Recent works have demonstrated that it is possible to obtain sparse networks that are competitive with dense networks. The outperformance of sparse networks have been investigated primarily in terms of test set accuracy. In this post, I will attempt to summarize the paper Can You Win Everything with A...
Introduction Prior work has demonstrated that increasing the parameter count while increasing the network width improves the performance. Since these studies did not control for increased parameter count, it was hard to establish the exact cause behind the improved performance. A recent paper Are wider nets better given the same...
It has been nearly 5 years since lottery ticket hypothesis (LTH) was proposed. Countless theoretical papers have been written on it. Even after achieving sparsity as high as 95% without a significant drop in performance, the application of LTH in industry remains relatively limited. One of the main factors that...
Introduction Recently I came across an interesting paper: Condensing CNNs with Partial Differential Equations by Kag et al. The paper proposes a way to enforce constraints on the intermediate activations in a CNN model. What is the advantage of enforcing these constraints ? Through extensive experiments on different datasets [ImageNet...
TL;DR I have written a more formal summary of my findings in here. Github: https://github.com/sidml/interpret-fc-layer You can replicate my experiment results by running the notebooks in git repo or by running this Kaggle Notebook for fitting ALD and this for visualizing the internal split. The results of fitting ALD to...
Context: It is often hard to find a large collection of correctly labelled data. This scarcity of labelled real data exists for gravitational waves as well. One way to combat this shortage is to train on simulated data and hope the trained model is useful for the real dataset as...
Recently I participated in Kaggle’s Rainforest Connection Species Audio Detection challenge with my friend Ee Kin. In this post I share the psuedo labels generation procedure, performance of different model architectures and loss functions in more detail. Ee Kin’s had also posted on Kaggle Forum. I have created a notebook...
Recently I came across a very nice paper Objects as Points by Zhou et al. I found the approach pretty interesting and novel. It doesn’t use anchor boxes and requires minimal post-processing. The essential idea of the paper is to treat objects as points denoted by their centers rather than...
Recently Google AI Research published a paper titled “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”. In this paper the authors propose a new architecture which achieves state of the art classification accuracy on ImageNet while being 8.4x smaller and 6.1x faster on inference than the best existing CNN. It...
I got curious about KL Divergence after reading the Variational Auto Encoder Paper. So, I decided to investigate it to get a better intuition. KL Divergence is a measure of how one probability distribution $P$ is different from a second probability distribution $Q$. If two distributions are identical, their KL...
Spatially Regularized Correlation Filters In this post I discuss the basics of the SRDCF framework. In most standard DCF approaches like MOSSE, the size of target search region used for training the classifier is limited. A naive increase in search region may lead to more emphasis on background instead of...
Introduction The aim of visual tracking is to track an arbitrary object in any environment given the initial position and size of the target object. It is an important area of research with applications in wide variety of tasks like automated surveillance, vehicle tracking, traffic accident detection and robotics. Many...
At various points in my life, I have held beliefs that were either deductively/inductively invalid. As I have grown older, I have grown confused by what it m...
Introduction Prior work has demonstrated that increasing the parameter count while increasing the network width improves the performance. Since these studies...