activation-function-exploration

This is the official implementation of our ICML 2024 paper "MultiMax: Sparse and Multi-Modal Attention Learning""

An Artificial Neural Network (ANN) built using PyTorch to classify images in the Fashion MNIST dataset, utilizing GPU support for improved training performance.

This repository features hands-on Jupyter Notebooks, covering everything from fundamental concepts to advanced neural network architectures.

Open ideas. Code less. Publish more.

By Ikansh Mahajan, 3C53, 102303754, COE'27

Interactive deep learning project to predict IPL innings scores from match context such as teams, venue, batsman, bowler, runs, wickets, and overs.

We introduce two novel hybrid activation functions: S3 (Sigmoid-Softsign) and its improved version S4 (Smoothed S3)

Linear and Non Linear Activation Functions : Linear, ReLU, Sigmoid, Softmax, Tanh

This is a Fake and AI image prediction using Transfer Learning

This repository demonstrates Artificial Neural Networks (ANNs) using TensorFlow and Keras, exploring various activation functions, gradient descent techniques, and optimization algorithms through classification and regression tasks.

Parametric Sigmoid Linear Unit

This is a custom-built neural network that detects handwritten numbers from image inputs. It uses ReLU activation in the hidden layers and a softmax activation function in the output layer for classification. The model is trained using backpropagation with a loss function to minimize prediction errors, achieving over 99% accuracy when predicting

A mini supervised learning framework

This repository contains the official implementation of the paper: "DAPA: Distribution Aware Piecewise Activation Functions for On-Device Transformer Inference and Training" (Accepted at DAC 2026).

Streamlit app demonstrating how neural networks approximate nonlinear functions using different activation functions.

Neural network quickly made within 3 days as a personal project, then used as a substitute for a final projects from the algorithms & data structures class in university.