Awesome Knowledge Distillation

A school management Software

Pytorch implementation of various Knowledge Distillation (KD) methods.

Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)

DALI: a large Dataset of synchronised Audio, LyrIcs and vocal notes.

Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals. WSDM 2021.

[ICLR 2021 Spotlight Oral] "Undistillable: Making A Nasty Teacher That CANNOT teach students", Haoyu Ma, Tianlong Chen, Ting-Kuei Hu, Chenyu You, Xiaohui Xie, Zhangyang Wang

College Based Data Management System

A Comprehensive Survey on Knowledge Distillation

It is an online Student-Teacher portal wherein teachers can upload various assignments related to their subjects which the student can download.

Swirl.jl is an interactive learning platform for Julia that runs entirely in the console/terminal, inspired by the popular swirl package for R. It provides a hands-on, guided learning experience where users type real Julia code and get immediate feedback.

PyTorch implementation of "Distilling the Knowledge in a Neural Network"

Deep Neural Network Compression based on Student-Teacher Network

This project implements knowledge distillation from DINOv2 (Vision Transformer) to convolutional networks, enabling efficient visual representation learning with reduced computational requirements.

A production framework for DSPy implementing the Teacher-Student pattern. Distill the reasoning of expensive models (Teacher) into optimized prompts for cheap, fast models (Student) to reduce inference costs by up to 50x.

Student Teacher interactive platform

The Pytorch implementation of Graph convolution network (Kipf et.al. 2017) with vanilla Teacher-Student architecture of knowledge distillation (Hinton et.al 2015).

REST API in Django using Django REST Framework.

Mobile-first education software for teachers.

Teaching materials for Procedural Programming Lab