Summer Tree is Cyan

Thinking will not overcome fear but action will.

Distilling the Knowledge in a Neural Network

文章已经表明,对于将知识从整体模型或高度正则化的大型模型转换为较小的蒸馏模型,蒸馏非常有效。在MNIST上,即使用于训练蒸馏模型的迁移集缺少一个或多个类别的任何示例,蒸馏也能很好地工作。对于Android语音搜索所用模型的一种深层声学模型,我们已经表明,通过训练一组深层神经网络实现的几乎所有改进都可以提炼成相同大小的单个神经网络,部署起来容易得多。 对于非常大的神经网络,甚至训练一个完...

Deep Mutual Learning

文章提出了一种简单且普遍适用的方法,通过与同辈和相互蒸馏进行的队列训练来改善深层神经网络的性能。 通过这种方法,我们可以获得比那些强大但静态的teacher提炼的网络性能更好的紧凑网络。 DML的一种应用是获得紧凑,快速和有效的网络。 我们还表明,这种方法也有望改善大型强大网络的性能,并且可以将以此方式训练的网络队列作为一个整体进行组合,以进一步提高性能。 论文:https://i...

MUTUAL MEAN-TEACHING|PSEUDO LABEL REFINERY FOR UNSUPERVISED DOMAIN ADAPTATION ON PERSON RE-IDENTIFICATION

Unsupervised Domain Adaptation Person Re-ID

为了减轻噪音伪标签的影响,文章提出了一种无监督的MMT(Mutual Mean-Teaching)方法,通过在迭代训练的方式中使用离线精炼硬伪标签和在线精炼软伪标签,来学习更佳的目标域中的特征。同时,还提出了可以让Traplet loss支持软标签的soft softmax-triplet loss”。 该方法在域自适应任务方面明显优于所有现有的Person re-ID方法,改进幅度高达1...

Feature Space Regularization for Person Re-Identification with One Sample

Few-shot Person Re-ID

Abstract Targeting to solve the issues above, we propose two simple and effective solutions. (a) We design the Feature Space Regularization (FSR) Loss to adjust the distribution of samples i...

A Concise Review of Recent Few-shot Meta-learning Methods

小样本元学习

@toc 1 Introduction In this short communication, we present a concise review of recent representative meta- learning methods for few-shot image classification. We re- fer to such methods as few-sh...

少标签数据学习 Few labeled data learning

宾夕法尼亚大学课程

Few-shot image classification Three regimes of image classification Problem formulation Training set consists of labeled samples from lots of “tasks”, e.g., classifying cars, cats, dogs, planes...

Elements of Meta-Learning 关于元学习和强化学习

卡耐基梅隆大学 Probabilistic Graphical Models 课程

Goals for the lecture: Introduction & overview of the key methods and developments. [Good starting point for you to start reading and understanding papers!] Probabilistic Graphical Models...

Probabilistic Graphical Models

Statistical and Algorithmic Foundations of Deep Learning

Probabilistic Graphical Models Statistical and Algorithmic Foundations of Deep Learning Author: Eric Xing 01 An overview of DL components Historical remarks: early days of neural networks 我们...

最新小样本学习综述: A Survey on Few-shot Learning

Multitask Learning、Embedding Learning、Learning with External Memory、Generative Modeling

相关阅读: A Survey on Few-Shot Learning | Introduction and Overview A Survey of Few-Shot Learing | Data 给定少数样本的,仅使用简单模型(例如线性分类器)就可以选择较小的H (假设空间)[92,94]。 但是,现实世界中的问题通常很复杂,并且不能由小H的假设h很好地表示[45]。 因此,在FSL中...

A Survey on Few-shot Learning | Data

当前最新小样本学习综述

本节中的FSL方法使用先验知识来增强数据,从而丰富了E中的监督信息。(图4)。 Data augmentation via hand-crafted rules is usually used as pre-processing in FSL methods. They can introduce different kinds of invariance for the model ...