Most of today's popular deep architectures are hand-engineered to be generalists. However, this design procedure usually leads to massive redundant, useless, or even harmful features for specific tasks. Unnecessarily high complexities render deep nets impractical for many real-world applications, especially those without powerful GPU support. In this paper, we attempt to derive task-dependent compact models from a deep discriminant analysis perspective. We propose an iterative and proactive approach for classification tasks which alternates between (1) a pushing step, with an objective to simultaneously maximize class separation, penalize co-variances, and push deep discriminants into alignment with a compact set of neurons, and (2) a pruning step, which discards less useful or even interfering neurons. Deconvolution is adopted to reverse 'unimportant' filters' effects and recover useful contributing sources. A simple network growing strategy based on the basic Inception module is proposed for challenging tasks requiring larger capacity than what the base net can offer. Experiments on the MNIST, CIFAR10, and ImageNet datasets demonstrate our approach's efficacy. On ImageNet, by pushing and pruning our grown Inception-88 model, we achieve more accurate models than Inception nets generated during growing, residual nets, and popular compact nets at similar sizes. We also show that our grown Inception nets (without hard-coded dimension alignment) clearly outperform residual nets of similar complexities.

Qing Tian

Tal Arbel

James J. Clark

Computer Vision and Image Understanding

该杂志的中心焦点是图片信息的计算机分析。计算机视觉和图像理解发表的论文涵盖了图像分析的各个方面，从早期视觉的低级标志性过程到识别和解释的高级象征性过程。涵盖了图像理解领域的广泛主题，包括提供与主流观点不同的见解的论文。研究领域包括：• 理论• 早期视觉• 数据结构和表示• 形状• 范围• 运动• 匹配和识别• 建筑和语言• 视觉系统

The journal "Computer Vision and Image Understanding" focuses on the computer analysis of image information. It publishes papers covering various aspects of image analysis, ranging from low-level feature processing in early vision to high-level symbolic processes of recognition and interpretation. The topics covered are broad and include, but are not limited to: theory, early vision, data structures and representations, shape, depth, motion, matching and recognition, architecture and language, and visual systems.

《Computer Vision and Image Understanding》杂志专注于图像信息的计算机分析。该杂志发表的论文涵盖了图像分析的各个方面，从早期视觉的低级特征处理到高级的识别与解释过程。研究主题广泛，包括但不限于：理论、早期视觉、数据结构与表示、形状、深度、运动、匹配与识别、建筑与语言以及视觉系统。

Grow-push-prune: Aligning deep discriminants for effective structural network compression

This paper proposes an iterative and proactive approach for classification tasks which alternates between a pushing step, with an objective to simultaneously maximize class separation, penalize co-variances, and push deep discriminants into alignment with a compact set of neurons, and a pruning step, which discards less useful or even interfering neurons.

Category	Quartile
计算机科学	3
计算机科学, 计算机人工智能	3
计算机科学, 工程电子与电气	3

Grow-push-prune: Aligning deep discriminants for effective structural network compression

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
ENGINEERING, ELECTRICAL & ELECTRONIC	1