Self-supervised Learning and Domain Adaptation for Visual Analysis PDF Download

Author: Kevin Lin
Publisher:
ISBN:
Category :
Languages : en
Pages : 131

Get Book Here

Book Description
Supervised training with deep Convolutional Neural Networks (CNNs) have achieved great success in various visual recognition tasks. However, supervised training with deep CNNs requires large amount of well-annotated data. Data labeling, especially for large-scale image dataset, is very expensive. How to learn an effective model without the need of training data labeling has become an important problem for many applications. A promising solution is to create a learning protocol for the neural networks, so that the neural networks can learn to teach itself without manual labels. This technique is referred as the self-supervised learning, which has recently drawn an increasing attention for improving the learning performance. In this thesis, we first present our work on learning binary descriptors for fast image retrieval without manual labeling. We observe that images with the same category should have similar visual textures, and these similar textures are usually invariant to shift, scale and rotation. Thus, we could generate similar texture patch pairs automatically for training CNNs by shifting, scaling, and rotating image patches. Based on the observation, we design a training protocol for deep CNNs, which automatically generates pair-wise pseudo labels describing the similarity between the given two images. The proposed method performs more favorably than the baselines on different tasks including patch matching, image retrieval, and object recognition. In the second part of this thesis, we turn our focus to the task of human-centric analysis applications, and present our work on learning multi-person part segmentation without human labeling. Our proposed complementary learning technique learns a neural network model for multi-person part segmentation using a synthetic dataset and a real dataset. We observe that real and synthetic humans share a common skeleton structure. During learning, the proposed model extracts human skeletons which effectively bridges the synthetic and real domains. Without using human-annotated part segmentation labels, the resultant model works well on real world images. Our method outperforms the state-of-the-art approaches on multiple public datasets. Then, we discuss our work on accelerating multi-person pose estimation using a proposed concatenated pyramid network. We observe that each image may contain an unknown number of people that can occur at any scale or position. This makes fast multi-person pose estimation very challenging. Different from the earlier deep learning approaches that extract image features by using a series of convolutions, our proposed method extracts image features from each convolution layer in parallel, which better captures image features in different scales and improve the performance of human pose estimation. Our proposed method eliminates the need of multi-scale inference and multi-stage detection, and the proposed method is many times faster than the state-of-the-art approaches, while achieving better accuracy on the public datasets. Next, we present our work on 3D human mesh construction from a single image. We propose a novel approach to learn the human mesh representation without any ground truth mesh. This is made possible by introducing two new terms into the loss function of a graph convolutional neural network (Graph CNN). The first term is the Laplacian prior that acts as a regularizer on the mesh construction. The second term is the part segmentation loss that forces the projected region of the constructed mesh to match the part segmentation. Experimental results on multiple public datasets show that without using 3D ground truth meshes, the proposed approach outperforms the previous state-of-the-art approaches that require 3D ground truth meshes for training. Finally, we summarize our completed works and discuss the future research directions.