SIERRA · Sparse Structured Methods for Machine Learning

Machine learning is now a core part of many research domains, where the abundance of data has forced researchers to rely on automated information processing. In practice, today, machine learning techniques are applied in two stages: practitioners first build a large set of features; then, off-the-shelf algorithms are used to solve the appropriate prediction tasks, such as classification or regression. While this has led to significant advances in many domains, I believe that the potential of machine learning is far from being fulfilled. The tenet of this proposal is that to achieve the expected breakthroughs, this two-stage paradigm should be replaced by an integrated process where the specific structure of a problem is taken into account explicitly in the learning process. This will allow the consideration of massive numbers of features, in both numerically efficient and theoretically well-understood ways. I plan to attack this problem through the tools of regularization by sparsity-inducing norms. The scientific objective is thus to marry structure with sparsity: this is particularly challenging because structure may occur in various ways (discrete, continuous or mixed) and my targeted applications in computer vision and audio processing lead to large-scale convex optimization problems. My research program is expected to have a high impact on statistical machine learning research, notably by providing new solutions to the open problem of non-linear variable selection. Moreover, my general methodology will be directly applied to domains where the natural structure of data has been recognized as crucial but is still underused by learning techniques, namely computer vision (object recognition, image denoising) and audio processing (speech separation, music recognition).