About me

Keywords: Music Information Retrieval, Deep learning

Contact: jongpillee.brian (at) gmail.com



  • 2017. Summer Research Intern, Naver corp, Korea


  • Multi-Level and Multi-Scale Feature Aggregation Using Sample-level Deep Convolutional Neural Networks for Music Classification

Music tag words that describe music audio by text have different levels of abstraction. Taking this issue into account, we propose a music classification approach that aggregates multilevel and multi-scale features using pre-trained feature extractors. In particular, the feature extractors are trained in sample-level deep convolutional neural networks using raw waveforms. We show that this approach achieves state-of-the-art results on several music classification datasets.
Jongpil Lee, Juhan Nam
International Conference on Machine Learning (ICML) Machine Learning for Music Discovery Workshop, 2017

Recently, the end-to-end approach that learns hierarchical representations from raw data using deep convolutional neural networks has been successfully explored in the image, text and speech domains. This approach was applied to musical signals as well but has been not fully explored yet. To this end, we propose sample-level deep convolutional neural networks which learn representations from very small grains of waveforms (e.g. 2 or 3 samples) beyond typical frame-level input representations. Our experiments show how deep architectures with sample-level filters improve the accuracy in music auto-tagging and they provide results comparable to previous state-of-the-art performances for the Magnatagatune dataset and Million Song Dataset. In addition, we visualize filters learned in a sample-level DCNN in each layer to identify hierarchically learned features and show that they are sensitive to log-scaled frequency along layer, such as mel-frequency spectrogram that is widely used in music classification systems.
Jongpil Lee, Jiyoung Park, Keunhyoung Luke Kim, Juhan Nam
Sound and Music Computing Confenrence (SMC) (Accepted), 2017

Music auto-tagging is often handled in a similar manner to image classification by regarding the 2D audio spectrogram as image data. However, music auto-tagging is distinguished from image classification in that the tags are highly diverse and have different levels of abstractions. Considering this issue, we propose a convolutional neural networks (CNN)-based architecture that embraces multi-level and multi-scaled features. The architecture is trained in three steps. First, we conduct supervised feature learning to capture local audio features using a set of CNNs with different input sizes. Second, we extract audio features from each layer of the pre-trained convolutional networks separately and aggregate them altogether given a long audio clip. Finally, we put them into fully-connected networks and make final predictions of the tags. Our experiments show that using the combination of multi-level and multi-scale features is highly effective in music auto-tagging and the proposed method outperforms previous state-of-the-arts on the Magnatagatune dataset and the million song dataset. We further show that the proposed architecture is useful in transfer learning.
Jongpil Lee, Juhan Nam
IEEE Signal Processing Letters (Accepted), 2017

This research focuses on two distinctive determinants of DJ popularity in Electronic Dance Music (EDM) culture. While one's individual artistic tastes influence the construction of playlists for festivals, social relationships with other DJs also have an effect on the promotion of a DJ’s works. To test this idea, an analysis of the effect of DJs’ social networks and the audio features of popular songs was conducted. We collected and analyzed 713 DJs’ playlist data from 2013 to 2015, consisting of audio clips of 3172 songs. The number of cases where a DJ played another DJ's song was 15759. Our results indicate that DJs tend to play songs composed by DJs within their exclusive groups. This network effect was confirmed while controlling for the audio features of the songs. This research contributes to a better understand of this interesting but unique creative culture by implementing both the social networks of the artists’ communities and their artistic representations.
Hyeongseok Wi, Kyung hoon Hyun, Jongpil Lee, Wonjae Lee
International Computer Music Conference (ICMC), 2016