Jianyu Fan

​

    I am Jianyu Fan, an Applied Scientist at Microsoft working on Machine Learning. Previously, I did my Ph.D. at Metacreation Lab, Simon Fraser University working with Dr. Philippe Pasquier. My Ph.D. research lies in the field of Affective Computing, Machine Listening, Human-Computer Interaction, and Computational Creativity.

    Research Highlights

    AudioMetaphor: A Framework for Computer-Assisted Sound Design System

    Miles Thorogood, Jianyu Fan, Philippe Pasquier

    Journal of New Music Research, vol. 48, no. 3, pp. 264-280, 2019

    Multi-label Sound Event Retrieval Using a Deep Learning-based Siamese Structure with a Pairwise Presence Matrix

    Jianyu Fan, Eric Nichols, Daniel Tompkins, Ana Elisa Méndez Méndez, Benjamin Elizalde Philippe Pasquier 

    International Conference on Acoustics, Speech, and Signal Processing, 2020

    A Comparative Study of Western and Chinese Classical Music Based on Soundscape Models

    Jianyu Fan,  Yi-Hsuan Yang, Kui Dong, Philippe Pasquier

    International Conference on Acoustics, Speech, and Signal Processing, 2020

    Soundscape Audio Signal Classification and Segmentation using Listeners Perception of Background and Foreground Sound

    Miles Thorogood, Jianyu Fan, Philippe Pasquier

    Journal of the Audio Engineering Society, vol. 64, no. 9, pp. 484-492, 2016

    DJ-MVP : An Automatic Music Video Producer

    Jianyu Fan, William Li, Jim Bizzocchi, Justine Bizzocchi, Philippe Pasquier

    International Conference on Advances in Computer Entertainment Technology, 2016

     
     

    Peer-Reviewed Publications

    2021

    • Weina Jin, Jianyu Fan, Diane Gromala, Philippe Pasquier and Ghassan Hamarneh. EUCA: A Practical Prototyping Framework towards End-User-Centered Explainable Artificial Intelligence. arXiv:2102.02437, 2021. [PDF]​

    2017

    • Jianyu Fan, Miles Thorogood, and Philippe Pasquier.  Emo-Soundscape: A Database for Soundscape Emotion Recognition.  In Proceedings of the Affective Computing and Intelligent Interaction (ACII), Texas, USA, 2017. [PDF]

     

    • Jianyu Fan, Kıvanç Tatar, Miles Thorogood, and Philippe Pasquier.  Ranking-Based Experimental Music Emotion Recognition.  In Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR), Suzhou, China, 2017. [PDF]

     

    • Jianyu Fan, Philippe Pasquier, Luciane Maria Fadel, and Jim Bizzocchi.  ViVid - A Video Feature Visualization Engine.  In Proceedings of the 19th International Conference on Human-Computer Interaction, Vancouver, Canada, 2017. [PDF]

     

    • Jim Bizzocchi, Arne Eigenfeldt, Philippe Pasquier, Jianyu Fan, and Le Fang.  Berlin Remix.  In Proceedings of the Electronic Visualization and the Arts, London, UK, 2017. [PDF]

    2018

    • Jianyu Fan, Fred Tung, William Li, and Philippe Pasquier.  Soundscape Emotion Recognition via Deep Learning. In Proceedings of the 15th Sound and Music Computing (SMC), Limassol, Cyprus, 2018. [PDF]

     

    • Jianyu Fan, Miles Thorogood, Kıvanç Tatar, and Philippe Pasquier.  Quantitative Analysis of the Impact of Mixing on Perceived Emotion of Soundscape Recordings.  In Proceedings of the 15th Sound and Music Computing (SMC), Limassol, Cyprus, 2018. [PDF]

     

    • Weina Jin, Jianyu Fan, Diane Gromala, and Philippe Pasquier.  Automatic Prediction of Cybersickness for Virtual Reality Games.  In Proceedings of the IEEE Games, Entertainment, and Media Conference (GEM), Galway, Ireland, 2018. [PDF]

     

    • Min Fan, Jianyu Fan, Sheng Jin, Alissa N. Antle, and Philippe Pasquier.  EmoStory: A Game-based System Supporting Children's Emotional Development.  In Extended Abstracts of the Conference on Human Factors in Computing Systems (CHI), ACM Press, Montreal, Canada, 2018. [PDF]

     

    • William Li, Omid Alemi, Jianyu Fan, and Philippe Pasquier.  Ranking-Based Affect Estimation of Motion Capture Data in the Valence-Arousal Space.  In Proceedings of the 5th International Conference on Movement and Computing (MOCO), Genoa, Italy, 2018. [PDF]

    2016

    • Jianyu Fan, Miles Thorogood, and Philippe Pasquier.  Automatic Soundscape Affect Recognition Using A Dimensional Approach.  Journal of the Audio Engineering Society, vol. 64, no. 9, pp. 646-653, 2016. [PDF]

     

    • Miles Thorogood, Jianyu Fan, and Philippe Pasquier.  Soundscape Audio Signal Classification and Segmentation Using Listeners Perception of Background and Foreground Sound.  Journal of the Audio Engineering Society, vol. 64, no. 7/8, pp. 484-492, 2016. [PDF]

     

    • Jianyu Fan, William Li, Jim Bizzocchi, Justine Bizzocchi, and Philippe Pasquier.  DJ-MVP: An Automatic Music Video Producer.  In Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology (ACE), Osaka, Japan, 2016. Bronze Paper Award [PDF]

    2020

    • Jianyu Fan, Eric Nichols, Daniel Tompkins, Ana Elisa Méndez Méndez, Benjamin Elizalde, and Philippe Pasquier. Multi-label Sound Event Retrieval Using a Deep Learning-based Siamese Structure with a Pairwise Presence Matrix. In Proceedings of the 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Barcelona, Spain, 2020. [PDF]

    ​

    • Jianyu Fan, Yi-Hsuan Yang, Kui Dong, and Philippe Pasquier. A Comparative Study of Western and Chinese Classical Music based on Soundscape Models. In Proceedings of the 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Barcelona, Spain, 2020. [PDF]

    ​

    • Victor Cheung, Alissa N. Antle, Shubhra Sarker, Min Fan, Jianyu Fan, and Philippe Pasquier. Techniques for Augmented-Tangibles on Mobile Devices for Early Childhood Learning. In Proceedings of the ACM Interaction Design for Children Conference (IDC), ACM Press, London, UK, 2020. [Link] 

    2019

    • Miles Thorogood, Jianyu Fan and Philippe Pasquier. A Framework for Computer-Assisted Sound Design Systems Supported by Modelling Affective and Perceptual Properties of Soundscapes. Journal of New Music Research, vol. 48, no. 3, pp. 264-280, 2019. [Link]

     

    • Daniel Tompkins, Eric Nichols and Jianyu Fan. Hierarchical Sound Event Classification. In Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE), New York, USA, 2019. [PDF]

    ​​

    • Min Fan, Jianyu Fan, Alissa. N. Antle, Jin Sheng, Dongxu Yin, and Philippe Pasquier. Character Alive: A Tangible and Augmented Reality System for Chinese Children with Dyslexia. In Extended Abstracts of the Conference on Human Factors in Computing Systems (CHI), ACM Press, Glasgow, UK, 2019. [PDF]

    Earlier

    • Jianyu Fan, Miles Thorogood, Bernhard Riecke, and Philippe Pasquier.  Automatic Recognition of Eventfulness and Pleasantness of Soundscape.  In Proceedings of the 10th Audio Mostly, Thessaloniki, Greece, 2015. [PDF]

     

    • Miles Thorogood, Jianyu Fan, and Philippe Pasquier.  BF-Classifier: Background/Foreground Classification and Segmentation of Soundscape Recordings.  In Proceedings of the 10th Audio Mostly, Thessaloniki, Greece, 2015. [PDF]

     

    • Jianyu Fan, Spencer Topel.  SonicTaiji: A Mobile Instrument for Taiji Performance.  In Proceedings of the 10th International Community for Auditory Display (ICAD), New York, USA, 2014. [PDF]

     

    • Jianyu Fan, Michael A. Casey.  Study of Chinese and UK Hit Song Prediction.  In Proceedings of the 10th International Symposium on Computer Music Multidisciplinary Research (CMMR), Marseille, France, 2013. [PDF]

     

    Books, Journals, Conferences & Workshops Reviewer

    • International Joint Conference on Artificial Intelligence, IJCAI, 2021, 2019, 2017.

    • IEEE Workshop on Artificial Intelligence for Art Creation, 2021

    • International Society for Music Information Retrieval Conference, ISMIR, 2020, 2019, 2018.

    • International Conference on Computational Creativity, ICCC, 2020, 2018.

    • International Workshop on Musical Metacreation, MUME, 2020, 2018, 2016.

    • Neural Processing Letters, 2020.

    • IEEE Transactions on Affective Computing, 2019, 2018.

    • IEEE Transactions on Multimedia, 2019, 2018.

    • IEEE Transactions on Cognitive and Developmental Systems, 2019.

    • International Journal of Social Robotics, 2019, 2018.

    • IEEE Virtual Reality Conference, 2019.

    • ML4Creativity, SIGKDD 2017 Workshop on Machine Learning for Creativity, 2017.

    • Integrating Cognitive Architectures Into Virtual Character Design. Engineering Science Reference. 2016.

    • Applied Science, 2016.

    • ACM Computers in Entertainment, 2015.

    • International Conference on Evolutionary and Biologically Inspired Music and Art, 2015.

    • International Community for Auditory Display, ICAD 2015.

    Miscellaneous

    Honors and Awards

    • R. Murray Schafer Soundscape Award (Simon Fraser University, 2020)

    • Graduate Fellowship Award (Simon Fraser University, 2019)

    • President's Ph.D. Scholarship (Simon Fraser University, 2018)

    • The Ralph M. Howatt Family Graduate Scholarship (Simon Fraser University, 2017)

    • Travel & Minor Research Award (Simon Fraser University, 2017)

    • Bronze Paper Award 13th International Conference on Advances in Computer Entertainment Technology (2016)

    • Graduate Fellowship Award (Simon Fraser University, 2016)

    • Travel & Minor Research Award (Simon Fraser University, 2015)

    • Scholarship for Ph.D. Program (Simon Fraser University, 2014)

    • Full Tuition Scholarship and Annual Stipend (Dartmouth College, 2012)

    • First Prize, National Undergraduate Electronic Design Contest (National Ministry of Education, 2011)

    • First Class Scholarship for Academic Contest  (Beihang University, 2011)

    • Third Prize, “Fengru Cup” Scientific and Technology Innovation Competition (Beihang University, 2011)

    • Third Prize, National English Speaking Contest (Beijing area) (China Daily, 2011)

    • First Prize, National Art Talent Competition—Piano (China Conservatory, 2007)

    • First Prize, Beijing Student Art festival (Beijing Municipal Commission of Education, 2007)

     

    Artistic Works

    ​

    ​

    International Symposium on Electronic Art, 2016

    ​

    CML.png

    ​

      

    Album made by Contemporary Music Lab at Dartmouth College, 2014

    ​

    ​

    ​

      

    Interactive Performance, Dart Music Festival, 2014

    ​

    ​

    ​

    ​

    Deep Listening: Art/Science 2nd Annual International Conference on Deep Listening, 2014

    ​

    ​

    ​

    ​

    Invisible Places | Sounding Cities Conference, 2014

    ​

    ​

    Performances:

    © 2018 by Jianyu Fan.