I am an Assistant Professor in the Department of Software and Computer Engineering at Ajou University, starting in March 2024, where I lead SAIL, Speech AI Lab.. Prior to this, I worked as a postdoctoral researcher in AI Research Center, Korea University, Seoul, South Korea. I received the Ph.D. degree in the Department of Brain and Cognitive Engineering, Korea University in 2023. In March 2016, I started my integrated M.S.&Ph.D. in Pattern Recognition & Machine Learning (PRML) Lab at the Korea University in Seoul, Korea, under the supervision of Seong-Whan Lee.

• E-mail: [email protected], [email protected]
• Google Scholar: Link
• Speech AI Lab. (SAIL): Link
• PRML Speech Team (Supervisor: Seong-Whan Lee): Link

• Speech Synthesis (2019-, HierSpeech++, DDDM-VC, Diff-HierVC)
• Neural Vocoder (2021-, PeriodWave, PeriodWave-Turbo, Fre-GAN, Fre-GAN2)
• Neural Audio Codec (2024-)
• Singing Voice Synthesis (2022-, MIDI-Voice, HiddenSinger)
• Speech-to-Speech Translation (2023-, TranSentence)
• Brain-Computer Interface (2019-2020, Brain-to-Speech System)
• Reinforcement Learning (2017-2018, AI Curling Robot Curly)

• 2025.01: One paper has been accepted to ICLR 2025 (PeriodWave).
• 2025.01: One paper has been accepted to IEEE Trans. on Affective Computing (DurFlex-EVC).
• 2025.01: One paper has been accepted to IEEE Trans. on Audio, Speech and Language Processing (SDT).
• 2024.09: One paper has been accepted to Neural Networks (HiddenSinger). This project was funded by Netmarble AI Center, Netmarble Corp. in 2022.

• Accelerating High-Fidelity Waveform Generation via Adversarial Flow Matching Optimization, S.-H. Lee, H.-Y. Choi, and S.-W. Lee, 2024. (Under Review) [Code]
• HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation by Hierarchical Variational Inference for Zero-shot Speech Synthesis, S.-H. Lee, H.-Y. Choi, S.-B. Kim, and S.-W. Lee, 2023. (Under Review) [Demo] [Code] [Gradio]

• PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation, S.-H. Lee, H.-Y. Choi, and S.-W. Lee, ICLR, 2025. [Code]
• DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations Without Text Alignment, H.-S. Oh, S.-H. Lee, D.-H. Cho, and S.-W. Lee, IEEE Trans. on Affective Computing, 2025. [Demo] [Code]
• Personalized and Controllable Voice Style Transfer with Speech Diffusion Transformer, H.-Y. Choi, S.-H. Lee, and S.-W. Lee, IEEE Trans. on Audio, Speech and Language Processing, 2025
• HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Models, J.-S. Hwang, S.-H. Lee, and S.-W. Lee, Neural Networks, 2025 [Demo]

• DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training, H.-S. Oh, S.-H. Lee, and S.-W. Lee, IEEE Trans. on Audio, Speech and Language Processing, 2024 [Demo] [Code]
• Cross-lingual Text-to-Speech via Hierarchical Style Transfer, S.-H. Lee, H.-Y. Choi, and S.-W. Lee, ICASSPW, 2024.
• Audio Super-resolution with Robust Speech Representation Learning of Masked Autoencoder, S.-B. Kim, S.-H. Lee, H.-Y. Choi, S.-W. Lee, IEEE Trans. on Audio, Speech and Language Processing, 2024.
• TranSentence: Speech-to-Speech Translation via Language-agnostic Sentence-level Speech Encoding without Language-parallel Data, S.-B. Kim, S.-H. Lee, and S.-W. Lee, ICASSP, 2024.
• MIDI-Voice: Expressive Zero-shot Singing Voice Synthesis via MIDI-driven Priors, D.-M. Byun, S.-H. Lee, J.-S. Hwang, and S.-W. Lee, ICASSP, 2024.
• DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion, H.-Y. Choi*, S.-H. Lee*, and S.-W. Lee, AAAI, 2024. [Demo] [Code] [Poster]

• "METHOD AND SYSTEM FOR SYNTHESIZING SPEECH," 10-2663162, 29, Apr., 2024.
• "METHOD TO TRANSFORM VOICE," 10-2439022, 29, Aug., 2022.
• "METHOD AND APPARTUS FOR VOICE CONVERSION BY USING NEURAL NETWORK," 10-2340486, 14, Dec., 2021.
• "SYSTEM AND METHOD FOR CURLING SWEEPING CONTROL," 10-2257358, 21, May, 2021.
• "APPARATUS AND METHOD FOR RECOMMENDATION OF CURLING GAME STRATEGY USING DEEP LEARNING," 10-2045567, 11, Nov., 2019.
• "APPARATUS AND METHOD FOR DELIVERY AND SWEEPING AT CURLING GAME," 10-1948713, 11, Feb., 2019.

2016.03-2023.02: Integrated M.S.&Ph.D, Dept. of Brain and Cognitive Engineering, Korea University

2012.03-2016.02: B.S, Dept. of Life Science, Dongguk University

Reviewer: NeurIPS, ICLR, ICML, AAAI, ICASSP, IEEE/ACM Transactions on Audio, Speech, and, Language Processing

2022.02.25: Paper Award (Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram Generation with Adversarial Style Combination for Speech Synthesis), Korea University