Skip to content
View gmlwns2000's full-sized avatar
  • Anyang, Korea

Highlights

  • Pro

Organizations

@Kawaian @NeuralAction

Block or report gmlwns2000

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this userโ€™s behavior. Learn more about reporting abuse.

Report abuse
gmlwns2000/README.md

I am Heejun Lee. I am a Ph.D. student at the Korea Advanced Institute of Science and Technology AI. Hello, World! ๐Ÿฑ

My research interest includes:

  • Neural Network Compression and Computation Optimization (Currently, I am focusing on an attention mechanism)
  • Large-scale Machine Learning System Design, Development, and Serving (This is why I am working with DeepAuto.AI)
  • General Purpose AI (NLP/CV)

Education

  • KAIST | Sep. 2024 - | Ph.D. in Artificial Intelligence
  • KAIST | Mar. 2020 - Aug. 2024 | B.S. in Computer Science

Publications

[A2] HiP Attention: Sparse Sub-Quadratic Attention with Hierarchical Attention Pruning
[paper][code] Heejun Lee*, Geon Park*, Youngwan Lee*, Jaduk Suh*, Jina Kim, Wonyoung Jeong, Bumsik Kim, Hyemin Lee, Myeongjae Jeon, Sung Ju Hwang. Arxiv 2024

[A1] Training-Free Exponential Extension of Sliding Window Context with Cascading KV Cache
[paper][code] Jeffrey Willette, Heejun Lee, Youngwan Lee, Myeongjae Jeon, Sung Ju Hwang. Arxiv 2024

[C2] SEA: Sparse Linear Attention with Estimated Attention Mask
[paper][code] Heejun Lee, Jina Kim, Jeffrey Willette, Sung Ju Hwang. ICLR 2024

[C1] Sparse Token Transformer with Attention Back Tracking
[paper][code] Heejun Lee, Minki Kang, Youngwan Lee, Sung Ju Hwang. ICLR 2023

(*: equal contribution)

Experiences

  • KAIST MLAI. Jul 2021 - Aug 2024. Undergraduate Intern (I am continuing my Ph.D. program in this lab)
  • DeepAuto.ai. Jan 2023 - May 2023. Research Engineer
  • Transverse. Oct 2020 - Mar 2021. AI Lead Developer

Awards

  • 23'F KAIST College of Engineering Leadership Award on Research Excellence
  • 22'F KAIST College of Engineering Leadership Award on Research Excellence
  • 22'S KAIST College of Engineering Dean's List
  • 18' Intel International Science and Engineering Fair Finalist & 4th Grand Award in Robotics and Intelligent Machines
  • 18' ์ œ 9ํšŒ ํ•œ๊ตญ์ฒญ์†Œ๋…„๊ณผํ•™์ฐฝ์˜๋Œ€ํšŒ(ISEF-K) 1๋“ฑ (1st Place)
  • 17' ์ œ 34ํšŒ ํ•œ๊ตญ์ •๋ณด์˜ฌ๋ฆผํ”ผ์•„๋“œ(Korea Olympiad in Informatics; KOI) ๊ณต๋ชจ๋ถ€๋ถ„ ๊ธˆ์ƒ (Gold Medal; 2nd Place)
  • 16' ์ œ 33ํšŒ ํ•œ๊ตญ์ •๋ณด์˜ฌ๋ฆผํ”ผ์•„๋“œ(Korea Olympiad in Informatics; KOI) ๊ณต๋ชจ๋ถ€๋ถ„ ๊ธˆ์ƒ (Gold Medal; 2nd Place)

Academic Services

Pinned Loading

  1. DeepAuto-AI/hip-attention DeepAuto-AI/hip-attention Public

    Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.

    Python 23 3

  2. sea-attention sea-attention Public

    Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)

    Python 7 1

  3. sttabt sttabt Public

    [ICLR2023] Official code of Sparse Token Transformer with Attention Back-Tracking

    Jupyter Notebook 6

  4. NeuralAction/NeuralAction NeuralAction/NeuralAction Public

    Neural Action is a real-time CNN-based gaze tracking application providing human-machine interface to improve accessibility.

    C# 49 6

  5. NeuralAction/Vision NeuralAction/Vision Public

    Computer Vision And Neural Network with Xamarin

    Jupyter Notebook 56 14

  6. sharkshark-4k sharkshark-4k Public

    Upscale Twitch stream and restream into Twitch or RTMP or File.

    Python 15 1