# Numpy 60라인으로 GPT 구현하기

> Clean Markdown view of GeekNews topic #8464. Use the original source for factual precision when an external source URL is present.

## Metadata

- GeekNews HTML: [https://news.hada.io/topic?id=8464](https://news.hada.io/topic?id=8464)
- GeekNews Markdown: [https://news.hada.io/topic/8464.md](https://news.hada.io/topic/8464.md)
- Type: news
- Author: [xguru](https://news.hada.io/@xguru)
- Published: 2023-02-13T10:03:02+09:00
- Updated: 2023-02-13T10:03:02+09:00
- Original source: [jaykmody.com](https://jaykmody.com/blog/gpt-from-scratch/)
- Points: 32
- Comments: 0

## Topic Body

- Python, Numpy에 익숙하다는 전제하에 간단히 GPT를 구현하고, OpenAI가 공개한 GPT-2 모델을 로드하여 텍스트 생성해보는 튜토리얼   
### 목차  
- GPT란 무엇인가   
- 셋업 : Encoder, Hyperparameters, Parameters   
- Basic Layers : GeLU, Softmax, Layer Normalization, Linear   
- GPT Architecture   
  - Embeddings  
  - Decoder Stack  
  - Project to Vocab  
  - Decoder Block  
  - Position-wise Feed Forward Network  
  - Multi-Head Causal Self Attention   
- 다 합치기   
- What's Next?  
  - GPU/TPU Support  
  - Backpropagation  
  - Batching  
  - Inference Optimization  
  - Training  
  - Evaluation  
  - Architecture Improvements

## Comments



_No public comments on this page._