# Turing-NLG, MS가 공개한 170억개(17b) 파라미터를 가진 언어모델

> Clean Markdown view of GeekNews topic #1504. Use the original source for factual precision when an external source URL is present.

## Metadata

- GeekNews HTML: [https://news.hada.io/topic?id=1504](https://news.hada.io/topic?id=1504)
- GeekNews Markdown: [https://news.hada.io/topic/1504.md](https://news.hada.io/topic/1504.md)
- Type: news
- Author: [xguru](https://news.hada.io/@xguru)
- Published: 2020-02-12T14:27:41+09:00
- Updated: 2020-02-12T14:27:41+09:00
- Original source: [microsoft.com](https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/)
- Points: 2
- Comments: 0

## Topic Body

- Transformer 기반

- BERT-Large 340M, RoBERTa 355M, OpenAI GPT-2 1.5b 등에 비해 10배 이상의 파라미터를 가진 최대크기 모델

- DeepSpeed 와 ZeRO가 있었기에 가능

소개 페이지의 요약문 자체도 Turing-NLG에 의해서 만들어졌다고

"Turing Natural Language Generation (T-NLG) is a 17 billion parameter language model by Microsoft that outperforms the state of the art on many downstream NLP tasks. We present a demo of the model, including its freeform generation, question answering, and summarization capabilities, to academics for feedback and research purposes."

## Comments



_No public comments on this page._