# 구글, 1.6조개 파라미터 AI NLP 모델 Switch Transformer를 오픈소스로 공개

> Clean Markdown view of GeekNews topic #3765. Use the original source for factual precision when an external source URL is present.

## Metadata

- GeekNews HTML: [https://news.hada.io/topic?id=3765](https://news.hada.io/topic?id=3765)
- GeekNews Markdown: [https://news.hada.io/topic/3765.md](https://news.hada.io/topic/3765.md)
- Type: news
- Author: [xguru](https://news.hada.io/@xguru)
- Published: 2021-02-19T10:43:21+09:00
- Updated: 2021-02-19T10:43:21+09:00
- Original source: [infoq.com](https://www.infoq.com/news/2021/02/google-trillion-parameter-ai/)
- Points: 3
- Comments: 0

## Topic Body

- 기존 T5(Text-to-Text Transfer Transformer) 모델 대비 7x의 학습속도 개선

- Switch Routing 이라는 변형 MoE(Mixture-of-Experts) 알고리듬으로, 입력되는 값별로 다른 파라미터를 적용

- 모델 학습을 위해 Mesh-Tensorflow 사용 (Model Parallelism)

## Comments



_No public comments on this page._