# NodeJS용 Robots.txt 파서

> Clean Markdown view of GeekNews topic #5752. Use the original source for factual precision when an external source URL is present.

## Metadata

- GeekNews HTML: [https://news.hada.io/topic?id=5752](https://news.hada.io/topic?id=5752)
- GeekNews Markdown: [https://news.hada.io/topic/5752.md](https://news.hada.io/topic/5752.md)
- Type: news
- Author: [xguru](https://news.hada.io/@xguru)
- Published: 2022-01-15T10:17:01+09:00
- Updated: 2022-01-15T10:17:01+09:00
- Original source: [github.com](https://github.com/samclarke/robots-parser)
- Points: 4
- Comments: 1

## Topic Body

- IETF 의 Robots Exclusion Protocol 드래프트와 호환을 목표로 개발

- 지원 규칙

ㅤ→ User-agent, Allow, Disallow, Sitemap, Crawl-delay, Host

ㅤ→ Path : Wildcard(*) 와 EOL($) 매칭

## Comments



### Comment 8413

- Author: xguru
- Created: 2022-01-15T10:18:01+09:00
- Points: 1

- Wikipedia 의 robots.txt https://news.hada.io/topic?id=90