Robots.txt for the NYT has a specific exclusion for an 1996 news article
20 by mikeortman | 2 comments on Hacker News.
Sunday, March 28, 2021
Home
Hacker News
New top story on Hacker News: Robots.txt for the NYT has a specific exclusion for an 1996 news article
New top story on Hacker News: Robots.txt for the NYT has a specific exclusion for an 1996 news article
Subscribe to:
Post Comments (Atom)
Post Bottom Ad
Responsive Ads Here
No comments:
Post a Comment