TopHot
The New York Times blocks OpenAI’s web crawler
The New York Times blocks OpenAI’s web crawler Illustration by Alex Castro / The VergeThe New York Times has blocked OpenAI’s web crawler, meaning that OpenAI can’t use content from the publication to train its AI models. If you check the NYT’s robots.txt page, you can see that the NYT disallows GPTBot, the crawler that OpenAI introduced earlier this month. Based on the Internet Archive’s Wa...
Category:
