| GPTbot should be blocked on any user input content such as forum boards, forum posts in particular, blog entries, news comments (which is just another place for the forum comments. The news articles are fine to be crawled and scraped though in my opinion), and possibly reviews since users might not want chat GPT hijacking their reviews necessarily (though some might so that's why im unsure on this one). https://arstechnica.com/information-technology/2023/08/openai-details-how-to-keep-chatgpt-from-gobbling-up-website-data/ I know MAL already has a robots.txt but it only is generically blocking certain components of the site. I think GPTbot has a different context as to why it might be good to block it since it's not just a search engine crawler but a web scraper that places content in databases. http://myanimelist.net/robots.txt |
traedFeb 15, 9:50 PM
More topics from this board
» Could MAL match users with the most similar watched anime and ratings?mur_koshka - Apr 7 |
14 |
by FruitPunchBaka
»»
6 hours ago |
|
» is it possible to add a like/heart feature for those who prefer not to rate things?FruitPunchBaka - 7 hours ago |
0 |
by FruitPunchBaka
»»
7 hours ago |
|
» Cap the maximum YT embeds / Don't preload YouTube embeds in spoiler blocksDaviljoe193 - May 21 |
5 |
by xLoop
»»
Yesterday, 8:31 AM |
|
» MAL needs a "Year in review" function at the end of every year.OhlLonesomelMe - May 20 |
0 |
by OhlLonesomelMe
»»
May 20, 11:27 PM |
|
Poll: » Change picture of favorite character ( 1 2 )gehoti2822 - Nov 12, 2022 |
68 |
by Shion_Rosenheim
»»
May 20, 9:15 PM |