Skip to content

Commit dc36ca1

Browse files
committed
Update robots.txt comment
1 parent de7184c commit dc36ca1

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

robots.txt

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
User-agent: *
22
Allow: /
33

4-
# Block GPTBot from accessing articles. This is a little brittle, but it works
5-
# on the basis that all my blog posts were published at some point in the 21st
6-
# century. TODO: Update when I’m 109.
4+
# Block AI bots from accessing articles for training purposes.
5+
#
6+
# This is a little brittle, but it works on the basis that all my blog posts
7+
# were published at some point in the 21st century.
8+
#
9+
# TODO: Update when I’m 109.
10+
711
User-agent: GPTBot
812
Disallow: /20*
913

0 commit comments

Comments
 (0)