gh-79638: Treat an unreachable robots.txt as "disallow all"#138555

serhiy-storchaka merged 5 commits intopython:mainpython/cpython:mainfrom

serhiy-storchaka:robotparser-unreachableserhiy-storchaka/cpython:robotparser-unreachableCopy head branch name to clipboard