Abyss of links 🔗

I previously mentioned bottomless pits of links.

If your capsule has URLs with numbers in them, and the resulting pages have a link to "next" that adds a constant to the current number, you have this issue, and crawlers may well crawl all the possible numbers. I found another one of these that accepted very big numbers indeed, up to the point where it gave an error about not being able to translate a string to a number.

I told the #hashtag crawler to stop going down that specific rabbit hole.

#hashtags

#crawler

back to gemlog