guess who showed up... https://github.com/tootsuite/mastodon/issues/4486
@angristan This aint good is it? :] Is there any way to turn link previews off?
@ftw there isn't, as far as I know
@angristan Hmm ok. Wouldn't mind claiming back all that storage space as well.
@angristan Ah, had to check, and we set that up a while ago. We're just using 50MB on preview cards right now. Not bad, not bad ^^
When you post a link Twitter sends like 10 crawlers to it. Which isn't the worst. However, as I ran QuadFile at the time, I noticed that if I posted a link to for example an mp4 or webm file. Twitter's crawlers would snatch the WHOLE thing.
So find a chunky file on any website. Say a 1 GB video file or something. Post a link to it on twitter. And voila, you just made Twitter send 10 bots at once to pull 10 GB of data from that poor server using their huge server farm.
But honestly it's amazing that mastodon hasn't even attempted to fix it from the look of things.
@quad I remember this!
So post the same link 2 seconds later and poof, another 10 GB of traffic.
I managed to peg file.quad.moe at 500+ Mbit/s for like 10 minutes just by placing a large file on the web server and posting the same link in succession, don't even need to use different accounts.
Truly what one would call quality software.
@quad but that was just bad code (or they did this intentionally but I can't see why?) For mastodon, it's a decentralisation/scale problem
I mean if you just wget or curl a URL without thinking of course it's going to fetch the whole file if it's a direct link to a file.
For mastodon it's a bit more of a mess yes.
@quad I wonder how much bandwidth they wasted doing that
Amazing that a site as big as Twitter has probably had that DDoS tool lying around for so long.
@quad @angristan doesn't Mastodon have this exact problem now that they stopped doing a head request first?
@quad @angristan I haven't confirmed it in the code, but that's what gargon says in the above thread.
@rune @quad looks like it. https://github.com/tootsuite/mastodon/commit/9ba40a6bfdb65f6e48eb7de07b2d55314c54fa83 But it makes sense for Mastodon's issue
(for me the best solution we be to let the original instance fetch and distribute the preview)