Follow
guess who showed up... https://github.com/tootsuite/mastodon/issues/4486
@angristan This aint good is it? :] Is there any way to turn link previews off?
@ftw there isn't, as far as I know
@angristan Hmm ok. Wouldn't mind claiming back all that storage space as well.
@angristan Ah, had to check, and we set that up a while ago. We're just using 50MB on preview cards right now. Not bad, not bad ^^
@angristan Twitter actually had the same bug with direct file links, but even worse. I managed to report it around 2015
When you post a link Twitter sends like 10 crawlers to it. Which isn't the worst. However, as I ran QuadFile at the time, I noticed that if I posted a link to for example an mp4 or webm file. Twitter's crawlers would snatch the WHOLE thing.
So find a chunky file on any website. Say a 1 GB video file or something. Post a link to it on twitter. And voila, you just made Twitter send 10 bots at once to pull 10 GB of data from that poor server using their huge server farm.
When you post a link Twitter sends like 10 crawlers to it. Which isn't the worst. However, as I ran QuadFile at the time, I noticed that if I posted a link to for example an mp4 or webm file. Twitter's crawlers would snatch the WHOLE thing.
So find a chunky file on any website. Say a 1 GB video file or something. Post a link to it on twitter. And voila, you just made Twitter send 10 bots at once to pull 10 GB of data from that poor server using their huge server farm.
@angristan How a giant like twitter managed to not notice this for so long baffles me.
But honestly it's amazing that mastodon hasn't even attempted to fix it from the look of things.
But honestly it's amazing that mastodon hasn't even attempted to fix it from the look of things.
@quad I remember this!
@angristan The worst part was that they didn't cache anything.
So post the same link 2 seconds later and poof, another 10 GB of traffic.
I managed to peg file.quad.moe at 500+ Mbit/s for like 10 minutes just by placing a large file on the web server and posting the same link in succession, don't even need to use different accounts.
Truly what one would call quality software.
So post the same link 2 seconds later and poof, another 10 GB of traffic.
I managed to peg file.quad.moe at 500+ Mbit/s for like 10 minutes just by placing a large file on the web server and posting the same link in succession, don't even need to use different accounts.
Truly what one would call quality software.
@quad but that was just bad code (or they did this intentionally but I can't see why?) For mastodon, it's a decentralisation/scale problem
@angristan Probably just bug.
I mean if you just wget or curl a URL without thinking of course it's going to fetch the whole file if it's a direct link to a file.
For mastodon it's a bit more of a mess yes.
I mean if you just wget or curl a URL without thinking of course it's going to fetch the whole file if it's a direct link to a file.
For mastodon it's a bit more of a mess yes.
@quad I wonder how much bandwidth they wasted doing that
@angristan Probably way too much. All because they sent a GET request without trying a HEAD request and checking stuff like content type first.
Amazing that a site as big as Twitter has probably had that DDoS tool lying around for so long.
Amazing that a site as big as Twitter has probably had that DDoS tool lying around for so long.
@angristan Ok so the big deal seems to be to know if unintentionally putting down websites is DoS or not (instead of ways to avoid putting down websites)
(for me the best solution we be to let the original instance fetch and distribute the preview)