So the PostgreSQL server was under really high load whenever the web worker was running, which affected all of the instance.
The solution was on the PostgreSQL side: https://pgtune.leopard.in.ua
(Thanks a lot to Jackster on the Discord server)
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!