cross-posted from: https://lemmy.world/post/2357075
It seems that self hosting, for oneself, a federated service, like Lemmy, would only serve to increase the traffic in the network, and not actually serve the purpose of load balancing between servers.
As far as I understand it, the way federation is supposed to work is that the servers cache all the content locally to then serve to the people that are registered to that server. In doing so, the servers only have to transmit a minimal amount of data between themselves which lowers the overhead for small servers – this then means that a small server doesn’t get overwhelmed by a ton of people requesting from it. Now, if, instead, you have everyone self hosting their own server, you go right back to having everyone sending a ton of requests to small servers, thereby overwhelming them. It seems that it’s really only beneficial to the network if you have, say, hundreds of medium sized servers instead of, say, thousands, of very small servers. While there is the resilience factor, the overhead of the network would be rather overwhelming.
Perhaps one possibility of fixing this is to use some form of load balancer like IPFS to distribute the requests more evenly, but I am no where even remotely close to being knowledgeable enough in that to say anything definitively.
A big issue with Lemmy right now is how picture storage works. All photos are cached as they enter the instance and there isn’t much to do to turn it off. It’s ridiculous, especially for server scaling. The database in of itself is small, it’s really the pictures that are an issue and grow rapidly.
That’s why it’s stated in the Lemmy docs to use an image host instead of uploading directly. Unfortunately, most users don’t do that.
You can also configure pict-rs to run on object storage so that all your users’ images are stored on S3 rather than your local disk.
I was looking at that earlier and grabbing an S3 bucket or setting up MinIO does not appeal to me. I think I’m just burned out from IRL work.
Now if we had a federated image service that would be used by default to upload images, this would mitigate having the images on the lemmy servers :)
and instead of the fediverse protocol, it could be more like i2p, everyone help caching images, even the apps could implement that