Follow

juicedata/juicefs: JuiceFS is a distributed POSIX file system built on top of Redis and S3 github.com/juicedata/juicefs

currently trying it out, it works pretty well!

@angristan I guess redis isn't just caching, so if there's no enough space in RAM for metadata there's no places lefts for new files too ? 😬

@angristan
I've been looking for something like this for years, was almost going to make one that exposes iSCSI instead.

@r000t there's a bunch of alternatives like s3fs, goofys, s3ql, riofs, utahfs, etc

So far s3ql is the best in terms of features and performance, but juicefs looks very promising

@angristan s3fs is the only one I've tried and iirc it had trouble keeping inodes predictable, which screws with an awful lot of sync and backup software. also had trouble streaming data from it, so videos and photos weren't very fun to browse.

I'll probably point juicefs at a b2 bucket later today and put it through its paces. how does s3ql do with streaming media and keeping metadata straight?

@r000t yeah I didn't have good experience with s3fs either. I use s3ql to stream videos and it's great. It stores data in chunks (so no 1:1 mapping like s3fs, the s3 bucket by itself is unusable) but that allows it to be more optimized. It has local data cache, dedup, compression, encryption, etc. I have little to no buffering when streaming stuff from plex with s3ql as a backend. It's not great at handling lots of small files from my experience, but it's not my current use case so it's fine

@r000t juicefs also stores chunks and has caching, so it's looking good. And it's made with Go (vs Python)

Sign in to participate in the conversation
Mastodon

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!