How to Mount S3 Bucket as File System

Background:

S3 the AWS service for object storage, it’s not using POSIX protocol and hence we can’t mount it directly, however, there are some open-source tools that help in mounting your S3 bucket as a standard file system.

Why Mounting S3 as File System?

in many cases, you may have an application that has no S3 support, or it can only deal with a local filesystem, also you may have distributed application where it’s hosted under multiple servers and they all need to read or write on these s3 files. for example configuration files or media files. also if you have non AWS environment then you want to be able to use a service like EFS to be mounted under multiple servers at once.

Mounting S3 Tools:

Here is a list of the common tools with our opinion for each tool:

S3FS:

Source: https://github.com/s3fs-fuse/s3fs-fuse

s3fs is the most popular tool and it supports both Linux and macOS, we found from our experience that its the most stable tool and it has many configurations.

however, if you have a large set of files then you need to be careful as it created a connection per each file which becomes a burden on the server and can create many connection errors.

RioFS:

Source: https://github.com/skoobe/riofs

RiosFS is another cool feature that handles connections better than S3FS and it’s stable, however, it does not support renaming folders or updating files, so it mainly supports list, view, create, and delete but not update.

if your tool does not require any updates to S3 objects so this is our recommendation to use.

ObjectFS

Source: https://objectivefs.com/

ObjectFS is a scalable service that supports high performance and encryption to your S3 objects, however, it’s not open-source.

goofys:

Source: https://github.com/kahing/goofys

a high-performance, POSIX-ish Amazon S3 file system written in Go, it claims to have better performance, however it’s not a real filesystem or 100% POSIX compliance support, foe example Goofys does not have an on-disk data cache, and consistency model is close-to-open.

Conclusion:

From the list above we can see, there are a few options to mount S3 as Filesystem, however, from our long experience, we highly recommend you to use EFS  if you are on AWS environment or using s3cmd by creating a cron job to synchronize your folder with S3.