Mount AWS S3 bucket to Ubuntu file system

on S3FS, AWS S3, Mount S3 Bucket, DevOps, Ubuntu

Do you know how people use AWS S3 if they don't have any helps from libraries (paperclip, carrierwave...) and they also don't know how to call AWS S3 API to upload file?

You want to use AWS S3 to quicky store your files on cloud, how can you do?

Solution

S3FS is a solution to resolve above issues, it helps us to mount our S3 bucket as a normal folder in our file system.

Install

I cannot find any Ubuntu PPA that has latest S3FS, so we will need to compile it from source to have latest version

sudo apt-get install build-essential git libfuse-dev libcurl4-openssl-dev libxml2-dev mime-support automake libtool
sudo apt-get install pkg-config libssl-dev
git clone https://github.com/s3fs-fuse/s3fs-fuse
cd s3fs-fuse/
./autogen.sh
./configure --prefix=/usr --with-openssl
make
sudo make install

S3FS needs AWS access key id and secret key to work. Create .passwd-s3fs file at ~/.passwd-s3fs with content

<AWS Access Key ID>:<AWS Secret Access Key>

Example

AKIAJAXZBXCKFAOWRE4A:K6Az0uBN76I8xIHArdkr5gvrNQLR1fE3m8OywhJB

Mount AWS S3 bucket to file system

mkdir /tmp/cache # To be used as cache for S3FS
chmod 777 /tmp/cache
mkdir /mnt/s3 # To mount to, use any path you want
s3fs -o use_cache=/tmp/cache mybucket /mnt/s3 # With mybucket is your bucket name

Now, you can use /mnt/s3 as normal folder, files added/removed in it will be synced to S3 automatically.

Tips
  • If you are using S3FS for web app, you will need to allow apache or www-data or ... (user runs web server) access mounted S3. Add option allow_other to archive this, so the command will be

    s3fs -o allow_other,use_cache=/tmp/cache mybucket /mnt/s3

  • In most cases, you will want S3 to be mounted automatically after each reboot. So, add below line to /etc/fstab. Please note that fstab is run by root user, so you need to copy .passwd-s3fs to root home directory if it was not there (or you can use /etc/passwd-s3fs for system-wide configuration)

    s3fs#mybucket /mnt/s3 fuse allow_other,use_cache=/tmp/cache 0 0

  • If you have more than 1 bucket to mount to same system, you will need to use startup script instead of fstab because if you use fstab, only 1 bucket will be mounted. With Ubuntu, using local.rc may be the best choice for startup script


A full stack developer with Ruby on Rails as the main framework. Eager to learn and share.