Did you ever want to mount an AWS S3 bucket to your local macOS filesystem to sync files back and forth quickly? Maybe you're working exclusively with Amazon AWS s3 buckets through the macOS command line and need to copy and paste files to a directory. A FUSE-based file system could be what you're looking for! There are some caveats, however, so be sure to read the section S3 considerations portion of this blog. One solution for mounting an Amazon S3 bucket as a local file system exists with the third-party s3fs-fuse utility. The s3fs utility allows Linux, macOS, and FreeBSD to mount an S3 bucket via FUSE(Filesystem in Userspace).

s3fs-fuse Features

According to it’s GitHub repository s3fs-fuse has numerous features, including:

• POSIX, including reading/writing files, directories, symlinks, mode, uid/gid, and extended attributes

• Compatibility with Amazon S3, and other S3-based object stores

• Large file upload via multi-part upload

• Renaming via server-side copy

• Optional server-side encryption

• Data integrity with MD5 hashes

• Caching

• User-specified regions, including Amazon GovCloud

• Authentication with v2 or v4 signatures

Important Considerations When Using Amazon S3 as a File System

Can Amazon S3 object storage be used as a file system or drive? Yes and no. The real question is, should you? There are so many Amazon cloud services, and many times, this can lead to confusion where one selects a service that may not be ideal for your particular use case. It's important to note that Amazon S3 is a cloud storage solution that acts as object storage and not a file system solution; using S3 as a file system is considered an anti-pattern. Using S3 as a filesystem with 3rd party tools such as s3fs-fuse can have quirks, costs, and general unforeseen issues. In other words, it's less than ideal. A proper and much better solution for mounting an Amazon file system to your local machine is using a service such as AWS Storage Gateway, Amazon S3 File Gateway, or an Amazon Elastic File System (EFS).

Mounting AWS S3 Buckets on macOS: s3fs-fuse Installation & Usage 

The following section will explore the necessary steps to install s3fs on macOS. We will install macFUSE and s3fs-mac, configure AWS authentication, and mount and unmount AWS S3 buckets to our local filesystem.

Security Policy Required for macFUSE on Apple Silicon

To use s3fs on macOS we need to ensure that our environment is set up to work with macFUSE. This becomes even more important if we use Apple Silicon Macs, which requires an additional step to allow third-party kernel extensions through the harddrive security policy found in the macOS Recovery Environment.

If we use an Apple Silicon machine (macOS 13>), we may encounter the following error when installing macfuse and s3fs-mac.

This Error is due to the fact that we are attempting to load a kernel extension from a third-party. We need to allow the installation of system extensions, which requires a reboot and entering the macOS Startup Security Utility.

We can allow these extensions by following the following steps.

Enable System Extensions

We first need to click on the Enable System Extensions setting on our macOS Silicon device which is under Privacy & Security settings on our Mac computer.

Shut Down Mac

Once we have clicked Enable System Extensions we will be prompted to Shut Down our Mac computer in order to enter the Startup Security Utility. Save your work and select Shut Down.

Launch Startup Security Utility

Once our Mac is shut down, in order to access the Startup Security Utility, we need to hold the power button, which will load the macOS Recovery Environment. Once loaded into this environment, we can enter the Startup Security Utility. Once inside these settings, select Security Policy on the macOS drive that we want to enable system extensions on.

Edit Security Settings

Once inside the Startup Security Utility enable the ensure that the security policy is configured as follows.

Once the correct security policy is enabled click OK and reboot your Mac. From here we can proceed to install s3fs-fuse.

Installing s3fs-fuse (s3fs-mac)

With the most recent macOS versions, we can use Homebrew to install macfuse and s3fs-mac on our system. Since we properly configured the macOS security policy to allow system extensions we can proceed to install s3fs-fuse and macFUSE.

In a macOS terminal, enter the following commands:

We may be prompted to allow software from "Benjamin Fleisher" to be in our Mac's Privacy & Security settings. This is to enable the macfuse to extension software from the "setting up macFUSE" step.

We may be prompted to allow software from "Benjamin Fleisher" to be in our Mac's Privacy & Security settings. This enables the macfuse to extension software from the "setting up macFUSE" step.

Configuring s3fs-fuse Authentication with AWS S3

Luckily s3fs supports the AWS credentials file inside of ~/.aws/credentials. We can also use the s3fs credential file .passwd-s3fs, for which we need access to our AWS ACCESS_KEY_ID and SECRET_ACCESS_KEY values.

If you do not have an AWS ACCESS_KEY_ID and SECRET_ACCESS_KEY, you need to set them up through the AWS IAM service with the correct user permissions to interact with the AWS S3 service. You can follow the AWS documentation on configuring tool authentication with AWS.

If you've already set up an AWS ACCESS_KEY_ID and SECRET_ACCESS_KEY through a service such as the aws cli you can copy and paste them from the ~/.aws/credentials file into the following command.

Once done, we must set the correct permissions on the .passwd-s3fs file.

Once we have configured authentication with s3fs-fuse we can move onto setting up a demo S3 bucket.

Setup an AWS S3 Bucket

To demonstrate the s3fs-fuse utility, I created a basic S3 bucket called s3-mounted-bucket through the AWS GUI, which contains a single folder called share.

Next, I want to make sure I can access this bucket using the AWS CLI utility; this ensures that my AWS ACCESS_KEY_ID and SECRET_ACCESS_KEY are working correctly and have the correct permissions.

To check that I can access my bucket I can run the following aws command from my Mac terminal.

Once you've executed this command you should see a listing of your s3 bucket. If you encounter a permissions error you may need to check your IAM permissions on the user for whose access key you entered.

Mount AWS s3 Bucket with s3fs-fuse

Once authentication is configured mounting an S3 bucket to a Apple macOS machine is simple. Run the following command ensuring that you've changed the bucket name, folder, and local mount point on your Mac.

Your S3 bucket should now be mounted; you can test this by creating a simple file inside your local FUSE-mounted directory.

If we navigate to the Amazon S3 service within the Amazon GUI console we should now see the file we just copied into the S3 bucket from our local Mac command-line terminal.

Unmounting an AWS S3 Bucket with s3fs-fuse

To unmount our S3 bucket from our local file system, we can use the diskutil command. Here is an example of un-mounting my own S3 bucket:

Using the above command will unmount our S3 bucket from our local Mac.

Using AWS S3 as a Filesystem Conclusion

Utilizing an Amazon S3 bucket as a local filesystem can be an interesting solution, especially for those working extensively with S3 buckets and MacOS. Although not without its limitations and issues (the most significant being that S3 is an object storage and not a file system, this use case is considered an anti-pattern), the s3fs-fuse utility provides a solution that bridges the gap. However, given the alternatives, such as AWS Storage Gateway or Amazon Elastic File System (EFS), one should always consider whether their use case suits this particular application. This guide navigated through setting up macFUSE on Apple Silicon, installing and configuring s3fs, and testing it by creating, mounting, and unmounting an S3 bucket. We hope you'll find it beneficial for your technical needs. As always, though, due diligence is recommended to ensure that this approach fits within your technical architecture and pricing constraints.

How to Use SSHFS

Another great way to mount file systems is through the fantastic SSHFS utility, which uses the SSH protocol. I have written previously on these topics and how can find helpful information on how to use SSHFS on both MacOS and Microsoft Windows.

Other Resources

If this blog post was helpful, check out some of my other blog posts. I post routinely on general information technology, programming, and cybersecurity topics.