Stack Overflow
Stack Overflow

Questions tagged [s3fs]

s3fs is a FUSE filesystem which allows to mount an Amazon S3 bucket as a local filesystem.

s3fs allows Linux and macOS to mount an S3 bucket via FUSE. s3fs preserves the native object format for files, allowing use of other tools like s3cmd.

s3fs is stable and is being used in number of production environments, e.g., rsync backup to s3.

Important Note: Your kernel must support FUSE, kernels earlier than 2.6.18-164 may not have FUSE support (see issue #140 ). Virtual Private Servers (VPS) may not have FUSE support compiled into their kernels.

There is also a Python library called s3fs. For questions related to this library, please use .

194 questions
142
votes
8 answers

Set cache-control for entire S3 bucket automatically (using bucket policies?)

I need to set cache-control headers for an entire s3 bucket, both existing and future files and was hoping to do it in a bucket policy. I know I can edit the existing ones and I know how to specify them on put if I upload them myself but…
thattommyhall
  • 1,539
  • 2
  • 10
  • 7
73
votes
2 answers

How stable is s3fs to mount an Amazon S3 bucket as a local directory

How stable is s3fs to mount an Amazon S3 bucket as a local directory in linux? Is it recommended/stable for high demand production environments? Are there any better/similar solutions? Update: Would it be better to use EBS and to mount it via NFS to…
arod
  • 11,893
  • 5
  • 25
  • 33
54
votes
2 answers

How can I mount an S3 bucket to an EC2 instance and write to it with PHP?

I'm working on a project that is being hosted on Amazon Web Services. The server setup consists of two EC2 instances, one Elastic Load Balancer and an extra Elastic Block Store on which the web application resides. The project is supposed to use S3…
Ben Harold
  • 5,634
  • 5
  • 43
  • 69
26
votes
7 answers

Amazon S3 with s3fs and fuse, transport endpoint is not connected

Redhat with Fuse 2.4.8 S3FS version 1.59 From the AWS online management console i can browse the files on the S3 bucket. When i log-in (ssh) to my /s3 folder, i cannot access it. also the command: "/usr/bin/s3fs -o allow_other bucket /s3" return:…
ilansch
  • 4,484
  • 5
  • 40
  • 83
14
votes
2 answers

How to mount S3 bucket on Kubernetes container/pods?

I am trying to run my spark job on Amazon EKS cluster. My spark job required some static data (reference data) at each data nodes/worker/executor and this reference data is available at S3. Can somebody kindly help me to find out a clean and…
Ajeet
  • 535
  • 1
  • 5
  • 15
14
votes
2 answers

Is s3fs not able to mount inside docker container?

I want to mount s3fs inside of docker container. I made docker image with s3fs, and did like this: host$ docker run -it --rm docker/s3fs bash [ root@container:~ ]$ s3fs s3bucket /mnt/s3bucket -o allow_other -o allow_other,default_acl=public-read…
kochizufan
  • 1,664
  • 2
  • 18
  • 38
14
votes
5 answers

Allowing permission using S3FS bucket directory for other users

I'm having problem using S3FS. I'm using ubuntu@ip-x-x-x-x:~$ /usr/bin/s3fs --version Amazon Simple Storage Service File System 1.71 And I have the password file installed in the /usr/share/myapp/s3fs-password with 600 permission. I have succeeded…
Petra Barus
  • 3,414
  • 5
  • 38
  • 76
10
votes
4 answers

Mount S3 bucket as filesystem on AWS ECS container

I am trying to mount S3 as a volume on AWS ECS docker container using rexray/s3fs driver. I am able to do this on my local machine, where I installed plugin $docker plugin install rexray/s3fs and mounted S3 bucket on docker container. $docker…
Pratik Mungekar
  • 111
  • 1
  • 1
  • 5
9
votes
1 answer

libcurl.so.4: cannot open shared object file: No such file or directory

I am creating binary of s3fs-fuse on Ubuntu 14.04 by following the compilation link. In some systems I get the below error while doing the mount operation on kubernetes pods ---- ------ ---- ---- …
ambikanair
  • 2,563
  • 5
  • 33
  • 56
8
votes
3 answers

Change user ownership of s3fs mounted buckets

how can I modify the user:group ownership of a s3fs mounted bucket? I have a git installation that I would essentially like to store on my Amazon S3 account in a bucket, and then using Sparkleshare, via my web host, sync this data accross multiple…
itsricky
  • 387
  • 1
  • 3
  • 8
8
votes
3 answers

AWS: Mount S3 Bucket to an EC2 instance. (Later FTP Tunneling)

what do I want to do? Step1: Mount a S3 Bucket to an EC2 Instance. Step2: Install a FTP Server on the EC2 Instance and tunnel ftp-requests to files in the bucket. What did I do so far? create bucket create security group with open input ports…
Timo
  • 183
  • 1
  • 2
  • 12
8
votes
3 answers

How can I mount an S3 volume with proper permissions using FUSE

I have an Amazon S3 bucket (let's call it static.example.com) that I need to mount on an EC2 instance (Ubuntu 12.04.2). I've installed s3fs. I'm able to mount the volume, but I can't write to the bucket. I have tried: sudo s3fs static.example.com -o…
Ben Harold
  • 5,634
  • 5
  • 43
  • 69
7
votes
1 answer

SFTP Chroot Users to Mounted S3 bucket

I am trying to use Amazon EC2 Servers as my SFTP server where I can create authenticated users to sftp into my server. I have mounted s3 buckets onto the servers at location /mnt/buckets/{username} for each user using s3fs. Reading and writing onto…
Caleb Yoon
  • 71
  • 5
6
votes
1 answer

s3fs unmount: directory is not empty

I'm using s3fs and osxfuse to mount an S3 directory on my Mac: s3fs my-bucket-name $PWD/s3 When it's time to unmount it, I do this: % s3fs umount $PWD/s3 s3fs: MOUNTPOINT directory /Users/kwilliams/blah/s3 is not empty. if you are sure this is…
Ken Williams
  • 19,823
  • 7
  • 71
  • 126
6
votes
1 answer

What does s3fs cache in /tmp?

I'm using s3fs to mount a lot of files to an S3 bucket. It works fine except the fact that my local disk space is also growing a lot (the content in the /tmp directory). My command is: $ su ec2-user -c '/usr/bin/s3fs my-bucket-name -o…
DenCowboy
  • 10,114
  • 24
  • 80
  • 168
1
2 3
…
12 13