1

How do I integrate Amazon Cloud Front and S3 in a photo sharing application?

I currently upload to S3, and return the cloudfront url but this has not been very successful because it appears there is a latency between s3 and cloudfront such that the returned url is not immediately valid.

Does any know how I can work around this?

Facebook uses Akamai and if I upload an image it is immediately available.

Would appreciate some ideas on this.

F.O.O
  • 4,024
  • 4
  • 20
  • 31
  • 1
    There is no "latency" between S3 and CloudFront. Please provide examples, using the relevant output from a tool such as `curl -v` to document the responses that confirm what you are seeing. – Michael - sqlbot Jan 06 '16 at 11:43
  • Are you saying uploads to s3 are immediately distributed on cloudfront? I have to wait some minutes otherwise I am getting access denied. I use the Java Api to upload files. – F.O.O Jan 06 '16 at 11:44

1 Answers1

1

You must be trying to fetch the object immediately through cloudfront. I'm unsure why that might be, but you are hitting the limits of S3's eventual consistency model.

When you upload an object, the message takes a tiny amount of time to propagate across the S3 service. Generally this is well under one second and is hard to detect. (in a previous life job, we found we could reasonably guarantee all files arrived within 10 seconds, and 99.9% within 1 second)

Here's the official word from AWS; it's worth reading the whole page:

A process writes a new object to Amazon S3 and immediately lists keys within its bucket. Until the change is fully propagated, the object might not appear in the list.

There's a much longer discussion on this stackoverflow question; assuming you are using the standard S3 bucket, you need to change your endpoint slightly to take advantage of the read-after-write model.

Further reading: * Instrumental: Why you should stop using the us-standard Region in S3. Right Now™ * Read-After-Write Consistency in Amazon S3 (from 2009, contains dated info)

One way you can debug/prove this is by calling getObjectMetadata right before your CloudFront call. It should fail in this case.

tedder42
  • 20,195
  • 8
  • 74
  • 92