Showing posts with label s3. Show all posts
Showing posts with label s3. Show all posts

Wednesday, November 24, 2010

Downloading EXE files from S3

It is more than a little frustrating that Internet Explorer and Windows cannot do anything to match what the other major browsers and operating systems are doing, or so it seems.  Internet Explorer has long been the browser that most developers hate to support.  Today, I ran across something that wasn't HTML or CSS related but still caused issues on Windows using Internet Explorer.
As many of you know, I use Amazon Web Services to do much of my work and S3, Simple Storage Service, is a large part of what I work with.  Today, I was made aware that someone could not download an EXE file from S3 onto their desktop.  I tested it out, and it was fine for me.  Classic developer response right? Turns out, it worked for me because I was using Chrome on my Mac.  It didn't work quite right in Internet Explorer on Windows.  The file would download, but it would not bring down the extension to the file.  What I mean, is if the file was stored in S3 as "filename.exe", when it was downloaded to the Windows machine it would only be named "filename".   I was encouraged by some people that I know to simply chalk it up to a security concern, and while I do not encourage anyone downloading random .exe files and installing them, I knew that this just couldn't be the case.  As it turns out, Internet Explorer must be told the Content-Disposition of the file when it is an EXE or DLL.  This is done in the headers when the file is requested.  Luckily, S3 allows you to put metadata on a file that is passed down when the file is requested.  To allow Internet Explorer to recognize and save the file with the .exe extension you will just need to add the Content-Disposition:attachment; filename=<filename.exe> header.




You can do this as metadata in S3 through the AWS console.  Many of the software utilities that work with S3 also allow you to set this metadata programmatically when storing the objects.
I am used to all sorts of JavaScript, HTML, CSS problems when working with IE, but this one really threw me for a loop at first.  I hope that this helps you.

Sunday, April 19, 2009

Tips on Using Amazon CloudFront

If you are serving a lot of content with an audience that is spread out over the globe, then most likely you are in need of putting your content on a CDN (Content Delivery Network).  There are a host of options for you such as LimeLight, BrightCove and Akamai to name a few, but I have found that Amazon offers much of the same functionality at a better price.  Moving your static content to Amazon's CloudFront can really reduce the load on your servers and certainly enables you to serve your content from a place that is closer to the request.  The best thing about it is that it is very very simple to do.  There are some pieces of information that can really save you alot of time and energy if you are converting your site to serve content from CloudFront.  Here are some things that were important to me and that I had to address when moving our site:
  1. Speed
    • Faster delivery of all content from edge locations close to the request
    • Faster load time of a page in the browser
  2. Ability to serve content over HTTP and HTTPS
  3. Simple, straight-forward development
Speed
Amazon CloudFront allows you to manage the content in your S3 buckets and enable them for delivery from edge locations all over the world.  To enable a bucket for CloudFront, it is a simple API call to create a Distribution.  Once your Distribution is provisioned, which usually takes less than 15 seconds in my experience, you will be given a Distribution url which will look something like: http://abcd1234.cloudfront.net .  This Distribution URL can be simply though of as another URL to that bucket in S3.  For instance, if you have a bucket in S3 named "mys3bucket" and you create a Distribution for that bucket, which returned http://abcd1234.cloudfront.net, then you could reference that bucket as http://s3.amazonaws.com/mys3bucket/ or as http://abcd1234.cloudfront.net .  The only difference between the two is that the CloudFront URL will be served from the location closest to the request and the content from that distribution can be served at 1,000 Megabits per second and even more if needed and requested.  For the easy creation of your S3 Buckets and CloudFront distributions, I suggest using S3Fox for Firefox.  With a simple right click on your bucket through S3Fox, you can create a new Distribuion in seconds.
Beyond just offering the serving of your content from an edge location at high throughput, CloudFront distributions have another advantage that can speedup the loading of your sites pages.  Using a single Distribution, you can assign up to 10 CNAMEs to it.  This can really make a large difference when serving content for a page.  Most browsers block and only allow 2-4 parallel connections to a single host when loading content.  Creating 10 CNAME entries all pointing to the same distribution allows the browser to make 10 times the parallel connections it could using a single entry.  I would suggest Steve Souders book High Performance Web Sites: Essential Knowledge for Front-End Engineers for more tips on this.

HTTP and HTTPS Delivery
One of the drawbacks to CloudFront is that it does not support HTTPS delivery of the content.  However, there is some good news here because S3 does support HTTPS.  Because the content that you need to deliver over CloudFront sits in an S3 bucket, then you can also deliver that same content over HTTPS but you cannot take advantage of the edge locations.  This is especially useful when you need to serve a secure site that has a bunch of images that you are already serving from CloudFront.  If you try to serve those images over HTTPS, your users will get a security warning or no image at all because the browser will block it.  So, when you need to serve those secure pages, simply switch your delivery URL to the S3 location instead.  A word of advice here is to create your S3 bucket without using any '.'s.  I know that this really messes with you if you want your URL to look like it is coming from your servers, but it will allow you to serve secure content from S3.  The S3 bucket is served over HTTPS using a wildcarded certificate.  This allows Amazon to serve your content using any prefix as long as it does not have any extra dots ('.').  If your bucket name is 'mys3bucket', then you can create a CNAME record that points 'mys3bucket' to mys3bucket.s3.amazonaws.com.  Because the wildcarded SSL certificate is to *.s3.amazonaws.com, then you can serve your content from https://mys3bucket.s3.amazonaws.com.  While this isn't the perfect solution, it is really nice for serving the same bucket content over HTTP or HTTPS.


Simple, Straight-Forward Development
I am not sure that building scalable applications is ever simple, but I am always looking for ways to make it extremely repeatative and simple for myself and other developers. Because we are using Java for most of our applications, the easiest way for me to do that was through a tag library.  I included all of the logic for switching between the different CNAMES for CloudFront delivery and logic for detecting a secure connection and need to switch to straight S3 delivery right in the tag.  There were some interesting details that were necessary to take full advantage of CloudFront.  One was coming up with a hash scheme for a URL so that it could be cached in the browser and distributing the calls across the CNAMEs evenly.  The second was to make sure that the tag was written so that it was at the request level instead of the page level.  This helped ensure that even an included JSP would not cause a narrowing of the number of CNAMEs that I could use.  This is the method in the tag that does most of the work
protected String getCDNUrl() {
   String result = null;
   if (((PageContext) getJspContext()).getRequest().isSecure()) { 
      result = getCDNSecureUrl();
   } else {
      result = getCDNUrls().get((1 <= getCDNCount()) ? Math.abs(getFile().hashCode()) % getCDNCount() : 0);
   }
   return result;
}
CloudFront has proven to be very useful to me.  I hope with a few of these tips that you can improve the performance of your site.  Please share your experience with me or any other tips that you might have!

Wednesday, November 19, 2008

Amazon CDN CloudFront

That was fast!  It was just a couple of months ago that Amazon told us that they were working on a CDN.  Yesterday they took another step and told the world that it has arrived.  The new service is called CloudFront and it works in conjunction with S3.  To get started, you simply need to have content that you have placed into a publicly accessible bucket on S3 and a couple of API calls later, you have content that accessible from the cloud in CDN style.  Your S3 content is served from the closest location to the requester and is stored there for a period of time so that subsequent calls can be served quickly from that location.  The great thing is that it is not limited to the Seattle or DC locations for serving the content.  There are many other datacenters across the US (and the world) that are capable of serving the content helping to ensure speedy access.  I can only imagine that this will have other CDN providers very nervous.  They are already dropping their prices dramatically and this will squeeze them even further.

Friday, September 19, 2008

Amazon S3 as a CDN?

It is coming... Amazon announced today that they are currently working with a small group of private beta customers on a edge network solution that will allow users of S3 to deliver the content to their users with low latency and high speed making use of what they call a "global network of edge locations on three continents to deliver your content from the most appropriate location". It is well known that they are currently serving their content from data centers in Washington DC, Seattle, and in the EU. I am wondering where their other locations are to make up the global network. That will be interesting to see. We are currently serving customers around the globe and have seen good performance most of the time, even to our customers in Australia. I am hoping that this will make our service that much more effective. The press statements talks about using public buckets for the service. I am hoping that this will also be available for private buckets as well!  If Amazon can keep the costs way down, they will really start to disrupt the CDN market.  We are getting several vendors calling us from CDNs and the prices all seem to go lower with each call.  Let's see what this does.

Thursday, August 21, 2008

Amazon Web Services adds Elastic Block Store


We have known about it for months, but very few have been able to play with it until now. Several months ago Amazon started talking about "Persistant Storage" becoming available on their platform and today it is available to everyone. It is labeled as Amazon Elastic Block Storage (EBS). At DigitalChalk we have been looking forward to this for several reasons. The most immediate benefit that I see getting from this is the ability to have a block of storage that we can put the data store on for our databases. Right now we have very frequent and bandwidth intensive backup solutions in place that snapshot our data and put it into S3. With ESB we will be able to allocate a block of storage up to a TB in size and then simply attach that to any running EC2 instance, essentially making it a SAN in the clouds. Amazon states that "each storage volume is automatically replicated within the same availability zone" helping with the backup scenerios and reliability of data. It doesn't seem to have it across data centers, but this is at least going to help prevent failure due to a single piece of hardware. They have also included the ability to create a "point-in-time snapshot of volumes" which will be extremely useful for the movement of data and quickly standing up a duplicate instance for testing of any sort. This is also going to be helpful as we continue to consult and help companies move their infrastructure to the clouds. We are finding that many we help have a hard time mentally moving from the physical box where they can just add hard drive capacity and rely on a file system. This will allow us to gradually and easily move their applications "as is" to the clouds without having to introduce S3 from the start for reliable storage. I am looking forward to using this service to improve our process at DigitalChalk. The next on the list to come out of the private beta is SimpleDB for me. We have already architected some solutions for SimpleDB and and just itching for its release.

Wednesday, February 13, 2008

Amazon Web Services


One of the best things that we have done at DigitalChalk is to make use of Amazon Web Services. Take a look at this story that was posted today by Amazon about DigitalChalks use of their web services. Using Amazon's services has helped us in many ways. Not only has it provided a very cost effective method for delivering to our customers, but it has also given us a known, reliable backbone that we can depend upon. Knowing that we are running on the same infrastructure that Amazon is running on gives me great confidence in our reliability and uptime. The services have proven to be easy to use with a very active forum and user community. So much so, that I tell everyone that I know about the technologies we are using that Amazon provides. Right now, Amazon provides server virtualization through EC2, file, data storage and backup through S3 and a messaging service through SQS which help keep DigitalChalk running around the clock. I would highly encourage you to sign up and start playing around with the services for yourself. Amazon is taking Software as a Service to a new level providing these services to anyone that wants to use them. If you have any questions at all about how you can use them as well or need help, send me a note. I would be happy to talk about how it has helped us and how it can help you or even help you get started.