Beating cache issues for static content

Modern web applications are complicated and consisted of different pieces including css files, javascript files, images, AJAX calls, html pages and dynamic content served by different web framework. Web browser are responsible of loading static content as well as dynamic content of the application. To help speed up this process many corporation use proxy servers to cache static content from provider. This make sense, because request from user’s browser instead of traveling all the way across the globe to get static file, can hit local proxy and get content instantly.

This has a downside however: when changing your application, you want to make sure your users get the latest static content. The only way to do that is to change the name of these files (for example by adding a time stamp to their file name, or using their checksum as their file name) You must also propagate that change to all the files that refer to them.

If you call you static content from php file or .jsp page, then you can build mechanism of changing href into your server logic. But if you refer your files from plain html file then you have no choice but to change all file names and references during the build process.
For example, you have to rename your static file widget.js to something like static_1121.js or 12727334.js
and inside html file change href src=static.js to href src=12727334.js
As soon as new version of the file available, a new name and reference get generated.

In order to do it you have to have a solid build process to parse hundreds of files. For my deployment I use
tested Ant script and Groovy. Groovy ant task give me much more flexibility to code deployment scripts.

FileTransform to the rescue
for this task I use custom FileTransform (see FileTransform.java) The first step is to build the task from the source and define the custom task. This task was developed by Yahoo team and works for me very well.
So first you have to define the task inside you ant script or Groovy snippet:

<target name="-setup.build.tools" depends="-load.properties">
    <mkdir dir="${build.dir}/tools/classes"/>
    <javac srcdir="tools/src"
           destdir="${build.dir}/tools/classes"
           includes="**/*.java">
        <classpath>
            <pathelement location="tools/lib/ant.jar"/>
        </classpath>
    </javac>
    <taskdef name="FileTransform"
        classname="com.yahoo.platform.build.ant.FileTransform"
        classpath="${build.dir}/tools/classes"/>
</target>


for Groovy:
<groovy>
	ant=new AntBuilder()
        ant.taskdef(name:"FileTransform",
classname:"com.yahoo.platform.build.ant.FileTransform")

Then we are going to replace all js file names with new one and store mapping between old and new in property file:
<target name="-copy.js.files">
<FileTransform todir="${build.dir}/WebRoot"
 changefilenames="true"
propertiesfile="${build.dir}/js.files.mapping.properties">
<fileset dir="${build.dir}/WebRoot">
<exclude name="dojo110/dojo/*.js"/>
<include name="*.js"/>
</fileset>	
</FileTransform>

Groovy code for the same thing:

ant.FileTransform(todir:"${properties.'app.dir'}/WebRoot",
changefilenames:"true",propertiesfile:
"${properties.'app.dir'}/js.files.mapping.properties") {   fileset(dir:"${properties.'app.dir'}/WebRoot") {
exclude(name:"dojo110/dojo/*.js")												include(name:"*.js")
/view/textresizedetector.js")
}
}

Then we have to replace all references to .js file inside our html files:

<replace dir="${build.dir}/WebRoot" replacefilterfile="${build.dir}/css.files.mapping.properties">
<include name="*.html"/>
</replace>

Groovy:
ant.replace(dir:"${properties.'app.dir'}/WebRoot",includes:"**/*.html",replacefilterfile:"${properties.'app.dir'}/../js.files.mapping.properties")

Replacefilter file is a key element here.
Now you application is ready for packaging into .ear archive or ssh over to your servers.
Anytime developer change something in .js or .css files, your build process will assign a new file name to the file and change references inside html files. After deployment user will get fresh new content, regardless of any proxy cache. Here what you will see in firebug:


Mapping property file:

art_collection.js=4168727054.js
art_display.js=2289106769.js
jquery.validate.pack.js=3876493005.js
art_viewer.js=2007022805.js
art_display.js.uncompressed.js=2862563724.js
greybox.js=2688020088.js
main.js=3117990409.js

Inside static html file:

<script type="text/javascript" src="dojo110/view/236151651.js"></script> 
<script type="text/javascript" src="dojo110/2256730315.js"></script> 

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>