In a large scale computing environment accurate time becomes a constant battle. Systems across multiple data centers must interact with each other, and often those systems rely on each servers time interpretation for transactions. Accurate time is a must, and unfortunately in Windows environment accurate time is not an easy task.

In Microsoft’s flawed NTP implementation administrators rely on time sync within their Active Directory domains.  Each Active Directory forest contains a PDC Emulator, which amongst other things serves as a source of accurate time for the forest.  PDC emulators in other domains within the forest will sync to the root domain’s PDC emulator.  Domain members then sync their own clock every 8 hours. Often this set it and forget it mentality works, but often enough it doesn’t.  The reason for this is that Microsoft in their infinite wisdom did not implement the full network time protocol (NTP), but instead implemented a subset of the full standard which they call Windows Time (aka W32TM).

W32TM works well in small-scale single data center environments, but it fails to scale out to large multi-site environments, while still retaining the necessary accuracy applications often demand. W32TM was designed to keep system clocks accurate enough for Kerberos to function within an Active Directory forest, but was never built for truly accurate time.  For Kerberos to function in Active Directory the time of systems only has to be within 5 minutes of the Domain Controller, which is far from accurate, thus why they can get away with an every 8 hour sync.  The problem is so bad that Microsoft actually admits in a MSDN article that W32TM shouldn’t be used if true time accuracy is needed:

“The W32Time service is not a full-featured NTP solution that meets time-sensitive application needs and is not supported by Microsoft as such”

W32TM is prone to inaccuracy when used across high latency network links and fails to function accurately when the individual systems become heavily taxed. The use of a single AD Forest PDC system as the time source for an organization is perhaps the largest flaw within the system. With systems spanning across multiple data centers, all time requests must travel over WAN links to the PDC emulator. Whether they be ISDN or 10GB Ethernet these links introduce significant latency that Windows Time was not designed to compensate for.

For those that require accurate time on their Windows systems it is possible to run a fully compliant NTP client on Windows system.  Meinburg, a German company that develops hardware for highly accurate time keeping, has developed a simple yet effective NTP client for Windows.  It runs as a service and doesn’t have any odd prerequisites like you might find in applications that make it from the *nix world to the Windows world.

Meinburg NTP can either be installed via the GUI installer or in a more automated fashion with an installer config file and the command line.  I’ve used the command line configuration to add Windows NTP support to the Opscode Chef NTP recipe.  I’m putting the finishing touches on this Chef recipe and hopefully it will make it into the upstream NTP cookbook release soon.   For those of you that want to automate the installation either via PowerShell or via your WDS deployment processes here’s how to setup a simple command line install:

 

1) Download the client Meinburg NTP client from http://www.meinberg.de/english/sw/ntp.htm

2) Create a ntp.conf file for your clients.  This is the same format as NTP conf files in *nix so you can simply copy this from one of your *nix systems.  Here’s my sample config

 

driftfile “C:\NTP\etc\ntp.drift”

 

server ntp.datacenter1.dmz iburst

server ntp.datacenter2.dmz iburst

server ntp.datacenter3.dmz iburst

server ntp.datacenter4.dmz iburst

 

restrict 127.0.0.1

restrict default nomodify notrap noquery

 

3) Create a file called ntp.ini.  This is what you will pass the installer to define your installation.  Here’s my sample config:

 

[Installer]

InstallDir=C:\NTP

UpgradeMode=Reinstall

Logfile=C:\NTP\install.log

Silent=yes

 

[Components]

InstallDocs=yes

InstallTools=yes

InstallOpenSSL=yes

CreateStartMenuEntries=yes

 

[Service]

ModifyFirewall=yes

ServiceAccount=@SYSTEM

DisableOthers=yes

AllowBigInitialTimestep=yes

EnableMMTimer=yes

AutoStart=yes

StartAfterInstallation=yes

 

[Configuration]

UseConfigFile=\\SERVER\path\to\config\file\ntp.conf

 

Keep in mind that I have enabled AllowBigInitialTimestep so if the system is off by a day the clock is going to jump all at once.  I’m ok with this, but it might not be acceptable in your environment.  Use caution.

 

4) Pass the ntp.ini file to the installer to silently install:  ntp-4.2.4p8@lennon-o-win32-setup.exe /USEFILE=\\SERVER\path\to\config\ntp.ini

 

5) Confirm the the installation and NTP status by running C:\NTP\bin\ntpstatus.bat

Website performance is a constant battle for any operations team.  As the proliferation of high bandwidth home and office connections continues so does the demand for near instant response times from websites by consumers.    The enormous complexity of modern websites has made this near instant response time a lofty goal, but one that can be met using a multitude of small tweaks in both web servers and web applications.

One of the easiest ways to speed web experiences is to reduce the amount of data sent to the client, while still maintaining the same client experience.  In my past blog posts I discussed http compression and reducing excessive files, which are both excellent methods to reduce the data payload of a site, but for this post I’ll be discussing Javascript and CSS minimizing.  Javascript and CSS minimizing is the reduction in non interpreted content from the Javascript and CSS in your site, and can easily shave at least 25% of the size of those files.  With the enormous amount of CSS stylus and Javascript content in a modern website a 25% reduction can add up to a huge boost in end user performance.

Javascript and CSS are rather simple markup language.  Snippets of code are placed on lines and neatly organized so that web designers can easily review their code and make modifications as necessary.  Unfortunately all that formatting isn’t actually necessary for the code to run and takes up a huge amount of space.  Comment blocks, spaces, and carriage returns can all be removed from your code to reduce the space.  Dozens of pages of CSS or Javascript code can be reduced to a single large line that is interpreted by your browser at the exact same speed as the nicely formatted file.

So what exactly does a snippet of minimized code look like?  Very plain is the answer:

 

Original code:

/*

Name: Your Product Name

Description: Handheld mobile device styles.

Version: 1

Author: Some guy at your company

Tags: handheld, mobile, iphone

 

*/

 

/*

—————————————————————-

H A N D H E L D

—————————————————————- */

@media screen and (max-device-width: 480px) { html { -webkit-text-size-adjust: none; } }

 

Minimized Code:

@media screen and (max-device-width:480px){html{-webkit-text-size-adjust:none}}

 

Notice that all comment blocks have been removed as have about a half dozen spaces that made the CSS content just a little bit easier for web designers to read.  The single line CSS content is interpreted exactly the same by all major web browsers and the user will never know the difference, except that the one line content loaded several times faster.

 

How Big is the Impact?

The impact can be enormous.  Here’s the stats of a sample corporate webpage I found that didn’t minimize code:

 

4 CSS Files:

- Default.css: 4940 bytes unminimized, 3844 bytes minimized (22% reduction)

- Style.css: 69523 bytes unminimized, 55848 bytes minimized (20% reduction)

- All-ie.css: 212 bytes unminimized, 177 bytes minimized (17% reduction)

- Profile-capture.css: 1485 bytes unminimized, 1302 bytes minimized (12% reduction)

 

JavaScript Files:

- jquery.js: 252880 bytes unminimized, 141924 bytes minimized (56% reduction)

- customeffects.js: 3230 unminimized, 1800 bytes minimized (55% reduction)

- eloqua.js: 646 bytes unminimized, 555 bytes minimized (86% reduction)

- jquery.form.js: 26750 bytes unminimized, 14525 bytes minimized (54% reduction)

- jquery-lightbox_me.js: 10577 bytes unminimized, 4117 bytes minimized (38.9% reduction)

- jquery.eloqua.js: 3123 bytes unminimized, 2571 bytes minimized (82% reduction)

- jquery.profile_caputre.js: 2272 bytes unminimized, 1527 bytes minimized (67% reduction)

- actions.js: 1000 bytes unminimized, 687 bytes minimized (68% reduction)

- priv-tools.js: 6117 bytes unminimized, 4866 bytes minimized (79% reduction)

The use of dozens of CSS and JS files in web products greatly increases the time that these sites take to load. The further a client is from the hosting server the longer the initial connection time is for each of these small files. When a server is located in the US and a client is located in Europe or Asia that initial connection time can become larger than the actual time the content takes to download. By reducing the number of JS and CSS files in a web site you can provide the same functionality, but reduce the time these sites take to load, without additional operational costs.

For this load time breakdown I will use a sample site: “Product XYZ”. The site is a modern web property, relying heavily on multiple CSS and JS files to provide an highly stylized interface. Product XYZ utilizes a content delivery network (Akamai Web Application Accelerator aka Akamai WAA) to locate these static CSS and JS files closer to customers.

CSS / JS files in Product XYZ: 42 files total

• 9 CSS files
• 33 JS files

Load Impact of a Single File in Product XYZ:
A single static JS or CSS file cached by Akamai WAA and retrieved close to the customer site has an average first byte connection time of .003 seconds. Without Akamai WAA that average time jumps to .267 seconds per file.

Total First Byte Time for Product XYZ

CSS:

• 0.021 seconds with Akamai
• 2.326 seconds without Akamai

JS:

• 0.105 seconds with Akamai
• 8.356 seconds without Akamai

Potential Improvement by Reducing File Count

If Product XYZ was to be optimized to a single JS and a single CSS file it could reduce the initial connection time within Product XYZ by .12 seconds (1.2%) when using Akamai WAA and 10.148 seconds (46%) without Akamai WAA. The real benefit to this reduction is the potential to eliminate the need for Akamai WAA caching entirely and thus greatly reduce operational costs. A non-cached Product XYZ with a single JS and CSS file would load in an estimated 11.852 seconds compared to approximately 10 seconds when using Akamai WAA caching. As Akamai bills based on data transfer this could reduce costs significantly by removing WAA caching.

HTTP Compression is one of the best bang for the buck optimizations available in Microsoft’s IIS webserver. HTTP compression works by compressing data before it’s sent to the client. Images and executables are already heavily compressed, but much of a website’s content is completely uncompressed. IIS can be setup to compress HTML, ASP, ASPX, JS, JSON, CSS, any other uncompressed content found on your web site.

Why Use HTTP Compression?
HTTP Compression first became popular in the days of incredibly slow dialup connections where shaving a few kilobytes off a website was essential. With modern high speed Internet connections it might seem like the compression / decompression time incurred when using HTTP compression would no longer make it beneficial, but there’s still a huge benefit to compression. Where compression really shines in modern web servers, are long distance connections, particularly those over high latency underwater fiber. Any site serving international customers will see a huge performance increase from compressing content prior to transmission. The latency from long haul connections greatly restricts the transmission speeds due to the time it takes acknowledgement flags from the client to reach the server. These TCP ACK flags act as a restrictor for international web transmissions and will leave your server trickling out data in small bursts. The more content that can be compressed the better as compressed data requires fewer packets and thus fewer ACK flags.

Improvements in IIS 7 HTTP Compression
HTTP Compression has long been an afterthought in IIS. IIS 5 required third party add-ons to enable compression and IIS 6 buried the necessary switches deep in the Metabase.xml file. In IIS 7 Microsoft finally provided a simple GUI method to enable both compression of static content (HTML, CSS, JS, etc) and of dynamic content (JSON, ASP, ASPX, etc). Additionally Microsoft changed much of the underlying functionality of their compression module including the method for determining what content to compress. In IIS 6 the metabase.xml file listed all file extensions that would be considered static and dynamic content. Only content on that list would be compressed, which required the list to be updated any time new file extensions were introduced. In IIS 7, compression is based on MIME types with text/*, message/*, and application/javascript being enabled by default. These 3 entries are a catch all for the majority of static and dynamic content, so out of the box most content will already be compressed. Another particularly welcome change is built in CPU based throttling that automatically ceases compression when CPU usage reaches critical levels. This feature alone makes enabling HTTP compression a no brainer for any admin, as there is no longer the worry that high cpu usage during the compression process will slow dynamic generated content delivery. Despite the improvements to the setup of http compression in IIS 7 an effective configuration still requires either manually editing your applicationHost.config file or using the new appcmd.exe configuration utility.

Compression Setup in IIS 6
First you need to enable both dynamic, static, and on demand compression. Dynamic and static are fairly self-explanatory, but On Demand compression requires a bit of explanation. When On Demand compression is enabled IIS will serve content it has never compressed in an uncompressed format to the client, and then compress the content in a background thread for future use. Enabling this feature improves response time for new content by not waiting to first compress the content before trasmitting.

cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/Parameters/HcDoStaticCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/Parameters/HcDoOnDemandCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/Parameters/HcDoDynamicCompression TRUE

Once you’ve enabled http compression you need to enable the two forms of compression supported by IIS, gzip and deflate. For each method you must specify which file extensions should be compressed as dynamic (ScriptFileExtensions) and which extensions should be compressed as static (FileExtensions). You also need to specify compression levels which range from 0-10, with 10 being the highest level of compression. Since we’ve enabled the on demand compression feature we can be a bit more aggressive with the compression levels, setting static to nine, and dynamic to a safer level of six.

cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcDoStaticCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcDoOnDemandCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcDoDynamicCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcFileExtensions “txt” “js” “css” “htm” “html” cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcScriptFileExtensions “exe” “dll” “asp” “aspx” “svc” “xml” cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcOnDemandCompLevel 9 cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/deflate/HcDynamicCompressionLevel 6

cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcDoStaticCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcDoOnDemandCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcDoDynamicCompression TRUE cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcFileExtensions “txt” “js” “css” “htm” “html” cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcScriptFileExtensions “exe” “dll” “asp” “aspx” “svc” “xml” cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcOnDemandCompLevel 9 cscript.exe C:\inetpub\AdminScripts\ADSUtil.vbs Set W3SVC/Filters/Compression/gzip/HcDynamicCompressionLevel 6

Once you’ve enabled compression and setup formats, extensions, and levels it might seem like everything would be complete. Many articles on compression in IIS 6 actually end at this step, but unfortunetly due to a bit of poor planning on Microsoft’s part there is one additional step. In order for compression to function you must create a Web Extension for compression and point it to the correct DLL. Without this nothing will work.

cscript.exe C:\Windows\system32\iisext.vbs /AddFile C:\Windows\System32\inetsrv\gzip.dll 1 “HTTP Compression” 1 “HTTP Compression”

At thit point you can run an iisreset and observe your newly compressed content.

Compression Setup in IIS 7
Enabling compression is IIS 7 became both easier and at the same time more complex. A basic setup requires far fewer commands, and a very basic setup can be completed in the GUI alone. At the same time though Microsoft introduced many additional commands for changing how IIS handles the compression of content. The first step is to install the dynamic compression module, by running the command below. Be warned it takes a while to run so be patient.

start /w pkgmgr.exe /iu:IIS-HttpCompressionStatic;IIS-HttpCompressionDynamic

Similar to IIS 6, you must enable the compression modules in IIS 7. This can be done in the GUI at the server or site level or you can run the two commands below to configure compression at the server level %inetsrv_location%\appcmd.exe set config /section:urlCompression /doStaticCompression:True %inetsrv_location%\appcmd.exe set config /section:urlCompression /doDynamicCompression:True

IIS 7 by default only enables GZIP compression, which is fine as almost all clients support both Gzip and Deflate and specify their preference for GZIP over Deflate in the HTTP header of their request. If you’d like to include support for deflate you can by running the following command %inetsrv_location%\appcmd.exe set config /section:httpCompression /+”[name='deflate',doStaticCompression='True',doDynamicCompression='True',dll='%Windir%\system32\inetsrv\gzip.dll']” /commit:apphost

As in IIS 6, IIS 7 includes compression levels from 0 to 10. IIS 7 includes additional functionality that automatically disables compression when the CPU hits high utilization. This feature allows you to set compression at more aggressive levels without compromising the response time of dynamically generated content. Despite this new functionality, a compression level of nine for static content and six for dynamic provide a balance of high compression and low CPU usage. %inetsrv_location%\appcmd.exe set config /section:httpCompression -[name='gzip'].dynamicCompressionLevel:6 %inetsrv_location%\appcmd.exe set config /section:httpCompression -[name='gzip'].staticCompressionLevel:9 %inetsrv_location%\appcmd.exe set config /section:httpCompression -[name='deflate'].dynamicCompressionLevel:6 %inetsrv_location%\appcmd.exe set config /section:httpCompression -[name='deflate'].staticCompressionLevel:9

With compression enabled and compression levels set you now need to specify additional mime types that are used by your application. By default IIS includes support for javascript and any mimetype starting with text/. You can add additional mime types by first removing the deny all rule at the end of the mimetype list and then adding your required mime types and a new deny all rule.

appcmd.exe set config -section:system.webServer/httpCompression /-”dynamicTypes.[mimeType='*/*',enabled='False']” /commit:apphost appcmd.exe set config -section:system.webServer/httpCompression /+”dynamicTypes.[mimeType='application/json',enabled='True']” /commit:apphost appcmd.exe set config -section:system.webServer/httpCompression /+”dynamicTypes.[mimeType='*/*',enabled='False']” /commit:apphost

At thit point you can run an iisreset and observe your newly compressed content.