2006/08/03

ISA Server 2004: fwsrv stopped responding to all requests

We recently installed ISA Server 2004 Service Pack 2 and then applied the recommended update (point 3 is KB897716) Microsoft Internet Security and Acceleration (ISA) Server 2004 Standard Edition RPC Filter Blocks Outlook Traffic from Computers Running Windows Server 2003 Service Pack 1 (SP1) After doing it we did not have to restart the server and everything seemed to be working perfectly. I also read about BITS Caching (Background Intelligent Transfer Service, used by windowsupdate) being supported and decided to use it (you can read about how to do it searching for Creating the Microsoft Update Cache Rule in Planning, Deployment, and Integration for ISA Server 2004 Service Pack 2). Some days later, due to other reasons, we had to restart the server. After that, the Firewall Service failed to start, thus leaving us disconnected from the internet. All netwok traffic (VPNs, http, email) was blocked. As you can imagine, we were into real trouble. In our application event log there were 2 kind of error messages:
Event Type: Error
Event Source: Microsoft ISA Server Control
Event Category: None
Event ID: 14079
Date:  8/2/2006
Time:  12:30:01 PM
User: N/A
Computer: SERVERNAME
Description:
Due to an unexpected error, the service fwsrv stopped responding to all
requests. Stop the service or the corresponding process if it does not
respond, and then start it again. Check the Windows event Viewer for
related error messages.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Event Type: Error
Event Source: Microsoft ISA Server
Event Category: None
Event ID: 1000
Date:  8/2/2006
Time:  12:28:55 PM
User: N/A
Computer: SERVERNAME
Description:
Faulting application wspsrv.exe, version 4.0.2165.610, stamp 442d48f1,
faulting module w3filter.dll, version 4.0.2165.610, stamp 442d48dd,
debug? 0, fault address 00094cff.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
In the system event log we had:
Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7034
Date: 8/2/2006
Time: 12:29:08 PM
User: N/A
Computer: SERVERNAME
Description:
The Microsoft Firewall service terminated unexpectedly. It has done this 6
time(s).

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
With those clues, we found The Firewall service stops responding and Event IDs 14079, 1000, and 14057 are logged in the Application event log in ISA Server 2004 that suggested installing Update for HTTP issues in Internet Security and Acceleration Server 2004 Service Pack 2. We did it but it did not solve the problem either. One of the symptoms in the latter document was:
917134 The "Background Intelligent Transfer Service" option is incorrectly available for any non-Microsoft Update cache rule that you create in ISA Server 2004
I revised the cache rules that I had configured, but the only one that had the BITS Cache enabled was the one created by the wizard. None of the rest had that option enabled. Just in case, I deleted the Microsoft Update Cache Rule that I had created some days earlier. No luck either: the service kept on stopping. And then I had another idea: since all the clues guided me towards the cache... why not to disable the cache completely, delete c:\urlcache\Dir1.cdat file manually, and then re-enable the cache again? If the problem was the data that ISA had already saved in the cache, that would be the only way to get rid of it. The idea was quite simple and seemed risk-free. We had not read a word about this self-made procedure but after doing it, the Firewall service restarted without any problem. Summing up:
  1. Install ISA Server 2004 SP2.
  2. Install KB897716.
  3. Install KB916106.
  4. Do NOT add any cache rule with BITS Caching enabled (just in case).
If you run into the same problems as I did:
  1. Remove any of the rules that had BITS Caching enabled.
  2. Disable caching for all drives.
  3. Manually delete the cache files.
  4. Re-enable the caching for the drives you had previously configured.
Links:

4 comments:

Anonymous said...

Excellent post... your steps worked perfectly and saved me very fast!... Thank you!!!!

E Turnipseed
Minnesota

Anonymous said...

Really Thx for your sharing!

Anonymous said...

Wonderful.
Thanks alot!

Anonymous said...

Well I must say that your steps have worked perfectly for me. Thank you for posting.