Skip to main content

Hero Write-up: Now this is Customer Service!

My best friend Scott is president/C-everything of a small northeastern Ohio manufacturing concern, KirkKey Interlock. I hadn't spoken with him for a while and wanted to see how Canton fared with the Blizzard of '08 (that's what they're calling it...not me).

I say, "So what's new?" He replies that on Tuesday his primary server (which essentially runs the business) came up with lame with not one, but [a statistically improbable] *two* physical disk failures on a RAID5 hardware array. My friend attempts the fix but gives up pretty quickly after seeing some Linux nasty-grams on the boot screen.

His service provider is an old college buddy who lives down in Raleigh, Cerient Technologies led by Jason Tower. Scott couldn't email out because Exchange was on the toasted server. Being creative, Scott started Treo-emailing photos of the screen. Unfortunately, Jason couldn't receive email because a storm had knocked out a lot of local hosting. [Sigh]

After understanding the server's disks were hosed, Jason hops in his Mazda and drives the 10 hours to Canton overnight. He arrives Thursday around 6AM and without having slept the night before, works until he falls asleep Friday morning just after midnight.

Miraculously, Jason rebuilds the array. I'm no MCSE but I thought this was impossible if you lost 2/5 disks. Anyway, he fixed the array, replaced the disks, and rebuilt--*no* data loss. Wow.

About the time he finishes, the Blizzard of '08 hits. So Jason is stuck in Canton, Ohio through Saturday and is making a run for home today...but it's unlikely he'll get through the mountains of West Virginia on I-77. Further, he's under the gun because a lot of his clients will implode because of how their systems might react to daylight savings. Oui vey!

Anyway, big props to Jason for going above and beyond to service his client. He's a service provider I think we all could emulate and learn from. Safe travels, Jason.

Comments

I'm one of Jason's "local" clients in the Raleigh, NC area. This story is not surprising - not only the fact that he would make the extra effort to get the job done, but that he miraculously saved the data on a catastrophic failure. Quite a knowledgeable guy and dedicated to his "craft".

Caveat: Just don't ask him how to do something on a Windows box ;)
Jeff Hunsaker said…
@Brian I hear ya! Ironically, the implementation at my buddy's firm leverages VMWare on Linux hosting multiple *Windows* VM instances. Say that five times fast.
Oh I know... my comments were in jest. Jason has actually saved my Windows VM's several times as well. Furthermore, if he runs into a problem that he isn't comfortable with, he will either learn it or find an expert to help.

Popular posts from this blog

Get Your Team Foundation Server Hate On!

[Google ranking skyrockets... ;-)]

I'm a big fan of TFS/VSTS. However, there are a good pocket of folks who take issue with the way TFS handles or implements a certain feature. Well this is your chance to vent!

I'm planning a presentation around the "Top 10 TFS/VSTS Hates and How to Alleviate Them"...or something along those lines. But I need your help. Post a comment below detailing your dislike. If it's legitimate, I'll highlight it in the presentation and [hopefully] provide an alternative, resolution, or work-around.

Thanks in advance!

Update 7/19/2008: Version Control and Microsoft

Rollback a Ooops in TFS with TFPT Rollback

Rhut roe, Raggie. You just checked in a merge operation affecting 100's of files in TFS against the wrong branch. Ooops. Well, you can simply roll it back, right? Select the folder in Source Control Explorer and...hey, where's the Rollback? Rollback isn't supported in TFS natively. However, it is supported within the Power Tools leveraging the command-line TFPT.exe utility. It's fairly straightforward to revert back to a previous version--with one caveot. First, download and install the Team Foundation Power Tools 2008 on your workstation. Before proceeding, let's create a workspace dedicated to the rollback. To "true up" the workspace, the rollback operation will peform a Get Latest for every file in your current workspace. This can consume hours (and many GB) with a broad workspace mapping. To work around this, I create a temporary workspace targeted at just the area of source I need to roll back. So let's drill down on our scenario...
I'm workin…

Shrinking WSS (Sharepoint) SQL Server Log Files

Yesterday, while migrating a source repository from StarTeam to TFS, I received the following error:
"TF30042: The database is full. Contact your Team Foundation Server administrator."Excuse you? Sure enough, my 100+ GB drive was full on the server. But I'd only migrated around 1000 items. Surely SQL wasn't consuming 100MB per file.

Turns out (yes, there was a lot of crud on the drive but...) the majority of the space, almost 40GB was being consumed by the Windows Sharepoint Services WSS Content data and log SQL Server files. Huh? I still need to investigate and understand why this portal, which is 100% unused, grew so large. Regardless, here's what I did to resolve:

Since this is not yet a production database, I flipped the SQL recovery option from Full to Simple for WSS Content and several other databases. Detail here and here.
Executed the maintenance plan for all the databases to get backups and clear out some of these files. That didn't help much. The Auto_…