That’s all I have to say on the matter.

My company is rapidly growing. We’re in the process of replacing our current ERP system with a shiny new Oracle-based one in the next 10-12 months. While that happens, the company will grow another 40 percent.  Our mandate from management is “Make it scale!” We have existing applications that need to grow and new applications that need to be put in place.

A friend referred to me as a jack-of-all-trades DBA. I think he’s onto something. As the only DBA in the company, I do it all. Or at least most. That means everything from managing applications to performance tuning to server management to scaling applications that we’ve outgrown. I’m also working on a consolidated reporting instance as well as designing a short-term data warehouse solution.

Let me explain that our entire IT infrastructure team is four people, and that includes our Director of IT. It means we all wear several hats. We have a network/sever guy, security/network guy, and a DBA. If it touches a database or database server, I’m involved.

Part of this growth (and a looming data center move) required virtualizing everything. And I mean everything. The only database server that hasn’t been virtualized is more of an appliance that runs on vendor-provided hardware. Every other sql server is virtual. During our virtualization project, we consolidated 14 servers down to four primary servers.

Another thing I’m doing is making systems scale that were never meant to. We have a homegrown Access “application” used by our manufacturing organization. It was this little database that is used to capture data for hardware testing. Like most applications of this vein, it grew into a beast. And somehow, it grew into a business critical beast. It had become the authoritative data source for products shipped. Recently, while looking at the design of the thing, I realized that the DBA who upsized the thing didn’t use clustered indexes. It was a HEAPing mess.

Most recently, I’ve been tasked making this database talk to its little brothers at some of our business partners’ locations. These remote locations need to get updated information about customer orders, parts, and BOMs.  Not only that, we need to remote locations to report back to the mother ship.Historically, someone established a VPN connection and did this manually. Considering our tight network policies, automating this is quite a challenge.

Our existing collaboration/content management tool just isn’t going to work. We’re implementing Sharepoint. And we’re estimating the content databases to reach 500 GB within the next 18 months. I’ve had to prepare the database servers accordingly. This means content databases have to span multiple data files and multiple disks.

On top of all of this, we’re publicly traded and bound by Sarbanes Oxley. That means a good chunk of my time is spent satisfying auditors.

usermike doesn’t have access to crap, especially database servers. Let me put it this way, Claire in Marketing has more access on the network than usermike.

As a DBA, this situation can be hell. When I first started, I was launching a remote desktop session to my SQL servers in order to run SQL Management Studio. The only thing worse than running a remote desktop session to a SQL server is running SSMS in that remote desktop session. When we bought a new SQL tool, I realized that keeping RDC sessions open all day just wasn’t the solution. I went looking for a better solution.

One day, I stumbled accross the RUNAS command.  Basically, from a command line, I can launch an application and state the user that should be used to run the application. In my case, dbamike runs the application. It will prompt me for that user’s password, and off we go.  It looks something like this for SSMS.

runas /user:domain\dbamike "C:\Program Files\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\Ssms.exe"

After putting this in a batch file on the desktop, life became a lot simpler. My productivity soared.

Like most things, there is a gotcha. When I attempt to save a file, it will save it in dbamike‘s My Documents folder. That’s because the application is running as dbamike. It also means I can’t open files directly from usermike‘s Outlook mailbox. I have to save it to c:\mike first. It’s an extra step, but the trade-off is wel worth it.

I think this is a smart business decision for the company, and I’ve declined my boss’ offer for Oracle training. The company’s technology path and my career are going in opposite directions. I don’t want to be an Oracle DBA, and I don’t want to be cutting my teeth on a new technology during an ERP implementation. It takes years of experience with Oracle to manage their ERP system, and that’s why I think it’s a good idea to outsource that role.

This is an exciting time for me. The potential for a new opportunity is a bit overwhelming. Knowing that I will be out of a job is scary. At the same time, this is really good for me because a lot of the work I’m doing right now is application management and not that of a senior DBA. I have the time to really get it right.  To use a baseball analogy, I’m going to be cherry picking my opportunities.

Speaking at SQL Saturday last week gave me a new confidence. I’m good at what I do, and now I’m waiting for the right pitch to come across the plate.  In the meantime, I need to get my resume in order. I also need to get my work in a place where a more junior DBA can pick it up. And I need to start doing some serious networking to help provide a good supply of opportunities.

The real issue for me to figure out now is just what I want to do next in my career. Being in Boston gives me access to a lot of financial companies, which gives me an opportunity to grow in high availability and recovery. At the same time, those companies tend to be incredibly conservative. Being in Boston also gives me access to the high tech corridor and tons of startup companies. I’m not sure I’m ready to take on the growing pains of a rapidly growing company again. The last two taught me a lot about scalability and managing growth.

Consulting is an option, but I need to make sure I have access to good benefits. That leads me thinking that going solo isn’t a very good choice for me. Working for a small “boutique” consulitng shop would be incredibly appealing. I’d love to go out and do the tech work yet have someone else manage the business development end of it.

The good news here is that I don’t have to make any decisions right away. I have time to figure it out.

SQL Sat 71 Virtualization Success Story

During our migration to our new facility and data center a few weeks ago, we lost power to the building. Again this morning, we lost building power. Let me tell you what happened.

• Our servers didn’t even blink.
• The UPS jumped in right away.
• The backup generator kicked in right away.
• Emergency lights in the data center stayed on.
• The air conditioning stayed on.
• We were able to get into the data center with our access badges.
• Every network switch on each floor of the building ran on UPS power.

This sounds like one of those “well, duh!” moments, but I have to tell you, it was revolutionary for us.  Every one of these things was problematic in our old building.

It’s wonderful when things work the way they’re supposed to.

We have a company that does some outside work for us. We have a database server at their location. We need to allow the database server to talk to our DB server in a network DMZ.  We’d use a linked server from the remote site to talk to our DMZ server. I’m okay setting up the username/password and the firewall ports. The problem is that we need to have the traffic between the two servers encrypted.

To further complicate this, we don’t want all of the traffic on these servers encrypted, just the traffic that goes across the internet.

I’m completely befuddled on how to do this properly. There is a very good chance that I’m making this harder than it needs to be, and I’d be thrilled if someone gave me a simple solution.

The following six lines of code saved me a ton of time this morning.

ALTER DATABASE [tempdb] MODIFY FILE (name = tempdev, FILENAME = ‘D:\TempDB\tempdb.mdf’)
ALTER DATABASE [tempdb] MODIFY FILE (name = templog, FILENAME = ‘D:\TempDB\templog.ldf’)
ALTER DATABASE [msdb] MODIFY FILE ( NAME = msdbdata , FILENAME = ‘G:\system\MSDBData.mdf’ )
ALTER DATABASE [msdb] MODIFY FILE ( NAME = msdblog , FILENAME = ‘e:\system\MSDBLog.ldf’ )
ALTER DATABASE [model] MODIFY FILE ( NAME = modeldev , FILENAME = ‘G:\system\model.mdf’ )
ALTER DATABASE [model] MODIFY FILE ( NAME = modellog , FILENAME = ‘e:\system\modellog.ldf’ )

I just can’t explain how big of a deal this is to me. In my current envrionment, every server was set up with a single C: drive. Some of these file systems are over a terabyte. Now we’re breaking things up, and I can only imagine the performance boost we’re going to see.

Each DB server will live on its own VMWare host. We’re virtualizing so that we can utilize VMWare’s clustering. It also gives us the flexibility to move things around. My two database servers will have identical configurations. One will server internal applications, and the other will host customer-facing applications.

Hardware wise, I’m using IBM HS22 blades with 32 GB of RAM connected to an EMC Clariion CX4-120SAN. Each server has five datastores, each corresponding a LUN on the SAN. Each virtual drive will live in its own storage group. I have a drive for:

• OS – Aps
• TempDB
• Log (LDF) Files
• Backups
• Data (MDF/NDF) files

My preference would have been to have the virtual machines use an iSCSI initiator to talk to the SAN, but I got overruled on that one. Still, this is a better configuration than I had before. (We had no SAN at all.)

Everything is 64 bit SQL 2005.  Again, I wanted to use SQL 2008, but that’s the next project.

After installing Windows and SQL, I started putting together a checklist of items to do next. Brent Ozar has a great checklist and has been a great reference. Here is my list:

1. System Center Operations Manager Agent
2. Antivirus protection
3. Quest’s LiteSpeed
4. Service Packs, Service Packs, Service Packs.
5. Set default file locations
6. Move master DB
7. Move TempDB
8. Resize TempDB
9. Move Model
10. Move MSDB
11. Load Testing