Backup Active Directory from hardware, restore to Virtual sandbox

Let's say you experienced a disaster scenario. You had domain controllers hosted on Dell server hardware that are no longer accessible. You have a backup but only of one domain controller's system state from Windows Server 2003 R2's NTBackup. The Dell ran an OEM-licensed version of Windows Server 2003 R2 Standard Edition. You need to restore Active Directory to a virtual machine to a volume licensed version of Windows Server 2003 Standard (not R2). This is the closest to an imaginable scenario I could come up with for what I did.

I want to test out server software on a sandbox domain resembling our production environment. I want to learn VMWare (we're looking to virtualize, at least partially, for the sake of server consolidation, disaster planning, high availability, and colocation). Beyond a lab learning environment, I'd never had to recover Active Directory. All that being said, there are better ways to do what I'd done, and I knowingly went against Microsoft recommendations (more accurately, I did things Microsoft recommends against doing) in at least a couple of places. For example, I could have much more easily used VMWare Converter to virtualize servers, but I didn't want to install VMWare Converter on a production domain controller (DC). Microsoft recommends against seizing roles in favor of transferring them and recommends recovering all your DCs (then transferring roles and demoting servers). Microsoft recommends against locating the global catalog on the infrastructure master server (I am only in a single domain forest though).

Here's a play by play:
I backed up the system state of our first domain controller (DC) holding most operations master roles using NTBackup, which is running OEM installed Windows Server 2003 R2 Standard Edition on reliable, powerful Dell server hardware.
I created an up to date (Windows Updates) virtual Windows Server 2003 Standard server with VMWare Tools and set it up to mirror that production DC (set it up as the typical first server, with Active Directory Domain Services, DNS, and DHCP adding WINS). This server was not connected to the production network (only an internal virtual network).
I copied over the backup, rebooted into Directory Services Restore Mode (DS RM), and restored the system state.
Side note: Doing nothing else, if I rebooted, Windows Server would not start reliably (I think it started once... maybe). The hardware was too different. The HAL and kernel were for a multiprocessor system, the virtual server was a single processor. I could get it to boot by replacing hal.dll, ntoskrnl.exe, ntkrnlpa.exe, and kernel32.dll, but then I couldn't log in due to a licensing/activation issue. On Windows Server 2008 or 2008 R2, I may have been able to work around this but I couldn't figure it out on Windows Server 2003 (seen mention registry locations, wpa.dbl, licensing libraries - dlls, and executables - I'm sure a Microsoft guy would cringe at my attempt) .
I rebooted to the Windows Server 2003 installation media, pressed enter to install, accepted the license, and r to repair.
Side Note: Now some of the virtual machine was showing Windows Server 2003 R2. I could log in but could not join the domain ("The domain name [DOMAIN] might be a NetBIOS domain name. If this is the case, verify that the domain name is properly registered with WINS. ..." or "The following error occurred when DNS was queried for the service location (SRV) resource record used to locate an Active Directory Domain Controller for domain domain.fqdn: The error was: "This operation returned because the timeout period expired."...").
I logged in and did some cleanup. Some installed services didn't start as the programs weren't there. Some programs in Add/Remove Programs weren't there. For this, I used the Windows Installer Cleanup Utility, the sc delete command, CCleaner, and some altering of the registry. For Microsoft.NET, I had to do some ugly ripping from the registry (Microsoft cringe #2). I updated the server. I demoted the other domain controllers forcibly using Active Directory Sites and Services (delete the replication partners in NTDS Settings, delete NTDS Settings and delete the Server) (Microsoft cringe #3).
I removed the other DCs from Active Directory Sites and Services and DNS. Where possible, I tried to use the GUI but had to delete some DNS entries manually. When satisfied, I backed up the system state, saved it out. I shut down this first server. I created another a second "typical first server" Active Directory, DNS, and DHCP adding WINS up to date with Windows Updates and with VMWare tools. I copied the backup down, rebooted into DS RM, restored the system state and rebooted.
Along side of this more civilized and cleaner DC, I brought up a Windows Server 2003 server without roles but with updates and VMWare Tools. I installed the Active Directory role on this server as a second server on the domain to the cleaner first server on the domain. I added the DNS role and WINS. I made WINS a replication partner on both servers, replicated the data, deleted the replication servers and removed WINS from the other server. I added the DHCP role (initially unactivated), copied over the settings, deactivated the original DHCP server, activated the new DHCP server, and removed the DHCP server role from the original DHCP server. I removed the original DNS server from responding to requests, made the new DNS server the primary on zones, removed it from the zones, and removed the role from the first server. I transferred over the roles (PDC emulator, RID master, Infrastructure master, schema master, and domain naming master) using the interface. Finally, I removed the Active Directory role (demoting the server) and got rid of every server except the new AD, DNS, DHCP, and WINS server.

To summarize, the steps were as follows:
1) Backup the system state using NTBackup from the DC holding most master roles.
2) Create the virtual machine server (with updates and VM tools) in isolated environment. Use dcpromo or Add or remove server roles to add the domain controller role as a typical first server, configuring to mirror the environment to be restored. Restart at the end of the installation wizard.
3) Copy over the system state backup. Restart and start in Directory Services Recovery Mode. Restore the system state using NTBackup.
4) Insert the Windows Server installation media. Restart and boot to the installation media. Repair the installation (past the recovery console, a repair installation).
5) Clean up the wreckage. Forcibly demote the servers that will not be restored. Use Windows Installer Cleanup Utility, CCleaner's Add/Remove Programs entry deleter, CCleaner's registry cleaner, sc delete command, and regedit (to remove some services, drivers, and programs manually)
6) Backup the virtual domain controller's system state using NTBackup. Copy the backup file out and shut down this server.
7) Create a second virtual machine server (with updates and VM tools) in isolated environment. Use dcpromo or Add or remove server roles to add the domain controller role as a typical first server, configuring to mirror the environment to be restored. Restart at the end of the installation wizard.
8) Copy over the system state backup. Restart and start in Directory Services Recovery Mode. Restore the system state using NTBackup.
9) Create a third virtual machine server (with updates and VM tools) in isolated environment. Use dcpromo or Add or remove server roles to add the domain controller role. Add this domain controller to the second server's existing domain. Restart when complete.
10) Add the DNS and WINS rules. Add but do not configure the DHCP role.
11) In WINS on the second server, add the third server as a replication partner. On the third server, add the second server as a replication partner. Initiate replication from either or both servers.
12) Duplicate the DHCP settings from the second server (changing, where necessary, to reflect the third server's planned role). Unactivate DHCP from the second server and activate DHCP on the third server. Remove the DHCP role from the second server. If WINS has finished replication, remove the WINS role from the second server.
13) Configure the third server's DNS with the intention of it being the DNS server (make the third server the primary server in the zones). Remove the second server from the third server's DNS. Remove the DNS role from the second server.
14) Transfer roles. RID, PDC, and Infrastructure can be transferred from Active Directory Users and Computers from the second server by connecting to the third server domain controller. Transfer Domain Naming Master can be done in Active Directory Sites and Trusts from the second server connecting to the third. Schema Master can be transferred from Active Directory Schema, but you may need to register it (regsvr32 schmmgmt.dll) and open it from mmc (Microsoft Management Console). This all can be done from the command line using ntdsutil.
15) Remove the Domain Controller role from the second server (using Add or Remove Roles or dcpromo), demoting it.
16) Now you can get rid of the first and second servers.

VirtualBox Mac OS X 10.6.6

I found instructions elsewhere that Mac OS X can be run on Oracle VirtualBox. The instructions say to buy Mac OS X Snow Leopard from a retailer or official channels, which can be had for $39.99. However, if my understanding is correct, this is the "upgrade" price (you can't buy a Mac without OS X, so you either have the current version an older version). Also, I believe the price of the operating system is hardware subsidized - that if Apple sold operating systems independent of hardware, OS X might have to cost more. Also, I think you're only to virtualize Mac OS X Server on VirtualBox and only on Mac hardware, so no Mac OS X client on PC hardware. You need Intel VT-x. These are the instructions:

1) Go to virtualbox.org and download and install the latest version of Oracle VirtualBox.
2) Open VirtualBox, click New, give it a name and select OS Type, select Mac OS X and Mac OS X Server.
3) You can select the defaults (or more) for memory and hard drive. Note, fixed-sized storage is faster but takes up the total drive size on your hard drive regardless of how much space is being used. Finish the Create New Virtual Disk Wizard and the Create New Virtual Machine Wizard.
4) Download the EmpireEFI boot disk (search Google) (optional: also download NawCom boot disk).
5) Change the settings of the OS X Virtual Machine. Select the Virtual Machine and click the Settings button. Under System, uncheck Enable EFI. You can check Enable 3D Acceleration under Display. Click OK.
6) Click Start. For the installation media, select the EmpireEFI ISO. When you see the EmpireEFI boot screen, switch the EmpireEFI media for the Mac OS X media and press F5 (twice if necessary). Select the Mac OS X Install DVD and press Enter.
7) Select the language and click Next. Click Continue. Click Agree.
8) Click Utilities from the menu -> click Disk Utility -> select your disk from the left and click Partition from the middle bar. From Volume Scheme, choose a volume scheme (1 partition), give a name (Macintosh HD), click Apply and click Partition. Click Close.
9) Select the Hard drive and click Install.
10) The installation "fails". Restart and switch the boot media to the EmpireEFI iso again.
11) When the computer reboots, choose to boot to the Hard drive. It boots into Mac OS X, click OK and configure the keyboard. Select your region and Keyboard. Configure your account and time zone.
12) Double-click or open the Empire EFI disk in OS X, double-click Post-Installation, and run the myHack Installer. Run the installation with the defaults (Continue, continue, continue, agree, continue, install...).

To install the OS X update(s), download the full ComboUpdate and run the full download (like MacOSXUpdCombo10.6.6.dmg (if installed, remove SleepEnabler.kext). After the ComboUpdate completes, do NOT restart. Reinstall the Empire EFI. Remove the iso and reboot. Other updates work without individual downloads and without Empire EFI.

Optional portion - reboot to the NawCom iso, select the Macintosh HD and boot into OS X.

Note: You may not be able to copy to ISO (or to DVD DL) the Mac boot DVD DL. Ubuntu will do the trick, and you can use Live CD/DVD for this purpose.

McAfee VirusScan 8.8 from 8.7 on ePolicy Orchestrator 4.5

I use McAfee's ePolicy Orchestrator (ePO) 4.5 at work. When patches to McAfee VirusScan or the McAfee Agent (ePO Management Agent) are released, patches are checked in to the Software Repository and are deployed. New versions of software are a bit different, so when McAfee released VirusScan 8.8, I did not do everything needed to get it deployed initially. First, I only checked in the software; then I did everything (from the installation instructions) but update the client tasks. I'm recording what I had to do to make it work (update and deploy McAfee VirusScan 8.8 using ePO) for future reference:

1) Download the new version of VirusScan from McAfee using the Grant Number
2) Keep the zip file (needed to check in the software) and unzip the contents to a directory (needed to check in the extensions and to run EPOPolicyMigration).
3) Launch McAfee ePolicy Orchestrator 4.5.0 Management Console and login
4) Go to Menu - Software - Master Repository
5) Click Actions - Check In Package, click Browse, browse to the VSE880LML.zip file and click Next and Save.
6) Click Menu - Software - Extensions and click Install Extension
7) From the extracted VSE880LML folder, install the zip files epo_help_vse_880, virusscan8800, and virusscanreports120.
8) Run the EPOPolicyMigration tool the inside the extracted VSE880LML folder. Migration should complete successfully.
9) Update your client tasks to install VirusScan 8.8 (from within Menu - Systems - System Tree, pick your group, click the Client Tasks "tab" and edit any tasks that install VirusScan to change from 8.7 to 8.8).

It started working after that. Whew.

Network Shortcuts on Desktop slows Windows XP Performance

I spent two days trying to diagnose a laptop computer running Windows XP experiencing slow performance, all to conclude the issue was with old links on the desktop pointing to non-existent (no longer existent) network locations - decommissioned servers.

I ran the much vaunted Dell Diagnostic utilities four times, memtest86+ 44 passes, and the IBM Hitachi Drive Fitness Test. I started to update drivers, which unfortunately lead to a blue screen then a non booting system. I tried to get the system booting again by using chkdsk, fixboot, and fixmbr to no avail, leading me to repair Windows XP, update Windows, and update the drivers. It wasn't until I was asked to transfer over desktop files and folders to the loaner issued that I noticed that it took longer to log in as the user than to log in as me, and that there were old network shortcuts on the desktop.

In Windows XP, I have mistyped into the address bar computer names and IP addresses in the past to have Windows Explorer freeze until the requests time out after a few minutes. In Windows Vista or Windows 7, the same mistake does not cause that to happen. In essence, when loading or refreshing the desktop, Windows XP must be validating the shortcuts and freezing until the request times out. By extension, I would think even the valid shortcuts are checked, with some overhead. Shortcuts in folders on the desktop are probably not checked this way.

To summarize, network shortcuts on a Windows XP desktop can cause performance problems, and I would recommend storing shortcuts you want on the Desktop inside a folder on the desktop, as opposed the directly on the desktop. With Windows Vista and Windows 7, this is less of a concern.

Native-Boot Differencing VHDs in Windows 7 - I did it, now what?

Windows 7 has a feature called Native-Boot Virtual Hard Disks. You can boot to a file on your hard drive (a virtual hard disk file) that contains all the files and folders of your operating system - either Windows 7 Ultimate or Enterprise or some versions of Windows Server 2008 R2. You can create a file based on the container virtual hard disk (VHD) file that tracks the differences or changes you make when you're running from the that file called a differencing VHD. A differencing VHD file is useful if you need multiple VHD files that share a common base (or parent). It's quicker and uses less space as the number of VHDs that you need increases. Also, you can experiment in a differencing VHD without worrying about messing up the original VHD. You cannot (as yet) update many differencing VHD files simply by updating the parent VHD.

This was my experiment:
1) Boot from the Windows 7 Ultimate install DVD.
2) At the Install Now screen, press SHIFT+F10 to get a command prompt
3) Partition and format the drive as needed. Since I was using a Dell Latitude, I left the diagnostic, recovery, and Latitude Reader On (an instant boot feature) alone. I deleted the original OS partition, created a new partition, and formatted it. For me, the steps were as follows:
a) Type diskpart and press enter
b) Type list disk and press enter; identify the disk, type select disk 0 and press enter (replace 0 with the correct disk number)
c) Type list part and press enter; identify the partition, type select partition 3 and press enter (replace 3 with the correct partition number)
d) Type delete partition and press enter
e) Type create partition primary and press enter (you can specify a size using size=, otherwise the new partition claims the remaining space)
f) Type format fs=ntfs quick and press enter (without "quick", a full format is performed).
g) Assign a drive letter. To list used drive letters, type list volume and press enter; type assign letter=c (substitute c with an unused drive letter)
4) Create the vdisk and attach it:
a) (While still in diskpart) Type create vdisk file="c:\Windows7.vhd" type=fixed size=51200 and press enter (type can be fixed or expandable and size needs to be smaller than the drive you choose)
b) Type select vdisk file="c:\Windows7.vhd" and press enter.
c) Type attach vdisk and press enter.
d) Type Exit and press enter
5) Switch to the install screen and press Install Now. Install Windows 7 to the VHD you created (you can ignore the “Windows cannot install to this disk” warning note). I picked a custom install and installed to the unused space that represented the VHD.
6) Customize your parent or base VHD as desired. I installed drivers, Windows Updates, Microsoft Office, Adobe Acrobat, virus protection, and some of our company's vendor's software.
7) Reboot and boot from the installation DVD (I think this can be done in Windows 7 from an run as administrator command prompt, this is just how I chose to do it).
8) Press SHIFT+F10 to get the command prompt again.
9) Type diskpart and press enter.
10) Type create vdisk file="c:\Win7Diff.vhd" parent="c:\Windows7.vhd" type=expandable maximum=51200 and press enter (filename, type, and maximum size are up to you)
11) Type Exit and press enter
12) Type bcdedit /v and press enter.
13) Highlight and copy (right-click) the identifier for the Windows Boot Loader for the boot VHD.
14) Type bcdedit /copy [press space then right-click to paste the identifier] /d "Windows 7 Differencing VHD" and press enter (contained in the quotes is the name you want displayed from the boot menu, I chose "Windows 7 Differencing VHD").
15) Highlight and right-click to copy the new guid.
16) Type bcdedit /set [press space then right-click to paste the identifier] device vhd=[c:]\Win7Diff.vhd and press enter
17) Type bcdedit /set [press space then right-click to paste the identifier] device vhd=[c:]\Win7Diff.vhd and press enter
18) (If you want this to be your new default) Type bcdedit /default [press space then right-click to paste the identifier] and press enter.
19) Reboot

Like so many other things I experiment with before I have a reason to undertake the task, I was left with the feeling - cool, now what. This was my main source for the procedure: http://blogs.msdn.com/b/knom/archive/2009/04/07/windows-7-vhd-boot-setup-guideline.aspx

Bad Dell Service Experience Or "Reseat Touchpad Cable? Take Apart Whole Laptop!"

Background: Take this post with a grain of salt. On the whole, I have liked Dell Computer Corporation. I like the way that they make documentation and and drivers easy to get to and install. I like their Dell Diagnostics utilities and partition, which can help in troubleshooting (the Ultimate Boot CD is great for computers without diagnostics utilities). I own a Dell OptiPlex 740 at home and work with Dell Latitude D620, D630, E6400, and E6510s at work, as well as Dell PowerEdge servers, PowerConnect switches, and PowerVault storage. However, we have had more problems with the Dell Latitude D620s (with nVidia graphics more than Intel integrated graphics) than with any other model (D630s are a close second). I have the impression (perhaps faulty) that the Latitude D600 and D610s were very solid and newer Latitudes have been less so. At home, I use a Samsung NC20 netbook because the geek in me wanted to try the Via Nano processor (versus the Atom).

Story: Yesterday a new hire's laptop's touchpad buttons were acting up sporadically. One or both buttons did not work or buttons functions were swapped. Originally when I set the laptop up, I saw this problem, updated the Touchpad driver and considered it fixed. I gave the new hire an identical replacement laptop with the original drive and went about troubleshooting. Given idle time (to cool?), the laptop was behaving at my desk. After some time working with it, the problem started to crop up. Use and/or stress testing seemed to help uncover the problem.

Finally able to reproduce the problem, I set about contacting Dell Support using their Chat Support. It seemed like this problem might take a while, and I didn't want to hold the phone to my ear for a long time. Also, I wasn't sure I could articulate the problem well over the phone (perhaps paradoxically, how much harder is it to explain over chat?). Around 1:30 PM CST, I try to connect using chat and see this message: "Welcome to Latitude Chat Support Queue is closed/ or all agents are busy. Note: service lines days of and hours of operation: Monday 5am thru Friday 9pm CST; Closed on Saturday and Sunday."

At some point, I was given the option to initiate e-mail support and did. I tried Chat support again at 1:50 PM and got through to "CLKsmb" support. After 20 minutes of explaining, the agent was about to set up a replacement touchpad to be sent when the agent decided I needed to be "transferred" (his words -- this is chat, not a phone call) to the "correct department that supports your account", ie "ProSupport". The chat ended after 30 minutes. To troubleshoot, the agent had me boot into Windows Safe Mode. The problem did not occur. I rebooted in Normal mode and the problem did occur. After using the laptop some more again, I rebooted to Safe Mode and the problem occurred. I booted to Windows PE from a USB Flash Drive and the problem occurred. I let it sit then rebooted to Normal mode and the problem wasn't experienced.

Over the course of the chat with the ProSupport agent, it was implied that the problem might involve improper cable seating. Not wanting to leap before I looked, I asked and had confirmed that I was to reseat the keyboard and touchpad cables, so I endeavored to take the laptop apart. This was my first Latitude E6510 (or E-series for that matter) to take apart, so I didn't have experience to lean back on. I've taken apart many Latitude D-series laptops, for fun and for profit as they say, and I'm Dell certified in their Fast-Track Support program. It was through taking apart the laptop that I was to learn a couple of strange facts. First, to get at the touchpad ribbon (some call it a flat flexible cable), I practically had to take the entire laptop apart (!!). The laptop is layer-deigned and (on the plus side) many things are accessible with the removal of a single screw (memory, WWAN, fan and heatsinks, modem, wireless, WiMax, etc) . Second, I found that many parts (speakers, fingerprint reader) were connected to the motherboard using ribbon connectors (some call them flat flexible cables). In my mind, this is a step backwards, as these cables are delicate and the cables can slip and make unreliable connections if not connected properly. To me, it seems silly to have to take apart most of the laptop (possibly including removing the heatsinks and fan) to perform a more routine replacement. The keyboard was easy to get at and reseat though. In the time between, the agent never checked on me, and the chat disconnected me.

After waiting, I contacted chat support again and got a very good agent who finalized the issue with me in short order, sending over a replacement keyboard and palm rest (the pointing stick buttons were acting up as well and are part of the keyboard). I never received word about my e-mail request and finally heard back from the second agent, 3 hours after the request was finalized.

Maybe I am being overly critical. Maybe thin flexible cables aren't bad. And maybe the passing around experience is coloring this post. This one experience will not change that the company will continue to rely on Dell products for Information Technology. I do wonder about some design choices made though.

Acrobat 9.0 to Acrobat 9,4,1 In One Command

I talked about "slipstreaming" the Acrobat 9 updates to an administrative installation point. The downsides are now Adobe Acrobat wants to use the administration installation point as the source disc on repairs (possibly leading to needing to maintain multiple versions) and the admin install points are large (try ~940 MB versus 334 MB for install media).

Instead, I now use a single command: msiexec /i "\Acrobat 9 Standard\Acrostan.msi" /update "\AcroProStdUpd910_T1T2_incr.msp;\AcrobatUpd911_all_incr.msp;\AcrobatUpd912_all_incr.msp;\AcrobatUpd913_all_incr.msp;\AcrobatUpd920_all_incr.msp;\AcrobatUpd930_all_incr.msp;\AcrobatUpd932_all_incr.msp;\AcrobatUpd933_all_incr.msp;\AcrobatUpd940_all_incr.msp;\AcrobatUpd941_all_incr.msp"

A few notes: this has in mind a professional environment in which software (and updates) is stored on an accessible file server (you can do this locally as well though); I had to use quotes because my paths had spaces (like "..\Adobe\Acrobat Updates\"; to find out which updates to use, I had to read Adobe documentation, starting with the newest update (9.4.1 requires 9.4.0, requires 9.3.3, etc); and I placed that command into a batch file saved to the server