Saturday, March 5, 2011

Backup Active Directory from hardware, restore to Virtual sandbox

Let's say you experienced a disaster scenario. You had domain controllers hosted on Dell server hardware that are no longer accessible. You have a backup but only of one domain controller's system state from Windows Server 2003 R2's NTBackup. The Dell ran an OEM-licensed version of Windows Server 2003 R2 Standard Edition. You need to restore Active Directory to a virtual machine to a volume licensed version of Windows Server 2003 Standard (not R2). This is the closest to an imaginable scenario I could come up with for what I did.

I want to test out server software on a sandbox domain resembling our production environment. I want to learn VMWare (we're looking to virtualize, at least partially, for the sake of server consolidation, disaster planning, high availability, and colocation). Beyond a lab learning environment, I'd never had to recover Active Directory. All that being said, there are better ways to do what I'd done, and I knowingly went against Microsoft recommendations (more accurately, I did things Microsoft recommends against doing) in at least a couple of places. For example, I could have much more easily used VMWare Converter to virtualize servers, but I didn't want to install VMWare Converter on a production domain controller (DC). Microsoft recommends against seizing roles in favor of transferring them and recommends recovering all your DCs (then transferring roles and demoting servers). Microsoft recommends against locating the global catalog on the infrastructure master server (I am only in a single domain forest though).

Here's a play by play:
I backed up the system state of our first domain controller (DC) holding most operations master roles using NTBackup, which is running OEM installed Windows Server 2003 R2 Standard Edition on reliable, powerful Dell server hardware.
I created an up to date (Windows Updates) virtual Windows Server 2003 Standard server with VMWare Tools and set it up to mirror that production DC (set it up as the typical first server, with Active Directory Domain Services, DNS, and DHCP adding WINS). This server was not connected to the production network (only an internal virtual network).
I copied over the backup, rebooted into Directory Services Restore Mode (DS RM), and restored the system state.
Side note: Doing nothing else, if I rebooted, Windows Server would not start reliably (I think it started once... maybe). The hardware was too different. The HAL and kernel were for a multiprocessor system, the virtual server was a single processor. I could get it to boot by replacing hal.dll, ntoskrnl.exe, ntkrnlpa.exe, and kernel32.dll, but then I couldn't log in due to a licensing/activation issue. On Windows Server 2008 or 2008 R2, I may have been able to work around this but I couldn't figure it out on Windows Server 2003 (seen mention registry locations, wpa.dbl, licensing libraries - dlls, and executables - I'm sure a Microsoft guy would cringe at my attempt) .
I rebooted to the Windows Server 2003 installation media, pressed enter to install, accepted the license, and r to repair.
Side Note: Now some of the virtual machine was showing Windows Server 2003 R2. I could log in but could not join the domain ("The domain name [DOMAIN] might be a NetBIOS domain name. If this is the case, verify that the domain name is properly registered with WINS. ..." or "The following error occurred when DNS was queried for the service location (SRV) resource record used to locate an Active Directory Domain Controller for domain domain.fqdn: The error was: "This operation returned because the timeout period expired."...").
I logged in and did some cleanup. Some installed services didn't start as the programs weren't there. Some programs in Add/Remove Programs weren't there. For this, I used the Windows Installer Cleanup Utility, the sc delete command, CCleaner, and some altering of the registry. For Microsoft.NET, I had to do some ugly ripping from the registry (Microsoft cringe #2). I updated the server. I demoted the other domain controllers forcibly using Active Directory Sites and Services (delete the replication partners in NTDS Settings, delete NTDS Settings and delete the Server) (Microsoft cringe #3).
I removed the other DCs from Active Directory Sites and Services and DNS. Where possible, I tried to use the GUI but had to delete some DNS entries manually. When satisfied, I backed up the system state, saved it out. I shut down this first server. I created another a second "typical first server" Active Directory, DNS, and DHCP adding WINS up to date with Windows Updates and with VMWare tools. I copied the backup down, rebooted into DS RM, restored the system state and rebooted.
Along side of this more civilized and cleaner DC, I brought up a Windows Server 2003 server without roles but with updates and VMWare Tools. I installed the Active Directory role on this server as a second server on the domain to the cleaner first server on the domain. I added the DNS role and WINS. I made WINS a replication partner on both servers, replicated the data, deleted the replication servers and removed WINS from the other server. I added the DHCP role (initially unactivated), copied over the settings, deactivated the original DHCP server, activated the new DHCP server, and removed the DHCP server role from the original DHCP server. I removed the original DNS server from responding to requests, made the new DNS server the primary on zones, removed it from the zones, and removed the role from the first server. I transferred over the roles (PDC emulator, RID master, Infrastructure master, schema master, and domain naming master) using the interface. Finally, I removed the Active Directory role (demoting the server) and got rid of every server except the new AD, DNS, DHCP, and WINS server.

To summarize, the steps were as follows:
1) Backup the system state using NTBackup from the DC holding most master roles.
2) Create the virtual machine server (with updates and VM tools) in isolated environment. Use dcpromo or Add or remove server roles to add the domain controller role as a typical first server, configuring to mirror the environment to be restored. Restart at the end of the installation wizard.
3) Copy over the system state backup. Restart and start in Directory Services Recovery Mode. Restore the system state using NTBackup.
4) Insert the Windows Server installation media. Restart and boot to the installation media. Repair the installation (past the recovery console, a repair installation).
5) Clean up the wreckage. Forcibly demote the servers that will not be restored. Use Windows Installer Cleanup Utility, CCleaner's Add/Remove Programs entry deleter, CCleaner's registry cleaner, sc delete command, and regedit (to remove some services, drivers, and programs manually)
6) Backup the virtual domain controller's system state using NTBackup. Copy the backup file out and shut down this server.
7) Create a second virtual machine server (with updates and VM tools) in isolated environment. Use dcpromo or Add or remove server roles to add the domain controller role as a typical first server, configuring to mirror the environment to be restored. Restart at the end of the installation wizard.
8) Copy over the system state backup. Restart and start in Directory Services Recovery Mode. Restore the system state using NTBackup.
9) Create a third virtual machine server (with updates and VM tools) in isolated environment. Use dcpromo or Add or remove server roles to add the domain controller role. Add this domain controller to the second server's existing domain. Restart when complete.
10) Add the DNS and WINS rules. Add but do not configure the DHCP role.
11) In WINS on the second server, add the third server as a replication partner. On the third server, add the second server as a replication partner. Initiate replication from either or both servers.
12) Duplicate the DHCP settings from the second server (changing, where necessary, to reflect the third server's planned role). Unactivate DHCP from the second server and activate DHCP on the third server. Remove the DHCP role from the second server. If WINS has finished replication, remove the WINS role from the second server.
13) Configure the third server's DNS with the intention of it being the DNS server (make the third server the primary server in the zones). Remove the second server from the third server's DNS. Remove the DNS role from the second server.
14) Transfer roles. RID, PDC, and Infrastructure can be transferred from Active Directory Users and Computers from the second server by connecting to the third server domain controller. Transfer Domain Naming Master can be done in Active Directory Sites and Trusts from the second server connecting to the third. Schema Master can be transferred from Active Directory Schema, but you may need to register it (regsvr32 schmmgmt.dll) and open it from mmc (Microsoft Management Console). This all can be done from the command line using ntdsutil.
15) Remove the Domain Controller role from the second server (using Add or Remove Roles or dcpromo), demoting it.
16) Now you can get rid of the first and second servers.

2 comments:

  1. Your blog is great. Your thoughts are also very good and i am very inspired from your post. That is why I visit this blog again and again and will come back in future too.

    website hosting company

    ReplyDelete
  2. Your blog is great. Your thoughts are also very good and i am very inspired from your post. That is why I visit this blog again and again and will come back in future too.
    domain name web hosting

    ReplyDelete