Active Directory Replication issues since time jump to the year 2000

Active Directory replication fails with Event 2042 reporting “It has been too long since this machine last replicated” and replication status 8614: “The Active Directory cannot replicate with this server because the time since the last replication with this server has exceeded the tombstone lifetime.” and

Time of last successful replication:

2000-11-19 14:09:12

I got this error on a site with 3 DC’s and yes that puts you 12 years back, even before this company existed.
Normal solution would be demote and promote the problem DC, but, after looking around technet I bumped into this article : http://blogs.technet.com/b/askpfeplat/archive/2012/11/19/did-your-active-directory-domain-time-just-jump-to-the-year-2000.aspx and an updated article http://blogs.technet.com/b/askpfeplat/archive/2012/11/23/fixing-when-your-domain-traveled-back-in-time-the-great-system-time-rollback-to-the-year-2000.aspx, well that explains it all. the main PDC NTP was pointing at USNO.NAVY.MIL and on November 19th, 2012, time servers at USNO.NAVY.MIL incorrectly provided time samples listing CY 2000 as the current year between the hours of 21:07 UTC and 21:59 UTC (16:07-16:59 EST).

-Run W32tm /stripchart /computer:NTPServerAddress to make sure you are getting the correct time from your new source.

-Set the correct time settings to a known good source.

  1. Locate and then click the following registry subkey:HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters
  2. In the pane on the right, right-click NtpServer, and then click Modify.
  3. In Edit Value, type Peers in the Value data box, and then click OK.

-Run w32tm /config /update

NOw ofcourse you are having replication issues as AD replication would have happened when the time was set to the year 2000 and when your time is back from the past you are bound to have replication issues due to tombstone settings.
Proceed with making sure “Strict Replication Consistency” is enable in the registry value(HKLM\System\CurrentControlSet\Services\NTDS\Parameters\Strict Replication Consistency ) has been set to the default for Server 2003 and greater value, which is 0x1 before you do.“Strict Replication Consistency” will ensure that no lingering objects will be replicated out if they really exist after we force the replication to happen.

Note: This forces the replication even if the AD replication has failed with the partner for more than the tombstone lifetime. In most cases, if the server is really having the replication failure and is not caused by this unexpected time jump issue, you would see new replication errors caused by lingering objects as long as “Strict Replication Consistency” is 0x1; then no lingering objects would be really replicated out.

We will now want to follow  http://technet.microsoft.com/en-us/library/cc757610(WS.10).aspx

On the downstream domain controller where it reported the replication error code 8614 (ERROR_DS_REPL_LIFETIME_EXCEEDED), setup the “Allow Replication With Divergent and Corrupt Partner” registry value.

  1. Click Start, click Run, type regedit, and then click OK.
  2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters
  3. In the details pane, create or edit the registry entry as follows:If the registry entry exists in the details pane, modify the entry as follows:
    1. In the details pane, right-click Allow Replication With Divergent and Corrupt Partner, and then clickModify.
    2. In the Value data box, type 1, and then click OK.

    If the registry entry does not exist, create the entry as follows:

    1. Right-click Parameters, click New, and then click DWORD Value.
    2. Type the name Allow Replication With Divergent and Corrupt Partner, and then press ENTER.
    3. Double-click the entry. In the Value data box, type 1, and then click OK.

No restart is needed. Force replication in AD Sites and Services between the destination source and destination servers.

Remember to change the “Allow Replication With Divergent and Corrupt Partner” value back to 0x0 after the issue has been sorted out.
The DC I was working on had already been rebooted and one of the DC’s had kerberos related issues, so if you have noted this yr 2000 do not reboot DC’s.Event ID 4 is logged in system errors:

The kerberos client received a KRB_AP_ERR_MODIFIED error from the server <computer name>$. This indicates that the password used to encrypt the kerberos service ticket is different than that on the target server. Commonly, this is due to identically named machine accounts in the target realm (<domain name>), and the client realm. Please contact your system administrator.

To get around this, on the problem DC where you getting this error set the Kerberos Key Distribution Center Service to manual and stop the service.

-From a command prompt on the broken DC enter the following:
netdom resetpwd /s:name_of_working_DC or PDC/ud:domain\user /pd:*
where domain\user is an administrator of the domain in the domain_name\user_name format. You will be prompted to enter your password.

And now reboot this reboot clears the Kerberos ticket cache and so clears the broken credential attempts that it has stored, one can also run klist /purge and Y to all tickets to be deleted.
Support tools and commands useful to rectify this issue

– eventviewer

-repadmin /replsum

-repadmin /showrepl

-dcdiag

-replmon.exe

support tools windows 2003 32 bit: http://www.microsoft.com/en-gb/download/details.aspx?id=15326

Windows Server 2003 Resource Kit tools: http://www.microsoft.com/en-us/download/details.aspx?id=17657

Advertisements

Manage Exchange 2010 Calendar Permissions Using Powershell

You can add  Calendar Permissions Using Powershell for users using the following command:

if user2 needs to access user1’s calendar

Add-MailboxFolderPermission -identity “user1:\calendar” –user “user2” -AccessRights Reviewer

the same can be done to give permission to a security group

Add-MailboxFolderPermission -identity “user1:\calendar” –user “DomainName\securitygroup” -AccessRights Reviewer

Below is the list of access rights and explanation

None                                                   FolderVisible
Owner                                                CreateItems, ReadItems, CreateSubfolders, FolderOwner, FolderContact, FolderVisible, EditOwnedItems, EditAllItems, DeleteOwnedItems, DeleteAllItems
PublishingEditor                       CreateItems, ReadItems, CreateSubfolders, FolderVisible, EditOwnedItems, EditAllItems, DeleteOwnedItems, DeleteAllItems
Editor                                                 CreateItems, ReadItems, FolderVisible, EditOwnedItems, EditAllItems, DeleteOwnedItems, DeleteAllItems
PublishingAuthor                    CreateItems, ReadItems, CreateSubfolders, FolderVisible, EditOwnedItems, DeleteOwnedItems
Author                                              CreateItems, ReadItems, FolderVisible, EditOwnedItems, DeleteOwnedItems
NonEditingAuthor                   CreateItems, ReadItems, FolderVisible
Reviewer                                          ReadItems, FolderVisible
Contributor                                   CreateItems, FolderVisible

Redirect Exchange OWA to default site and force SSL in Exchange 2010

OWA redirect to default site forcing SSL

is useful If you have those one or two users who cant seem to remember to add the https or the OWA to the exchange OWA url.

To do this via IIS manger can be time consuming and tricky, below is the script which I came across (Mark Smith) which does it for you, just create a batch file with the script and run it once. The only thing you need to modify is https://mail.domain.com/owa

——-
c:
cd \Windows\System32\inetsrv

appcmd.exe ADD Backup “OWA REDIRECT BACKUP 01”

appcmd set config “default web site” -section:httpRedirect /childonly:true /enabled:true
appcmd set config “default web site” -section:system.webServer/httpRedirect -destination:”https://mail.domain.com/owa&#8221;

appcmd set config “default web site/Aspnet_Client” -section:httpRedirect /enabled:false
appcmd set config “default web site/Autodiscover” -section:httpRedirect /enabled:false
appcmd set config “default web site/ECP” -section:httpRedirect /enabled:false
appcmd set config “default web site/EWS” -section:httpRedirect /enabled:false
appcmd set config “default web site/Exchange” -section:httpRedirect -commit:apphost /enabled:false
appcmd set config “default web site/Exchweb” -section:httpRedirect -commit:apphost /enabled:false
appcmd set config “default web site/Microsoft-Server-ActiveSync” -section:httpRedirect -commit:apphost /enabled:false
appcmd set config “default web site/OAB” -section:httpRedirect /enabled:false
appcmd set config “default web site/OWA” -section:httpRedirect -commit:apphost /enabled:false
appcmd set config “default web site/PowerShell” -section:httpRedirect -commit:apphost /enabled:false
appcmd set config “default web site/Public” -section:httpRedirect -commit:apphost /enabled:false
appcmd set config “default web site/Rpc” -section:httpRedirect /enabled:false
appcmd set config “default web site/RpcWithCert” -section:httpRedirect /enabled:false

appcmd set config “default web site” -section:access -sslflags:”” -commit:apphost

appcmd set config “Default Web Site/Aspnet_Client” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/Autodiscover” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/EWS” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/ECP” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/Exchange” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/Exchweb” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/Microsoft-Server-ActiveSync” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/OWA” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “default web site/PowerShell” -section:access -sslflags:”” -commit:apphost
appcmd set config “default web site/OAB” -section:access -sslflags:”” -commit:apphost
appcmd set config “Default Web Site/Public” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost
appcmd set config “Default Web Site/Rpc” -section:access -sslFlags:Ssl -commit:apphost
appcmd set config “Default Web Site/RpcWithCert” -section:access -sslFlags:Ssl,Ssl128 -commit:apphost

cacls “C:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\OAB\web.config” /E /P “NT Authority\authenticated Users”:R

iisreset
——

Insufficient System resources in Exchange 2010, disable back pressure/resource monitoring

Starting from Exchange 2007,  the edge transport service uses the system resource monitoring feauture to determine vital resources such as hard drive space and memory and take action in an attempt to prevent service outages. This can be a pain as the whole thing is calculated on a formula,

100 * (hard disk size – fixed constant) / hard disk size

this feature is called Backup pressure, more info on http://technet.microsoft.com/en-us/library/bb201658.aspx and how this formula works http://www.msexchange.org/articles_tutorials/exchange-server-2010/management-administration/back-pressure-exchange-2010-part1.html

Now simply put, you have this error from other smtp servers trying to contact yours : Insufficient System resources, to get around this you will need to increase the hard drive space or memory, until then you could temporarly disable this monitoring feature so that you can start recieving important emails. WARNING: this is not a recommended permanent solution, do this until youve figured what resource needs looking at i.e hard drive space, RAM etc.

Make a backup copy of the edgetransport.exe.config file before you start.

  1. Open the EdgeTransport.exe.config file from \Exchange Server\bin directory using notepad
  2. Add the following key+value pair:
    <add key=”EnableResourceMonitoring” value=”false” />
  3. Save file
  4. Restart the Microsoft Exchange Transport Service (MSExchangeTransport):
    Restart-Service MSExchangeTransport
  5. and you’ve got mail

Other options are to

configure the edgetransport.exe config file to reflect your server configuration and resources (not recommended by MS)

OR

move the queue database to another volume which has lots of space, add the following key to change the path of the queudb.

  1. Open the following file by using Notepad: C:\Program Files\Microsoft\Exchange Server\V14\Bin\EdgeTransport.exe.config.
  2. Modify the following line in the <appSettings> section.
    <add key="QueueDatabasePath" value="<LocalPath>" />

    This example creates a queue database at the location C:\Queue\QueueDB.

    <add key="QueueDatabasePath" value="C:\Queue\QueueDB" />
  3. Save and close the EdgeTransport.exe.config file.
  4. Restart the Microsoft Exchange Transport service.
  5. Verify that the new Mail.que and Trn.chk files are created at the new location.
  6. Remove the unused Mail.que and Trn.chk files from the original location.

ref: http://technet.microsoft.com/en-us/library/f170cb0c-04a9-4fa7-b594-206e3a787e14.aspx
The following event logs will be logged:

  • Event ID 15004: Increase in the utilization level for any resource (eg from Normal to Medium)
  • Event ID 15005: Decrease in the utilization level for any resource (eg from High to Medium)
  • Event ID 15006: High utilization for disk space (ie critically low free disk space)
  • Event ID 15007: High utilization for memory (ie critically low available memory)

Have look out for the Event id which will help you in diagnose what resource needs to be looked at.

ref:http://exchangeserverpro.com/exchange-transport-server-back-pressure

Disk Cleanup tool keeps freezing or takes a long time to load in Windows XP

Every now again we cant escape the old desktop keeps freezing and running a disk cleanup, now When you try to compress old files on a Windows XP-based computer, the Disk Cleanup tool may stop responding, and you may receive the following message:

Disk Cleanup is calculating how much space you will be able to free on (C:).
This may take a few minutes to complete.
Scanning: Compress old files
This problem occurs when there is an incorrect entry in the registry that is used by the Disk Cleanup utility to locate compressed files. To work around this problem, follow these steps:

  1. Click Start, and then click Run.
  2. Type regedit in the Start Search box, and then press ENTER.
  3. Locate and then click the following registry key:
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VolumeCaches
  4. On the File menu, click Export, click Desktop, type VolumeCaches in the File name box, and then click Save.Note This step creates a backup of the VolumeCaches registry key. If you experience any problems after you complete the steps that are listed in this procedure, you can use this backup to restore the VolumeCaches key to its original state. To restore the key, double-click the VolumeCaches.reg file on your desktop, and then click Yes.
  5. Expand the following registry key:
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VolumeCaches
  6. Delete the Compress old files registry key.
  7. Exit Registry Editor.

OR
Download the fix for windows XP from here

Password Recovery Procedure for the Cisco VPN 3000 Series

The following method does not make you lose the configuration on the concentrator so far tested.

The factory default passwords for the Cisco VPN 3000 Series are:

username: admin

password: admin

Complete these steps in order to recover a password:

  1. Connect a PC to the VPN Concentrator with a straight-through RS-232 serial cable between the console port on the VPN Concentrator and the COM1 or serial port on the PC. Use software such as Putty  to connect to console.

    When the Concentrator boots, and after the power-up diagnostics check is complete, a line of three dots (…) appears on the console, a sample of which is shown here for reference. Press Ctrl-C within three seconds after you see these dots. This displays a menu that lets you reset the system passwords to the defaults.

  2. Boot-ROM Initializing... 
    Boot configured 128Mb of RAM. 
    ... 
    
    Loading image .......... 
    Verifying image checksum ........... 
    Active image loaded and verified... 
    Starting loaded image... 
    Starting power-up diagnostics... 
    
    ...
    
    !--- At this second set of three dots, press Ctrl-C.
    !--- On version 4.0.X and earlier returns with this information:
    
    
     
    Main Menu Options 
    ----------------- 
    1 - Reset Passwords 
    Q - Quit Main Menu
    
    !--- Newer version 4.1 and later returns with this information:
    
    
    Main Menu Options
    -----------------
    1 - Reset Administrator Accounts
    Q - Quit Main Menu

Deleted Mailbox missing in Disconnected Mailbox under Recepient configuration in Exchange 2010

This is because the clean up agent hasnt run yet, to run it use PS:

Clean-MailboxDatabase databasename

Invalid Fully Qualified Domain Names no longer accepted in Subject Alternative Names (SANS) in SSL certficates

After a recent legislation change, CA’s will no longer accept invalid fully qualified Domain name such as .local, servername.local as Subject Alternative Names for SSL’s expiring after 1 November 2015. If you already have got such an SSL certificate it will be revoked post this date.
I noticed this when renewing a 5 yr SSL certificate with Go-Daddy will give an error when you have Subject Alternative Names like .local, servername, servername.local.

As we all know if you use Exchange 2007/ 2010, you probably use an internal server name and an external name on your exchange service URLS like autodiscover. If you dont add this in your SAN you get the dreaded certificate mismatch error in Outlook.

“The name of the security certificate is invalid or does not match the name of the site”

Work Around:

To get around this issue, one can simply change the internal URLs to the external ones. For example you have internal name of servername.local, this will need to be changed to mail.contoso.com in the URLS, once this is done you will need to create a DNS zone for contoso.com on your internal DNS and create a host record for mail to point to the exchange servers internal IP address so that internal autodiscovery will work when you do the change.
The following will need to be done for Exchange 2007 via PS:

  1. Start the Exchange Management Shell.
  2. Modify the Autodiscover URL in the Service Connection Point. The Service Connection Point is stored in the Active Directory directory service. To modify this URL, type the following command and then press ENTER:
    Set-ClientAccessServer -Identity CAS_Server_Name -AutodiscoverServiceInternalUri
    https://mail.contoso.com/autodiscover/autodiscover.xml
  3. Modify the InternalUrl attribute of the EWS. To do this, type the following command, and then press ENTER:
    Set-WebServicesVirtualDirectory -Identity “CAS_Server_Name\EWS (Default Web Site)”
    -InternalUrl https://mail.contoso.com/ews/exchange.asmx
  4. Modify the InternalUrl attribute for Web-based Offline Address Book distribution. To do this, type the following command, and then press ENTER:
    Set-OABVirtualDirectory -Identity “CAS_Server_name\oab (Default Web Site)”
    -InternalUrl https://mail.contoso.com/oab
  5. Modify the InternalUrl attribute of the UM Web service. To do this, type the following command, and then press ENTER:
    Set-UMVirtualDirectory -Identity “CAS_Server_Nameunifiedmessaging (Default Web Site)”
    -InternalUrl https://mail.contoso.com/unifiedmessaging/service.asmx

    Note This command is required only in an Exchange 2007 environment. This command no longer exists in an Exchange 2010 environment. Instead, the WebServices URL is used for this purpose.
  6. Open IIS Manager.
  7. Expand the local computer, and then expand Application Pools.
  8. Right-click MSExchangeAutodiscoverAppPool, and then click Recycle.

 

The same for Exchange 2010

  1. Start the Exchange Management Shell.
  2. Modify the Autodiscover URL in the Service Connection Point. The Service Connection Point is stored in the Active Directory directory service. To modify this URL, type the following command and then press ENTER:
    Set-ClientAccessServer -Identity CAS_Server_Name -AutodiscoverServiceInternalUri
    https://mail.contoso.com/autodiscover/autodiscover.xml
  3. Modify the InternalUrl attribute of the EWS. To do this, type the following command, and then press ENTER:
    Set-WebServicesVirtualDirectory -Identity “CAS_Server_Name\EWS (Default Web Site)”
    -InternalUrl https://mail.contoso.com/ews/exchange.asmx
  4. Modify the InternalUrl attribute for Web-based Offline Address Book distribution. To do this, type the following command, and then press ENTER:
    Set-OABVirtualDirectory -Identity “CAS_Server_name\oab (Default Web Site)”
    -InternalUrl https://mail.contoso.com/oab
  5. Open IIS Manager.
  6. Expand the local computer, and then expand Application Pools.
  7. Right-click MSExchangeAutodiscoverAppPool, and then click Recycle.

Recover deleted Active directory user account and restore Mailbox in Server 2008 and Exchange 2010

Scenario:  Restore user account where the account was accidentally/ purposely deleted and the mailbox resides on an Exchange 2010 in a mixed 2003, 2008 environment. The deleted mailbox is now showing in disconnected mailboxes.

The first step is to recover the deleted user account in AD. Hoping that you have left the tombstones on the server to either 60 days or 180 days which is the default for windows 2003 and up.

There are several ways too restore the account, the most painstaking one using system state restore after booting into directory services restore mode which means downtime.

The easiest and simplest way is using adrestore CLI available from Microsoft, if you need GUI adrestore.net is the best.

Download it from the link and install on the DC.

Adrestore.net

The how to is available from the developers site http://blogs.microsoft.co.il/blogs/guyt/archive/2007/12/15/adrestore-net-rewrite.aspx and http://www.petri.co.il/recovering-deleted-items-active-directory.htm

Now that you have restored the user account the next problem is to connect the disconnected mailbox which is rather easy, just open EMC 2010 right-click on a disconnected mailbox, select Connect and choose a matching user in this case.

If the online maintainence hasn’t run then the deleted mailbox will not show in disconnected mailbox, you will need to run this command in PS

Clean-MailboxDatabase databasename

And now you can connect the mailbox to the user you just restored

Updating Exchange 2010 SP1 to SP2

Exchange 2010 has been there for a while now and SP2 was also released a while ago and the roll up being released in Feb 2012, now is a good time to install SP2.

So before you install SP2 on your Exchange 2010 environment have a read of the following technet articles regarding preparing your Active directory, schema, domain and legacy 2003 DC’s if you have any.

http://technet.microsoft.com/en-us/library/bb125224.aspx

The SP2 setup file does do schema changes if you don’t do them, but just in case you have multiple DC’s of 2003 and 2008, then the best will be to do schema prep on your schema master and then make sure you wait for the replication to complete of or do it force replication and then proceed with the installation.

All in all these are the following bits you need to do before installing SP2,

1. Have a good tested backup as always .

On the schema master perform the following using the setup file you download of Microsoft

2. Open CMD and then setup /PrepareLegacyExchangePermissions or setup /pl, this is only if you have 2003 DC’s in your domain

3. setup /PrepareSchema or setup /ps

4setup /PrepareAD

5. setup /PrepareDomain , this is only needed if you run multiple domains, otherwise prepareAD takes care of one local domain.

Once done, follow the steps to complete the pre-requisites: http://technet.microsoft.com/en-us/library/bb691354.aspx

For a standard install with 2008 R2 you will need to install these components from powershell. You can check this in the exchange-typical.xml which located in the scripts folder in the sp2 download.

In PS:

Import-Module ServerManager and then
Add-WindowsFeature NET-Framework,RSAT-ADDS,Web-Server,Web-Basic-Auth,Web-Windows-Auth,Web-Metabase,Web-Net-Ext,Web-Lgcy-Mgmt-Console,WAS-Process-Model,RSAT-Web-Server,Web-ISAPI-Ext,Web-Digest-Auth,Web-Dyn-Compression,NET-HTTP-Activation,Web-Asp-Net,Web-Client-Auth,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Http-Redirect,Web-Http-Tracing,Web-ISAPI-Filter,Web-Request-Monitor,Web-Static-Content,Web-WMI,RPC-Over-HTTP-Proxy -Restart

There is no harm installing all the components for peace of mind. Once your done rebooting, start the SP2 GUI installation and sit back and watch it complete flawlessly.