Sunday, September 3, 2017

Recover from failed attempt to rebuild Windows Server performance counters without backup

Scenario:

For any reason you try to recover/rebuild Windows Server performance counters using KB2554336 or other "smart" blog. After procedure you have some counters are missing and some 3rd party services cannot start at all. In my case it was one of Skype for Business Edge servers from Edge pool with Application Log errors like:
Log Name:      Application
Source:        usbperf
Date:          9/3/2017 11:23:41 AM
Event ID:      2001
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server1.contoso.com
Description:
Unable to read the "First Counter" value under the usbperf\Performance Key. Status codes returned in data. 
Log Name:      Application
Source:        Microsoft-Windows-IIS-W3SVC-PerfCounters
Date:          9/3/2017 11:23:41 AM
Event ID:      2002
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server1.contoso.com
Description:
Setting up Web Service counters failed, please make sure your Web Service counters are registered correctly. 
Log Name:      Application
Source:        Microsoft-Windows-Perflib
Date:          9/3/2017 11:20:53 AM
Event ID:      1008
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server1.contoso.com
Description:
The Open Procedure for service "aspnet_state" in DLL "C:\Windows\System32\aspnet_counters.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.
Log Name:      Application
Source:        Application Error
Date:          9/3/2017 12:14:05 PM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server1.contoso.com
Description:
Faulting application name: ReplicaReplicatorAgent.exe, version: 6.0.9319.0, time stamp: 0x55276ea8
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x00007ff7ae9db0f1
Faulting process id: 0xf28
Faulting application start time: 0x01d32494fba1def3
Faulting application path: D:\Skype for Business Server 2015\Server\Replica Replicator Agent\ReplicaReplicatorAgent.exe
Faulting module path: unknown
Report Id: 3b46686b-9088-11e7-80e6-00155d083802
Solution:

Option 1. Recover server from backup/snapshot/etc.

Option 2. In my case I didn't have backup. We'll focus on this recovery scenario.

Step 1. Make performance counters backup from similar neighbor "healthy" server.

"Healthy" server should be configured with exactly the same Windows Server and Application components (ex. server from a server pool, a node from failover or nlb cluster, etc.)

from cmd (Run As Administrator)
lodctr /S:C:\temp\counters_healthy-server.ini
Step 2. Copy counters_healthy-server.ini file from "healthy" to "unhealthy" server with missing counters.

Step 3. Make performance counters backup on "unhealthy" server (just for case)

from cmd (Run As Administrator):
lodctr /S:C:\temp\counters_unhealthy-server.ini
Step 4. Rebuild performance counters from counters_healthy-server.ini backup file on "unhealthy" server:

from cmd (Run As Administrator):
lodctr /R:C:\temp\counters_healthy-server.ini
Step 5. Resync the counters with Windows Management Instrumentation (WMI):

from cmd (Run As Administrator):
WINMGMT.EXE /RESYNCPERF
Step 6. Restart "unhealthy" server. Feel happy.

Moral: always make backup prior to any change even if it looks like very simple change.

References:

Tuesday, August 29, 2017

Validation errors for users/contacts/groups in the Office 365 portal

Scenario:

Let's say you get something like:

Get-MsolUser -UserPrincipalName user@domain.com | fl *status*,errors
OverallProvisioningStatus : Success
ValidationStatus          : Error
Errors                    : {Microsoft.Online.Administration.ValidationError}

How to get more details about the "ValidationStatus" Error?

Solution:

for users:

$errors = (Get-MsolUser -UserPrincipalName "user@domain.com").Errors

for contacts:

$errors = (Get-MsolContact -ObjectID <Object_ID>).Errors

for groups:

$errors = (Get-MsolGroup -ObjectID <Object_ID>).Errors

and then expand $errors variable:

$errors | % {"`nService: "+ $_.ErrorDetail.Name.split("/")[0]; "Error Message: "+ $_.ErrorDetail.ObjectErrors.ErrorRecord.ErrorDescription}  

Example:

$errors = (Get-MsolUser -UserPrincipalName user@domain.com).Errors

$errors | % {“`nService: ” + $_.ErrorDetail.Name.split("/")[0]; “Error Message: ” + $_.ErrorDetail.ObjectErrors.ErrorRecord.ErrorDescription}

Service: exchange
Error Message: Exchange can't disable the mailbox "NAMPR07A002.prod.outlook.com/Microsoft Exchange Hosted Organizations/xxxxx.onmicrosoft.com/Soft Deleted Objects/UserNameHere" because it is on In-Place Hold.

Reference:


  • You see validation errors for users in the Office 365 portal or in the Azure Active Directory Module for Windows PowerShell
  • Saturday, August 12, 2017

    Exchange Cutover Migration Endpoints. PowerShell configuration samples

    # Exchange Server migration account credentials
    $cred = Get-Credential

    # If migration account is not hidden from GAL and you are going to use Autodiscover
    $TSMA = Test-MigrationServerAvailability -ExchangeOutlookAnywhere -Autodiscover -EmailAddress administrator@contoso.com -Credentials $cred

    # If migration account is hidden from GAL and you are going to use Autodiscover
    $TSMA = Test-MigrationServerAvailability -ExchangeOutlookAnywhere -Autodiscover -EmailAddress administrator@contoso.com -Credentials $cred -SourceMailboxLegacyDN "/o=First Organization/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=3fd752ef466144ff83d418e6060da2
    26-Stani"

    # If migration account is not hidden from GAL and you are not going to use Autodiscover
    $TSMA = Test-MigrationServerAvailability -ExchangeOutlookAnywhere -EmailAddress administrator@contoso.com -Credentials $cred -RPCProxyServer "us.exg7.exghost.com" -ExchangeServer "ac2b1533-71c9-48e3-9a58-8f7ce004cf01@contoso.com"

    # If migration account is hidden from GAL and you are not going to use Autodiscover
    $TSMA = Test-MigrationServerAvailability -ExchangeOutlookAnywhere -EmailAddress administrator@contoso.com -Credentials $cred -RPCProxyServer "us.exg7.exghost.com" -ExchangeServer "ac2b1533-71c9-48e3-9a58-8f7ce004cf01@contoso.com" -SourceMailboxLegacyDN "/o=First Organization/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=3fd752ef466144ff83d418e6060da2
    26-Stani"

    New-MigrationEndpoint -ExchangeOutlookAnywhere -Name CutoverEndpoint -ConnectionSettings $TSMA.ConnectionSettings

    where:
    • the -RPCProxyServer parameter specifies the FQDN of the RPC proxy server for the on-premises Exchange server (ExternalHostname); 
    • the -ExchangeServer parameter specifies the FQDN of the on-premises Exchange server (InternalHostname). 
    Get-OutlookAnywhere | fl ExternalHostname,InternalHostname

    PowerShell command above gives you right values (assuming that Outlook Anywhere is configured correctly)

    Question: Why will you choose not to use Autodiscover for migration endpoint?

    Answer: After DNS MX and A/CNAME (autodiscover.contoso.com) records switch-over to Exchange Online you can continue mailbox items sync from source Exchange platform. If you use "autodiscover" method for Cutover endpoint then:

    • Per Microsoft "Autodiscover service will be used to connect to each user mailbox in the migration batch"
    • you will be pointed (after DNS switch-over autodiscover.contoso.com to autodiscover.outlook.com) to destination Exchange Online platform instead of source Exchange Server and mailbox sync could be broken.
    References:

    Keep installed drivers after running SYSPREP

    Scenario:

    I need to keep installed drivers on an OS even after running SYSPREP, how do I do this?

    Solution:

    There may be scenarios where its critical drivers are not removed from an OS instance when SYSPREP is executed. To do this run the following prior to SYSPREP
    • Run regedit and move to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\Sysprep\Settings\sppnp 
    • Select Edit - New DWORD value 
    • Enter a name of PersistAllDeviceInstalls 
    • Set to a value of 1 and click OK
    References:

    SCOM Management Pack is dependent on Microsoft.SystemCenter.SecureReferenceOverride

    Scenario:

    You try to delete SCOM 2012+ Management Pack. You get error:
    "Following Management Packs depend of the management pack you are trying to delete. Please delete the following management packs manually first.
    Microsoft.SystemCenter.SecureReferenceOverride"
    Solution:

    No manual XML file editing.
    • Open a PowerShell session with the Operations Manager module/snappin loaded.
    • Type: $MP = Get-SCOMManagementpack -Name Microsoft.SystemCenter.SecureReferenceOverride
    • Now we can view the referenced management packs by typing $MP.References
    • From the list of items in the Key column, note down the alias of your MP you wish to delete. If you are having trouble finding it, the Value column will list the full ID of the MP.
    • Now that we know the MP alias, we can remove it from the Secure Reference MP by typing $MP.References.Remove(“yourMPAliasGoesHere“)
    • Now we can verify the MP is valid by entering $MP.Verify() to ensure there are no orphaned overrides, etc.
    • Finally, we can save our changes by typing: $MP.AcceptChanges()

    Once this is done, give the SCOM management group a chance to catch up (you might have to wait a minute and then refresh your console). When you now check the Dependencies tab of the MP you want to delete, you’ll see that the SecureReferenceOverride MP is no longer listed and you’ll be able to remove your MP.

    Reference:

    Monday, August 7, 2017

    Can't Install-WindowsFeature on Windows Server using PowerShell

    Scenario:

    You can't Install-WindowsFeature (Add-WindowsFeature for 2008 R2) on Windows Server using PowerShell for various reasons.
    Recently I had a problem with broken WMI where I couldn't manage Windows Server features of affected server. My wish was to make Full Server backup prior to any changes using built-in "Windows Server Backup" and I couldn't install this feature using PowerShell.

    Solution:

    Following DISM commands running from an Elevated Command Prompt (Admin) saved my ass:
    Dism /online /enable-feature /featurename:WindowsServerBackup
     
    Deployment Image Servicing and Management tool
    Version: 6.3.9600.17031
    
    Image Version: 6.3.9600.17031
     
    Enabling feature(s)
    [==========================100.0%==========================]
    The operation completed successfully.
    Dism /online /enable-feature /featurename:WindowsServerBackupSnapin
     
    Deployment Image Servicing and Management tool
    Version: 6.3.9600.17031
     
    Image Version: 6.3.9600.17031
     
    Enabling feature(s)
    [==========================100.0%==========================]
    The operation completed successfully.
    You can install any Windows feature in similar way. Get list of all available features and their names on server:
    Dism /online /get-features /format:table | more
    References:

    WMI is broken. "Configuration refresh failed with the following error: Invalid class"

    Scenario:

    When we start Server Manager on Windows Server 2012 R2 the flag lights up "Red" almost immediately with the error:
    "Configuration refresh failed with the following error: Invalid class"


    Get-WindowsFeature ends with the same "Invalid class" error.

    Solution:

    Backup WMI:

    • Launch the WMI MMC snapin by Start -> Run -> then enter WMIMGMT.MSC
    • Right click WMI Control (Local) and click Properties
    • Tab Backup/Restore, press Back Up Now...

    Recompile .mof & .mlf files by running the following commands from an Elevated Command Prompt (Admin):
    CD C:\Windows\System32\WBEM
    dir /b *.mof *.mfl | findstr /v /i uninstall > moflist.txt & for /F %s in (moflist.txt) do mofcomp %s
    Refresh "Server Manager" or re-open it.

    References: