In order to configure SCOM for FIPS, you will need to download some items. This link contains everything you need for this process.
As a quick overview, FIPS stands for Federal Information Processing Standards and it’s a set of algorithms that has to do with encryption of data at rest and data in transit. FIPS is not turned on by default as many applications are not FIPS compliant out of the box. You see this in use more in high security environments. Like many applications, SCOM is not FIPS compliant by default. Changes need to be made in order for it to be compliant. There’s a KB article on this subject published here. This article changed at some point and it’s currently inaccurate in a few places and somewhat misleading. I’ve placed a request to get it updated, but to date it has not been updated. Updating for FIPS is a somewhat cumbersome process, so I’ll walk through the step by step in this blog.
First off, let’s determine if FIPS is enabled. That’s easy enough to do. Open the registry and navigate to the following key:
The “Enabled” DWORD value will be set to 1. This is often set via GPO in secure environments, so simply changing this will be reverted.
Turn on FIPS and reboot your system, and your webconsole will look something like this:
I would note that on some internal cases, there can be an internal server error listed here instead.
Now to the fix: WARNING: Backup all files you touch in this section before proceeding.
Step 1: The first thing you need to do is to get a DLL file off of the SCOM CD. IMPORTANT NOTE: This file is found only on the SCOM 2012 RTM or SCOM 2012 with SP1 CD. It is not on the SCOM 2012 R2 CD. They are located in the \SupportTools\AMD64 folder of the CD. Copy the Microsoft.EnterpriseManagement.Cryptography.dll file to C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server. (note, this is your SCOM installation directory. This is the default location, but that can vary based on your setup).
Also download a copy of gacutil.exe. This isn’t easily found. It’s a part of the SDK kit for Windows as well as some of the .NET development tools. It is not in the SCOM CD. I’d make it available for download, but that doesn’t seem to be an option for this source. The SDK kit is free, and can be found here. It’s a rather large download, so be patient. For the purposes of this step, place gacutil.exe on your desktop.
Step 2: Next, we will open up a command prompt in administrative mode and then type the following:
cd desktop and press ENTER
gacutil.exe /i “C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\Microsoft.EnterpriseManagement.Cryptography.dll” and press ENTER.
Tope notepad and press ENTER.
WARNING: Back these files up before editing.
What you’ve done to install the DLL file and then open up notepad in admin mode. You want this because the files you need to edit are in protected locations on the OS, and as such, it will not let you save over them. Opening notepad as an administrator gets you around that problem.
Step 3: From notepad, open the following file:
NOTE: You’ll need to switch from viewing only text files to all files to see this in notepad. As well, if your windows directory is not the standard directory, you’ll need to adjust the path. The location of where this addition should be is not specified in the article… BUT, location DOES matter. Scroll down to the very end. Paste this information right before the </configuration> line.
WARNING: There does seem to be an issue with my blogging software and copying/pasting these characters directly into notepad. If during saving you get prompted due to unicode characters or something like that… It will not work. I’d note that usually the issue is with the quotes not being the right font type. If you have access to PowerShell ISE or Notepad++, I highly recommend using these tools as they will highlight some of these issues for you.
Note, you may want to copy this off of the web link as my formatting isn’t quite the same. It should look something like this:
Save it. Do not close Notepad.
Step 4: Repeat step 3 for the following file locations:
Note: A simple test you can do here is open Server Manager once this is done. If you get an error and Server Manager doesn’t open, something was done wrong and you will want to revisit these steps.
Step 5: From notepad, open the following file:
C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\WebConsole\WebHost\web.config (the file path is slightly different for 2016 and 2019, but the file it self will be in the same location of that tree)
This file is a bit trickier to edit due to the fact that the tags you are looking for show up twice. In every case, the tag shows up first in a commented out section. If you make the edits there, it won’t work. You have to find the second tags (see screenshot below). First locate the <encryption> tag that is not commented out and add the following line underneath it:
<symmetricAlgorithm iv=”SHA256″ />
Second, locate the <Connection element. Replace the <Connection… and the <Session lines with the following lines:
<connection autoSignIn=”True” autoSignOutInterval=”30″>
<session encryptionKey=”SessionEncryptionKey” tokenAlgorithm=”SHA256″>
Save this. It will look something like this (note I circled the comment lines so you can see them as well).
Step 6: From notepad, open the following file:
C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\WebConsole\MonitoringView\web.config
Note this is also where the KB article goes wrong, pointing you to some .NET locations. Those tags do not exist in those locations. Once this open, you can repeat step 5 covering the <encryption> element and <connection elements. Those steps do not change. There is one caveat here though. The <session> tag is closed differently between this file and the one in step 5. If you replace these directly, you’ll remove the ‘/’ tag at then end which will cause a mal-formed XML. You can simply add a line and add a </session> immediately following the <session encryptionKey=”SessionEncryptionKey” tokenAlgorithm=”SHA256″> line. The issues with the comment marks are present here as well. Make sure you update the correct elements.
After that, you need to locate the <system.web> element and paste the following lines beneath it:
<machineKey validationKey=”AutoGenerate,IsolateApps” decryptionKey=”AutoGenerate,IsolateApps” validation=”3DES” decryption=”3DES” />
Save this and close notepad.
It wouldn’t hurt to reset IIS after this point or just reboot the system.
Relaunch the web console. It should look something like this:
If you made a mistake, there will be an issue loading the web page. Detailed error messages may need to be turned on if this has not already been done. To do this, you’ll need to make edits to the web.config file for both the monitoring host and web host web applications… Please note that these changes should be undone once completed as this can pose a security risk.
In the <System.Web> section, change the <CustomErrors> value to “Off” instead of On.
In the <System.WebServer> section, add the following line:
<httpErrors errorMode="Detailed" />