Using SCOM to Detect Scheduled Task Creation

One well known thing that attackers like to do is to create scheduled tasks to periodically execute their payloads.  Detecting Scheduled task creation is not terribly difficult.  While it does not use the standard logs, there is an operational task scheduler log available that generates an event ID 106 whenever a scheduled task is created.  Information on said event can be found here.  Creating a rule to do this is very straight forward:



The downside to this is that it is not necessarily a quiet proposition.  For those that know me, noise generated by management packs is something I’m not a fan of.  Alerts should be actionable, and generating alerts that are ultimately ignored due to constant false positives will not do anyone any good.  As such, this brings us to a few problems as many organizations have legitimate uses for scheduled tasks. For the person responsible for security monitoring, they need a way of knowing which tasks are legitimate and which ones are not.  One easy way would be to configure standard tasks before the SCOM agent is installed, or use maintenance mode on the server while it is being configured prior to maintenance.  Good process can certainly get around those alerts, but there’s another problem: some applications also create scheduled tasks.

In my own lab, this is noticeable as an anti-malware scan runs periodically, and when it is set, it is set by a scheduled task:


If this is common thing in a lab with a half-dozen servers in it, then it could likely be a noisy thing in your environment.  But it is correctable.  Enter Parameterization.  The scheduled task log has two parameters:  The logging user and the task name. The task name is parameter 1, so if you have a task that is periodically being created by an application, then the task name should be the same. I’d hesitate to filter by the user name in this case as a user name such as a machine name could be generated by malicious code depending on the context of whomever generated it.

Crafting and tuning that rule is simple.


You’ll likely have some noise to start, but this is hardly something that cannot be corrected over time. Just keep adding lines for “Parameter 1” and “Does not contain” with a value that is unique to your scheduled task.

Using SCOM to Detect Service Creation

For those that cover security, this piece of information is not all that new, but for those that happen to be SCOM administrators and not necessarily security people by trade, this might be useful.

Detecting service creation is the easy part. When a service is created on a system, event ID 7045 is generated in the system log.  Note that this is not on its own a security event, and as such, you will not see anything in the security log associated with service creation.  The rule is straight forward:


So why do we care?

The answer is remote execution tools.  In order to remotely execute software on another system, tools such as PSExec will execute a payload on a remote server that starts up a service that it can communicate with.  On it own, this is not necessarily a bad thing, as IT professionals use tools like PSExec from time to time.  The problem, however, is that the bad guys use the same tools too.  Malicious malware is also known to create benign looking services on a machine that will be easily not be spotted by the untrained eye.  They use these services to allow for ease of entry and remote execution in your environment.  Detecting services specific to malware can be a bit difficult.  Some attackers are lazy and simply use the well known tools such as PSEXEC. Others write and compile their own.

Implementation can be a bit tricky. As a general rule of thumb, domain controllers are static devices. Once that server is promoted to a DC, there really should not be any new services created on it.  As such, a simple rule such as this targeted to your domain controllers should suffice.  In a typical environment, this should not generate any noise.  Member servers, on the other hand, are a bit more difficult to deal with.  Service creation can happen when an application is installed.  As such, a generic rule looking for and alerting on 7045 events can prove to be a somewhat noisy.  This will likely generate alerts, and I’ve found that when alerts are of the “false positive” variety, they start getting ignored quickly.  If you have a process in place that installs SCOM after a server is officially production, that should get around most noise caused by environmental issues, at which point the original rule will suffice.  If you have your own internal tool that creates a service on the fly, you may want to consider adding a “does not contain” expression to filter out your service as well.  You may also want to consider the alternative which modifies the 7045 to look for services tied to known attack vectors.  It would look something like this:


While I don’t have a complete list of known services associated with remote execution tools, this will detect some of the common ones.  A good attacker can get around this fairly easily, especially if they are writing their own tools, but this can provide a means of defense in depth targeting your tier 1 environment.  And I definitely recommend watching for 7045s on domain controllers.  That should not be happening on your DCs.

I’m working on a management pack that contains these monitors along with additional security rules along with other security related items that SCOM can provide. My only test environment is my lab, which is hardly a production grade environment. I do welcome feedback. While I can only support this to the best of my ability, I can provide it if this is something you are interested in trying it out. My main goal is to keep the noise down to a minimum so that each alert is actionable. While that is not always easy to do, trying this out in other environments will go along way to getting it to that point.  If this is something you are interested in testing, please hit me up on linked in or have your TAM reach out to me internally.

Using SCOM to Detect Golden Tickets


For the three people that religiously read my blog, you know by now that I’ve been writing quite a bit on using SCOM to detect some of the anomalous events that are specific to an intruder in your environment.  This is the first, of what will hopefully be a few rules needed to detect the existence of the back doors that attackers leave behind so that they can re-enter your environment at will.

As a quick matter of definition, let’s boil this down to the basics.

  • In order to obtain a golden ticket, your environment had to be already compromised:  An attacker gets them by backing up the master Kerberos ticket’s key.  In a windows environment, this key only exists on a domain controller, which means that if the attacker acquired it, they had already compromised your DC.
  • Once the attacker has obtained this key, they can re-enter your environment at will, even if you’ve already evicted them: This is the big danger to golden tickets in that they don’t need to steal credentials of a domain admin again in order to re-enter the environment. They simply create themselves a ticket under any user context and give it domain admin rights. They can re-enter the environment and steal your AD database all over again or grab whatever else that they intended to come back and get.  It’s a forged krgtgt ticket, and as such, they don’t even need an account in your environment.
  • They have much longer expiration periods than a standard ticket:  The default expiration of a ticket is 10 hours in active directory. This is configurable, so it may be different in every environment, but in order for it to work, the attacker wants to have a long period of time in which they can re-enter an environment. When mimikatz, for instance, is used to generate a golden ticket, the default expiration of this ticket is 10 years.
  • They rely on having a valid Kerberos TGT key: This is the kicker to protecting yourself from them, but as long as the key used to sign any forged ticket is valid, the attacker can still re-enter your environment.
  • The krgtgt account’s password does not change automatically: The password itself is known only to DCs, and if you manually change it, the password you specify won’t be the actual password to the key, but by default this password does not reset. If you’re at a 2003 domain functional level and go to a 2008 domain functional level, this will do a one time reset of the password as a part of that process. Beyond that, however, this password does not change. Truthfully, this should be changed periodically, but most organizations do not do it. I’d note that the new AD MP for SCOM (if you configure client monitoring) will generate alerts on the krbtgt password if it has not been reset recently.  Microsoft does have instructions online, and it has PowerShell scripts which make it easier.
  • Standard eviction can often leave you exposed to this back door:  Simply removing the attacker’s malware does not protect you.  This will need to be done in conjunction with an AD recovery. You will need to both recover the AD environment (i.e. change all the passwords), remove the malware, and reset the krbtgt password… twice.  The reason for this is that Active Directory holds a copy of two keys:  the current krbtgt key as well as the previous key.  Both the current and previous keys need to be scrubbed from memory to invalidate any existing golden tickets.
  • This is not a vulnerability unique to windows:  The purpose of this blog is monitoring the Windows environment, but any Kerberos environment is vulnerable to this type of an attack.  It is the price one pays if their tier 0 is compromised.
  • Following the procedures below will not prevent an attacker from entering the environment:  All this will do is break existing golden tickets. If the attacker has not been completely evicted from your environment, they can and will get in.  They can then turn around and regenerate a new one.  There is definitely a cost benefit analysis that should be considered before doing a double reset of the krbtgt password as described below. You can certainly now detect a golden ticket if it is in use in your environment. However, as I’ve noted, you’ve already been compromised and have likely lost quite a bit. The golden ticket is primarily used to re-enter an environment without needing to steal credentials all over again.
  • The double reset procedure (described below) is part of an eviction process. Good practice would involve periodically resetting the krbtgt account. A double reset will likely generate noise, but periodic reset (say once every 90 days) should generate no noise unless someone is doing something that they should not.

This brings us to the detection stage.  The easiest way to detect if one is in your environment is if it is used after you’ve done the double reset of the Kerberos password. This will generate an event ID of 4769 on a domain controller with a failure code of 0x1F.  Creating a rule in SCOM for this event is very easy to do (see screenshot below), but keeping it from generating noise is the part that needs to be planned out.


The reason for this is that plenty of legitimate tickets can generate 0x1F failures if the double reset happens too quickly, and our goal is not to be concerned about the legitimate tickets, but to respond when we find a ticket that is a golden ticket. As such, you need to do a Kerberos reset very carefully if you want to detect the bad guy.  By default, our 4769 rule is not going to generate any failures in a normal operating environment.  Once we start with the password resets though, this can get very noisy if not done carefully as it will break all existing tickets until new ones are obtained.

  • Step 1:  Obtain the default expiration timeline.  This same policy will give you the clock skew information (more on this in a bit). Again, by default this is 10 hours, but you may have a GPO that sets it to a different time frame, and you NEED to know the timeframe. 
  • Step 2:  Force all passwords to be changed and do the first krbtgt password reset. DONT DO THIS UNTIL YOU UNDERSTAND THE EFFECT ON YOUR ENVIRONMENT.  Remember, that if an attacker has compromised accounts, they don’t need a golden ticket. This just lets them back in at some point after the initial compromise. If you’re dealing with an active recovery, you’re going to want to also remove internet access along with any other means that the attacker may have to enter the environment.  Otherwise, they would be able to potentially generate a new key during the waiting period. In this scenario, you may want to simply do a quick double reset, but do understand that every legitimate ticket will now fail and generate a 4769 with a 0x1F failure.  That will generate a lot of noise. The purpose of golden ticket monitoring, however, is to ensure that the bad guy doesn’t re-enter at some point down the road.  If you do a rapid reset, I would not have this rule turned on.  You can turn the rule on after this has happened and you’ve waited enough time to allow for existing tickets to regenerate.
  • Step 3:  Wait, but not long.  A couple things need to happen before this rule will work properly and not generate any noise. 
    • Active Directory needs to replicate across all Domain Controllers.
    • Once replication is complete, you need to wait for tickets signed with the old key to get new tickets issued with the new key.  You obtained that value in step 1.
    • You need to also account for time skew. This is a smaller waiting period, maybe 15 minutes or so, but this can come into play.  The default clock skew is 5 minutes, so waiting twice that will account for potential clock skew issues.
    • Please note that at this step, you are still vulnerable to a golden ticket attack.
  • Step 4:  Reset the krbtgt password again.  Remember that an attacker created a ticket that may not expire for years. They are hopefully not active in your environment when you perform this process.  Also, you need to remember that if they still have your existing credentials, they don’t need a golden ticket.  If you are not careful in following the steps, the only thing accomplished is giving yourself a false sense of security.   After this is done properly, when they attempt to use the golden ticket, the rule you created will generate an alert when a forged ticket is used.

Unfortunately, what this is telling us is that your environment was compromised at some point. If you’re doing this because of an active recovery, in that sense you already know it. However, if you’ve done this as a preemptive means of removing an attacker, you now have further investigation to do to see what was stolen previously.

Disclaimer:  I am not an active directory expert, and while I’ve done my fair share of AD administration, this is not something I’ve personally done, and I would not recommend taking it lightly. I strongly recommend something like that be done under the watchful eye of someone who knows AD at a far better level than a standard administrator.  You will want to use a trusted advisor. Microsoft does provide instructions on how to do this. Doing this properly will mitigate against most of the risks associated with it, and you will of course want to have validated that your backups work prior to doing this. Following the procedure carefully should mitigate against most of the risks associated with it, but I do recommend you make it a point of reading up on all of this and making sure you have thoroughly discussed this with Microsoft support or someone who has done this procedure before.

As you can see, this will help you detect golden tickets, but it is not exactly a non-invasive solution. Solutions such as Advanced Threat Analytics can actively detect Golden Tickets without the need to reset the Kerberos password; however, that does not mean that it would be wise to ignore this as a component to your security practice.  My next step will be to see if I can figure out how to use SCOM to detect this in a non-invasive manner.  I’m not sure if that is possible, but for now, this will work.

I’m working on a management pack that contains these monitors along with additional security rules along with other security related items that SCOM can provide. My only test environment is my lab, which is hardly a production grade environment. I do welcome feedback. While I can only support this to the best of my ability, I can provide it if this is something you are interested in trying it out. My main goal is to keep the noise down to a minimum so that each alert is actionable. While that is not always easy to do, trying this out in other environments will go along way to getting it to that point.  If this is something you are interested in testing, please hit me up on linked in or have your TAM reach out to me internally.