User accounts created to be used as service accounts rarely have their password changed. Group Managed Service Accounts (GMSAs) provide a better approach (starting in the Windows 2012 timeframe). The password is managed by AD …
For most of 2019, I was digging into Office 365 and Azure AD and looking at features.As I went through each of them, I found one that was very interesting. While Azure leverages Azure Active …
Many are familiar with Active Directory, the on-premises directory and authentication system that is available with Windows Server, but exactly what is Azure Active Directory? Azure Active Directory (Azure AD or AAD) is a multi-tenant …
Attacking and Defending the Microsoft Cloud (Office 365 & Azure AD) Sean Metcalf (Trimarc) & Mark Morowczynski (Principal Program Manager, Microsoft) The allure of the “Cloud” is indisputable. Organizations are moving into the cloud at …
Windows Server 2019 has several new features, though nothing in this list is related to AD. Note that there is no Windows Server 2019 AD Forest/Domain Functional Level. There are no new features for Active …
For most of 2019, I was digging into Office 365 and Azure AD and looking at features. As I went through each of them, I found one that was very interesting.
While Azure leverages Azure Active Directory for some things, Azure AD roles don’t directly affect Azure (or Azure RBAC) typically. This article details a known configuration (at least to those who have dug into Azure AD configuration options) where it’s possible for a Global Administrator (aka Company Administrator) in Azure Active Directory to gain control of Azure through a tenant option. This is “by design” as a “break-glass” (emergency) option that can be used to (re)gain Azure admin rights if such access is lost. In this post I explore the danger associated with this option how it is currently configured (as of May 2020).
The key takeaway here is that if you don’t carefully protect and control Global Administrator role membership and associated accounts, you could lose positive control of systems hosted in all Azure subscriptions as well as Office 365 service data.
Note: Most of the research around this issue was performed during August 2019 through December 2019 and Microsoft may have incorporated changes since then in functionality and/or capability.
Attack Scenario: In this scenario, Acme has an on-premises Active Directory environment. Acme embraced Azure Infrastructure as a Service (IAAS) as an additional datacenter and deployed Domain Controllers to Azure for their on-prem AD (as their “cloud datacenter”). Acme IT locked down the DCs following hardening advice and limited Azure administration to the VMs hosting the DCs. Acme has other sensitive applications hosted on servers in Azure.
Acme signed up for Office 365 and started a pilot. All of the Active Directory and Exchange admins (and many other IT admins) are granted temporary Global Administrator (aka Global Admin or GA) rights to facilitate the pilot. So, more than should be there and not well protected.
There are many ways an attacker can gain Domain Admin rights in Active Directory. This post is meant to describe some of the more popular ones in current use. The techniques described here “assume breach” where an attacker already has a foothold on an internal system and has gained domain user credentials (aka post-exploitation).
The unfortunate reality for most enterprises, is that it often does not take long from an attacker to go from domain user to domain admin. The question on defenders’ minds is “how does this happen?”.
The attack frequently starts with a spear-phishing email to one or more users enabling the attacker to get their code running on a computer inside the target network. Once the attacker has their code running inside the enterprise, the first step is performing reconnaissance to discover useful resources to escalate permissions, persist, and of course, plunder information (often the “crown jewels” of an organization).
While the overall process detail varies, the overall theme remains:
Malware Injection (Spear-Phish, Web Exploits, etc)
Reconnaissance (Internal)
Credential Theft
Exploitation & Privilege Escalation
Data Access & Exfiltration
Persistence (retaining access)
We start with the attacker having a foothold inside the enterprise, since this is often not difficult in modern networks. Furthermore, it is also typically not difficult for the attacker to escalate from having user rights on the workstation to having local administrator rights. This escalation can occur by either exploiting an unpatched privilege escalation vulnerability on the system or more frequently, finding local admin passwords in SYSVOL, such as Group Policy Preferences.
The past couple of years of meeting with customers is enlightening since every environment, though unique, often has the same issues. These issues often boil down to legacy management of the enterprise Microsoft platform going back a decade or more.
While the threats have changed over the past decade, the way systems and networks are managed often have not. We continue with the same operations and support paradigm despite the fact that internal systems are compromised regularly. We must embrace the new reality of “Assume Breach.”
Assume breach means that we must assume that an attacker has control of a computer on the internal network and can access the same resources the users who have recently logged on to that computer has access to. Note that when I describe risks and mitigations of Active Directory,this includes overall enterprise configuration.
Here are some of the biggest AD security issues (as I see them). This list is not complete, but reflects common enterprise issues.
I continue to find many of these issues when I perform Active Directory Security Assessments for organizations.
User accounts created to be used as service accounts rarely have their password changed. Group Managed Service Accounts (GMSAs) provide a better approach (starting in the Windows 2012 timeframe). The password is managed by AD and automatically changed. This means that the GMSA has to have security principals explicitly delegated to have access to the clear-text password. This is where it can get tricky.
Key Points for Group Managed Service Accounts (GMSAs) :
The GMSA password managed by AD.
Computers hosting GMSA service account(s) request current password from Active Directory to start service.
Configure the GMSA to allow computer accounts access to password.
If an attacker compromises computer hosting services using GMSA, the GMSA is compromised.
If attacker compromises an account with rights to request GMSA password, the GMSA is compromised.
Running the AD PowerShell cmdlet Get-ADServiceAccount, we can retrieve information about the GMSA is a member of the domain Administrators group which has full AD & DC admin rights to the domain. The screenshot shows that the password changed recently and won’t change for a few weeks. This means that if we can get the password for this account, we have almost a month to use the account credentials before it changes. There’s a group that can retrieve the password. We’ll take a look at this is a bit.
Many are familiar with Active Directory, the on-premises directory and authentication system that is available with Windows Server, but exactly what is Azure Active Directory?
Azure Active Directory (Azure AD or AAD) is a multi-tenant cloud directory and authentication service. Azure AD is the directory service that Office 365 (and Azure) leverages for account, groups, and roles. It is also an Identity Provider (IPD) and supports federation (SAML, etc). Note: given how rapidly the cloud changes, elements of this post may become out of date soon after the original post date. Azure AD is highly available and globally deployed.
Azure AD is deployed in over 30 datacenters around the world leveraging Azure Availability Zones where present. This number is growing rapidly as additional Azure Regions are deployed.
For durability, any piece of data written to Azure AD is replicated to at least 4 and up to 13 datacenters depending on your tenant configuration. Within each data center, data is again replicated at least 9 times for durability but also to scale out capacity to serve authentication load. To illustrate—this means that at any point in time, there are at least 36 copies of your directory data available within our service in our smallest region. For durability, writes to Azure AD are not completed until a successful commit to an out of region datacenter.
This approach gives us both durability of the data and massive redundancy—multiple network paths and datacenters can serve any given authorization request, and the system automatically and intelligently retries and routes around failures both inside a datacenter and across datacenters.
To validate this, we regularly exercise fault injection and validate the system’s resiliency to failure of the system components Azure AD is built on. This extends all the way to taking out entire datacenters on a regular basis to confirm the system can tolerate the loss of a datacenter with zero customer impact. … Azure AD is already a massive system running on over 300,000 CPU Cores and able to rely on the massive scalability of the Azure Cloud to dynamically and rapidly scale up to meet any demand. This can include both natural increases in traffic, such as a 9AM peak in authentications in a given region, but also huge surges in new traffic served by our Azure AD B2C which powers some of the world’s largest events and frequently sees rushes of millions of new users. … To support the health checks that gate safe deployment and give our engineering team insight into the health of the systems, Azure AD emits a massive amount of internal telemetry, metrics, and signals used to monitor the health of our systems. At our scale, this is over 11 PetaBytes a week of signals that feed our automated health monitoring systems.
Azure Active Directory is Not Cloud AD Azure Active Directory is not Active Directory hosted in the cloud. There is no standard AD authentication methods such as NTLM or Kerberos; no LDAP; and no group policy (GPO), so Azure AD won’t work for traditional on-prem applications.
There are cloud hosted Active Directory environments that can be used to manage cloud workloads in Microsoft Azure (Azure Active Directory Domain Services), Amazon AWS (Amazon Managed Microsoft AD), and Google Cloud (Managed Service for Microsoft Active Directory (AD)). These are all hosted Microsoft Active Directory environments which have 2 Domain Controllers (or more) and the tenant admins do not receive Domain Admin rights to the hosted AD environment; only delegated access is provided which often includes the ability to create/manage resources in a specific OU and specific GPOs.
Note: I don’t have room to include a comparison of these services here, but may write a future post if there’s interest (I did some research comparing Microsoft Azure vs Amazon AWS hosted AD service offerings in 2017).
Primary Management Tools The tool that most AD administrators are familiar with is Active Directory Users and Computers aka ADUC (MMC tool).
Azure Active Directory administrators will primarily use the web console at https://portal.azure.com to administer the environment.
Admins that manage Active Directory on-prem and now Azure AD/Office 365 will be using the on-prem MMC tools as well as the web admin portals (and various URLs associated with them). There are PowerShell cmdlets available for managing Azure AD (similar to on-prem), though cloud features often move faster than the PowerShell tools are released, which means that using the cloud admin portal should still be used, even when using PowerShell.
Interfacing with Azure Active Directory Since Azure AD doesn’t have LDAP, interfacing with AAD involves connecting via the Graph API (or PowerShell modules). I like PowerShell, so I use the PowerShell modules (or Portal websites) for management and reporting.
There are 2 primary PowerShell modules for interfacing with Azure AD: MSOnline and AzureAD. These can be installed through the PowerShell install feature: Install-Module -Name MSOnline -Force Install-Module -Name AzureAD -Force
The AzureAD module may eventually replace the MSOnline PowerShell module, but there are features available in MSOnline that haven’t been ported to the Azure AD module (yet).
Attacking and Defending the Microsoft Cloud (Office 365 & Azure AD) Sean Metcalf (Trimarc) & Mark Morowczynski (Principal Program Manager, Microsoft)
The allure of the “Cloud” is indisputable. Organizations are moving into the cloud at a rapid pace. Even companies that have said no to the Cloud in the past have started migrating services and resources. The Cloud is a new paradigm and the rapid update pace makes it difficult to keep up, especially when it comes to security.
This presentation focuses on the Microsoft Cloud (Office 365 & Azure AD) and explores the most common attacks against the Cloud and describes effective defenses and mitigation. While the content is focused on the Microsoft Cloud, some of the attack and defense topics are applicable to other cloud providers and are noted where applicable.
Key items covered:
Attacks against the Cloud
Account compromise and token theft
Methods to detect attack activity
Cloud identity firewall
Securing cloud infrastructure against attacks
Secure cloud administration
There are no new features for Active Directory in Windows Server 2019 except one performance update which doesn’t affect most deployments. This update is related to an updated algorithm that better supports the ESE version store on DCs. Ryan Ries describes this on the Ask DS blog:
The intent of the first section of this article is to discuss how Active Directory’s sizing of the ESE version store has changed in Server 2019 going forward. The second section of this article will discuss some basic debugging techniques related to the ESE version store.
Active Directory, also known as NT Directory Services (NTDS,) uses Extensible Storage Engine (ESE) technology as its underlying database.
One component of all ESE database instances is known as the version store. The version store is an in-memory temporary storage location where ESE stores snapshots of the database during open transactions. This allows the database to roll back transactions and return to a previous state in case the transactions cannot be committed. When the version store is full, no more database transactions can be committed, which effectively brings NTDS to a halt.
In 2016, the CSS Directory Services support team blog, (also known as AskDS,) published some previously undocumented (and some lightly-documented) internals regarding the ESE version store. Those new to the concept of the ESE version store should read that blog post first.
In the blog post linked to previously, it was demonstrated how Active Directory had calculated the size of the ESE version store since AD’s introduction in Windows 2000. When the NTDS service first started, a complex algorithm was used to calculate version store size. This algorithm included the machine’s native pointer size, number of CPUs, version store page size (based on an assumption which was incorrect on 64-bit operating systems,) maximum number of simultaneous RPC calls allowed, maximum number of ESE sessions allowed per thread, and more.
Since the version store is a memory resource, it follows that the most important factor in determining the optimal ESE version store size is the amount of physical memory in the machine, and that – ironically – seems to have been the only variable not considered in the equation!
The way that Active Directory calculated the version store size did not age well. The original algorithm was written during a time when all machines running Windows were 32-bit, and even high-end server machines had maybe one or two gigabytes of RAM.
As a result, many customers have contacted Microsoft Support over the years for issues arising on their domain controllers that could be attributed to or at least exacerbated by an undersized ESE version store. Furthermore, even though the default ESE version store size can be augmented by the “EDB max ver pages (increment over the minimum)” registry setting, customers are often hesitant to use the setting because it is a complex topic that warrants heavier and more generous amounts of documentation than what has traditionally been available.
Service accounts are that gray area between regular user accounts and admin accounts that are often highly privileged. They are almost always over-privileged due to documented vendor requirements or because of operational challenges (“just make it work”).
We can discover service accounts by looking for user accounts with Kerberos Service Principal Names (SPNs) which I call SPN Scanning. Service accounts without SPNs can also be discovered by querying AD accounts for ‘SVC’, or ‘Service’, or common vendor product names.
The following PowerShell commands require the Active Directory PowerShell module.
Discover service accounts (user accounts with SPNs):
While Domain Admins is the most commonly used AD admin group, there are several others that could be used.
Common privileged AD groups that may contain Service Accounts:
Administrators
Full administrative rights to the AD domain and Domain Controllers.
Domain Admins
Full administrative rights to computers joined to the domain (default) and full administrative rights to the AD domain and DCs (through membership in the Administrators group).
Backup Operators
Default rights to backup and restore Active Directory and Domain Controllers.
Server Operators
Able to logon to Domain Controllers and provides ability to perform some administrative actions on Domain Controllers.
Enterprise Admins
Full administrative rights to all domains and Domain Controllers in the AD forest (through membership in the Administrators group). Also has special forest admin rights such as DHCP. In a single domain forest, this group should remain empty until needed.
Schema Admins
Able to modify the AD schema for the forest. This group should remain empty until needed.
The Issue Recently a blog post was published by Dirk-jan Mollema titled “Abusing Exchange: One API call away from Domain Admin ” (https://dirkjanm.io/abusing-exchange-one-api-call-away-from-domain-admin/)which highlighted several issues with Exchange permissions and a chained attack which would likely result in a regular user with a mailbox being able to become a Domain Admin in the AD forest. Tools have been released to take advantage of this issue.
He highlights the key components of the issue in the blog post introduction:
This blog combines a few known vulnerabilities and known protocol weaknesses into a new attack. There are 3 components which are combined to escalate from any user with a mailbox to Domain Admin access:
* Exchange Servers have (too) high privileges by default * NTLM authentication is vulnerable to relay attacks * Exchange has a feature which makes it authenticate to an attacker with the computer account of the Exchange server … The main vulnerability here is that Exchange has high privileges in the Active Directory domain. The Exchange Windows Permissions group has WriteDacl access on the Domain object in Active Directory, which enables any member of this group to modify the domain privileges, among which is the privilege to perform DCSync operations. Users or computers with this privilege can perform synchronization operations that are normally used by Domain Controllers to replicate, which allows attackers to synchronize all the hashed passwords of users in the Active Directory.
Exchange Permissions in Active Directory
At the Microsoft Blue Hat in 2017, Sean Metcalf, Trimarc founder and Active Directory Subject Matter Expert (SME) highlighted issues with Exchange permissions. Some slides from this presentation are shown here as representative samples (the full presentation slide deck is in the Presentations section.
More information about this issue has been highlighted in presentations by Andy Robbins (@_wald0) and Will (@Harmj0y), including at Black Hat USA 2017.
The Bloodhound tool written by Andy Robbins, Rohan Vazarkar, and Will can identify attack paths involving Exchange permissions configured in Active Directory.
Microsoft recently published an article (https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/ADV190007) on how to configure EWS throttling which will mitigate the issue(s) that Dirk-jan Mollema raised. The issue with this is that EWS throttling could negatively impact applications like Outlook for Mac and doesn’t resolve the escalation path with Exchange permissions configured in AD.
It’s been almost 1.5 years since the Medium post by Shay Ber was published that explained how to execute a DLL as SYSTEM on a Domain Controller provided the account is a member of DNSAdmins.
I finally got around to posting here since many I speak with aren’t aware of this issue.
Shay describes this issue as follows (bolded text addedby me):
In addition to implementing their own DNS server, Microsoft has also implemented their own management protocol for that server, to allow for easy management and integration with Active Directory domains. By default, domain controllers are also DNS servers; DNS servers need to be reachable and usable by mostly every domain user. This, in turn, exposes quite some attack surface on domain controllers — on one part, the DNS protocol itself and on the other, the management protocol, which is based on RPC.
We will shallowly delve into the protocol’s implementation and detail a cute feature (certainly not a bug!) which allows us, under some circumstances, to run code as SYSTEM on domain controllers, without being a domain admin. Although this is certainly not a security vulnerability (so no panic is needed), as confirmed with Microsoft, it’s still a cute trick which can be useful as an AD privilege escalation in red team engagements.
So, how is this possible?
I will summarize Shay’s excellent technical review of this issue (this assumes DNS runs on Domain Controllers, which is the most common configuration).
At DerbyCon 8 (2018) over the weekend Will Schroeder (@Harmj0y), Lee Christensen (@Tifkin_), & Matt Nelson (@enigma0x3), spoke about the unintended risks of trusting AD. They cover a number of interesting persistence and privilege escalation methods, though one in particular caught my eye.
Overview
Lee figured out and presents a scenario where there’s an account with unconstrained delegation configured (which is fairly common) and the Print Spooler service running on a computer, you can get that computers credentials sent to the system with unconstrained delegation as a user.
I just uploaded the slides from my Black Hat & DEF CON talks from the past week in Vegas. They are a bit different with the BH talk more Blue (defensive) and the DC talk mostly Red (Offensive) in focus. Also note that the only real overlap in content is the MFA & password vault sections and those were updated in my DEF CON talk to focus on the attack aspect.
An important note: The methods I show are real and work well in many real-world customer deployments. The issues with MFA and password vaults I highlight are often deployment issues and not necessarily vendor best practices. With that noted, I have seen enterprise password vaults deployed with poor security so often that I don’t think customers are very familiar with the vendor security best practices.
Black Hat USA 2018 Talk: “From Workstation to Domain Admin: Why Secure Administration isn’t Secure and How to Fix it”
This talk walks the audience through how AD administration has evolved over time with newer, more “secure” methods and the potential ways to exploit modern AD administration. I explore some methods to exploit current implementation weaknesses in many deployments of multi-factor authentication (MFA) and enterprise password vaults. The latter third of the talk dives into the best defenses and how to employ and deploy them appropriately.
[Slides]
Black Hat Talk Agenda:
Current State
Evolution of Administration
Exploiting Typical Administration
Common Methods of Protecting Admins (& bypassing them)
MFA
Enterprise Password Vaults
Admin Forest
Building the Best Defenses
DEF CON 26 Talk: “Exploiting Active Directory Administrator Insecurities”
This talk repeats the slide concepts from my Black Hat talk specific to exploiting current implementation weaknesses in many deployments of multi-factor authentication (MFA) and enterprise password vaults. The talk adds in some challenges in properly discovering AD admins and some additional methods of exploiting current AD environments. I also cover how in many environments it may be possible to compromise a Read-Only Domain Controller to compromise the AD forest. This talk also includes a special, new sneaky AD persistence method which only the DEF CON audience was privy to (not in the slides, at least not directly). I will post a blog article as time allows. 🙂
[Slides]
DEF CON Talk Agenda:
Evolution of Admin Discovery
Exploiting Typical Administration
Multi-Factor Authentication (MFA)
Password Vaults
Admin Forest
Attacking RODCs
Thank you all for your support and your kind words!
– Sean
