In emails from CDK Global’s Customer Care division that many dealerships received over this past weekend, CDK anticipated that the restoration process would take “several days and not weeks” to bring their major applications back online. More recently, CDK’s Customer Care division has informed users that it is likely that many dealerships will not be live until after June 30. And even as recent as Wednesday of this week, CDK indicated that a small test group of dealerships had been brought back online successfully.
While portions of the system may soon become available to all dealerships, most experts agree that a full system restoration is unlikely, and that the more likely outcome is a partial system restoration that could take days, perhaps weeks, to fully implement. As a follow-up to our article, “Three Things Your Dealership Can Do to Deter Third-Party Cyber Incidents,” this alert is a list of action items to consider as your dealership prepares to return to the CDK environment.
Accounting & Controls
- Review key CDK setups for each department prior to resuming business as usual and before entering manual transactions into the system.
- Immediately produce a trial balance and compare it to your most recent trial balance pre-shutdown.
- Consider recreating a May 31, 2024 dealer financial statement and compare it to the one that was prepared before the CDK disruption. Investigate and reconcile any differences.
- Update daily cash reconciliations before any new manual entries are posted.
- After entering manual transactions that took place during the shutdown, consider performing physical inventories of new and used vehicles, service and loaner vehicles, and parts and accessories. Be sure to compare such physical inventory to other third-party sites outside of the CDK environment.
- Review deal jacket checklists for any missing paperwork that may need to be completed, needs a wet signature, etc.
- Payroll considerations:
- How have you been paying employees—calculated pay or advances?
- Consider commission calculations—manual versus system generated.
- Consider an out-of-sequence payroll and whether your payroll provider could accommodate this.
- Consider the timing of payrolls and related tax deposits for the second quarter.
- After posting manual transactions, review the accounting schedules for any unusual balances, control numbers, etc.
- Conduct a review of open repair orders and open parts tickets.
- Perform reconciliations of key accounts, e.g., cash, floorplan, factory accounts, etc. Investigate and resolve reconciling items as quickly as possible.
- With month-end approaching quickly, consider alerting your original equipment manufacturers (OEMs) as to a likely delay in producing a June financial statement.
- If your dealership group has outside financial statement reporting requirements and/or debt covenant compliance requirements, consider alerting your banker as to a likely delay in meeting those deadlines.
Audit Trail Considerations
We recognize that the security of your network and the integrity of your data will be your top priority as you return to the CDK environment, and at the same time, many dealership groups undergo an external financial statement audit. For those groups, preservation of the “audit trail” will be very important, and we will cover some tips and best practices around this topic in a future alert.
Network & Security
We believe it is safe to assume the worst from an overall network and security standpoint and would encourage you to react accordingly. As such, we have provided the following checklist of areas for IT staff members to assess to identify some potential gaps that, once remediated, would strengthen the dealership’s security posture. If a managed service provider (MSP) is being used to maintain dealership networks, these questions will assist in ensuring the MSP covers all of these areas.
- Privileged Account Procedures & Password Policies/Password Resets
- Perform a systemwide password reset for all domain users, administrators, and service accounts (if possible).
- Perform password audits to analyze current passwords.
- Prohibit the use of dictionary words and hybrids for passwords. Users should be trained to think in terms of passphrases rather than passwords.
- Prohibit variations of “password,” “month names,” “season names,” and “first names” as passwords.
- Cannot be the same as the previous 12 passwords.
- Maximum password age of indefinite, 365, or most days allowed by industry regulation.
- Minimum password age of one day.
- Minimum password length of 12 characters.
- Ensure domain controllers are not configured to store LAN manager (LM) hashes of passwords if backward compatibility is not required.
- Require 15-character passwords with complexity for administrator accounts and privileged accounts.
- No duplication of exact passwords among any of these three administrator accounts, e.g., domain administrator passwords should be different than both the local server and workstation administrator passwords.
- Perform password audits to analyze current passwords.
- Perform a systemwide password reset for all domain users, administrators, and service accounts (if possible).
- Asset Inventory
- Make sure the network blueprint and design documents are updated to include all assets.
- Include all external (internet-facing) and internal devices for all dealership infrastructures (on-premises and cloud-hosted).
- Ensure copies are stored off-network or air-gapped along with the incident response policy listed below.
- Attack Surface Management
- Perform a vulnerability scan of all internet-facing and internal devices.
- Eliminate all unnecessary services from the internet that do not need to be enabled. Logins to firewalls and logins to internet of things (IoT) devices should all be removed from the internet.
- All legacy devices should be removed or segmented from the enterprise networks via a firewall or access control list (ACL) because they are exploitable, and passwords for these devices should not mirror any other privileged account on other workstations or servers (local administrators on workstations and servers).
- Perform a vulnerability scan of all internet-facing and internal devices.
- Follow patch management procedures for all operating systems (OS), applications installed on end-user workstations and servers (Office, Adobe, etc.), and firmware updates to appliances (switches, routers, firewalls, etc.).
- End-User Awareness Training
- Threat actors try to take advantage of businesses during times of stress. Email all users and remind them to not provide sensitive information (passwords) or details to someone calling in pretending to be IT, a third-party vendor, or internal users like the CFO.
- Threat actors may compromise vendors' emails and send emails to dealerships pretending to be another organization via a legitimate email (BEC), so this includes potential emails that appear to be from a legitimate source too.
- No ACH or wire transfer should be performed without following all proper procedures regardless of the urgency, including if the email comes from your CFO.
- Backup & Restore Capabilities
- Ensure you have tested the backup and restore capabilities for servers and workstations, including application servers.
- Ensure backups are immutable or that you have golden images for all devices that are air-gapped from the enterprise network.
- Most organizations can restore servers easily, but how do you reimage all end-user workstations if ransomware events cross multiple locations? If this process is not highly automated, this a critical issue you should immediately address.
- Once the DMS has restored dealerships’ customer data, it can help if your dealership can also back up its data outside of the DMS. This would allow dealerships to have copies of their data outside of the system, store the data in their networks (cloud or on-premises), back up data daily, and review/test regularly to ensure dealerships have copies of core customer data.
- This security risk was identified in the most recent DMS security incident, where a critical/core vendor (DMS) had the potential to lose every dealership’s customer data and may have been unable to remediate this threat post-security incident.
- It can also be helpful for dealerships to audit core vendors to aid in ensuring their critical/core vendors are adhering to industry best practices, along with the dealership’s expectations and service level agreements (SLAs).
- Incident Response (IR)
- Ensure you have a formal and updated IR policy along with a contact tree for all key stakeholders, including vendors, that is air-gapped from the enterprise network. If the IR policy does not cover security incidents (core vendor being breached), then bring this up to date.
- It is also highly recommended to test the strength of the IR plan using a tabletop exercise to train the response team and identify gaps in the plan.
- Managed Detection and Response/Endpoint Protection (MDR/EPP)
- Ensure your MDR/EPP (antivirus) technologies are updated and full-disk scans are being performed. A gap assessment should be performed against asset reporting back, and assets that have these technologies deployed on them when comparing these to asset inventory of devices described above. Ensure these technologies are deployed enterprise wide.
- If you are running antivirus programs without EPP capabilities, you should investigate upgrading to an endpoint protection software.
- IDS/IPS Implementation & Automated Event Logging
- Ensure logging and alerting are enabled domainwide. Can you detect changes to the domain administrators group? Have IT staff add and remove a sample domain administrator to the domain controller and see if you have real-time visibility into this event via an alert to IT or the MSP.
If you cannot detect or “see” this event, then you will not see threat actors when they are in your network possibly triggering other critical events listed below:
- Encrypting files on endpoint host machines
- Disabling recovery services on endpoint host machines
- Deleting log files on endpoint host machines
- Suspending MDR/EPP technologies endpoint host machines
- Deleting shadow copies on endpoint host machines
- Establishing a remote connection through a firewall to a command and control (C2) server and internal endpoint
If any of those events were detected in real-time, it may be time to sever all enterprise internet connections.
- Are you monitoring your logs and alerts 24/7? If not, you may want to talk with a managed security services provider to assist.
- Firewalls
- Ensure firewalls are updated and logging is enabled and being reviewed daily.
- Ensure access control list and entries (ACLs/ACEs) are hardened grant based on IP address and services needed and clearly labeled.
- Ensure hardening procedures have been performed against a reputable benchmark guideline, such as the Center for Internet Security (CIS).
- Can your firewall detect the exfiltration of data to known indicators of compromise (IoC) via the C2 server?
- Web Traffic Filter
- Eliminate unnecessary end-user traffic to non-business-related websites, and block by categories (if possible).
- Ensure that outbound DNS traffic is inspected and filtered with appropriate technologies that prevent outbound DNS abuses by threat actors and their C2 servers.
- Email Filter/Email Hardening
- Ensure DMARC, SPF, and DKIM are configured appropriately for the enterprise email system.
- We recommend using an industry leader for email security along with a secondary behavior analysis technology leader that integrates with the security email gateway. The majority of data breaches occur via this vector, and using multiple layers of security is considered an industry best practice.
- Scripting & Hardening Procedures
- Ensure Windows command and PowerShell are restricted via group policy to only network administrator users.
- Ensure only the required objects have DCSync privileges.
- Ensure that only service accounts have service principal names (SPNs) for Kerberos support and have strong and complex passwords (>15 characters).
- Identify users that have privileges to perform shadow credential attacks and reduce their privilege levels to prevent the attack. You may also add additional security control parameters to deny the principle “everyone” from modifying the attribute msDS-KeyCredentialLink for any privileged account.
- Application Allow Listing
- This control is usually missing in most organizations’ networks, but it is a strong defense control in the event the MDR/EPP fails to prevent a threat actor’s payload from executing.
- Data Loss Prevention (DLP)
- Ensure your DLP controls and visibility capabilities have visibility into end-user endpoints, outbound traffic through firewalls, email servers, and potential C2 servers via DNS traffic.
- Network Segmentation
- Ensure the enterprise networks are segmented in a way to resist lateral movement attacks (firewalls between dealerships) to ensure the continuity of locations, where one location’s security does not affect another rooftop dealership under the same ownership group.
- Ensure Microsoft Local Administrator Password Solution (LAPS) is enabled and local administrator accounts are removed.
- Windows endpoint client-to-client should be blocked.
- Remote access protocols should be hardened and restricted to IT staff members or the MSP managing the dealership's network.
We recognize that the above list may seem overwhelming, yet we feel that this is an unprecedented event that requires a comprehensive response. If you would like further information or have questions about these network and security matters, please contact a professional at Forvis Mazars.