Tag: blueteam

Large-Scale Hunting and Collection from Immature Networks

Incident Responders often find themselves investigating computer incidents for clients who may not have the best security posture – lack of centralized SIEM logging, mature EDR deployment and general inability to centrally query or otherwise collect data from across the network.

This is a real problem when investigating – it means you can’t rapidly pivot on Indicators of Compromise (IOCs) such as IP addresses/Ports, Process information (names, commandlines, etc), User activity, Scheduled Task or Service metadata such as known-bad names or binaries and other system information. Without centralized tooling or logging, using IOCs or Indicators of Attack (IOAs) can be extremely difficult.

I’ve previously made some scripts to aid my own objectives to solve this problem such as WMIHunter (https://github.com/joeavanzato/WMIHunter/tree/main) and variations of using WMI at-scale in C# and Go respectively – but recently I wanted to revisit this problem and make a more modular and flexible solution.

I’d like to introduce a tool I wrote aimed at solving this problem and providing DFIR professionals another open-source solution – hence, omni [https://github.com/joeavanzato/omni].

At its core, omni is an orchestration utility providing analysts the means to execute commands on hundreds or thousands of remote devices simultaneously and transparently collect and aggregate the output. This means any command, script, tool or anything else that you as an analyst want to execute and collect some type of output from, omni helps make that easy to achieve at-scale.

Can omni help you?

Ask yourself these questions – if the answer to any of these is ‘yes’, omni can help you.

  • Do you have a need to execute and collect the results of one or more commands/scripts/tools on multiple devices concurrently?
  • Do you need to collect data from a large amount of devices that are not connected to the internet?
  • Have you ever run into issues trying to rapidly pivot on indicators of compromise across a large number of devices due to lack of data/logging/agents?
  • Does the current environment lack a centralized logging solution or EDR that can help you quickly query devices?
  • Do you need to execute a series of triage scripts on 1 or more networked devices?

As an example, let’s consider running processes and TCP connections – both are extremely common to collect to aid reactive hunts on known-bad during an engagement. omni works by allowing users to build a YAML configuration file containing command directives to be executed on targets – we can add, subtract or modify from this file as needed to serve any type of unique requirements. Below is an example of one way you could capture this data with omni:

command: powershell.exe -Command "Get-WmiObject -Class Win32_Process -Locale MS_409 -ErrorAction SilentlyContinue | Select PSComputerName,ProcessName,Handles,Path,Caption,CommandLine,CreationDate,Description,ExecutablePath,ExecutionState,Handle,InstallDate,Name,OSName,ProcessId,ParentProcessId,Priority,SessionId,Status,TerminationDate | Export-Csv -Path '$FILENAME$' -NoTypeInformation"
file_name: $time$_processes.csv
merge: csv
id: processes
tags: [quick, process, processes, builtin]

The above configuration tells omni to run the PowerShell command, automatically replacing any placeholder variables with the specified file-name – then omni knows that once collection is done, this file-name should be collected from the targets.

It is also possible to copy a script to the target and execute this, allowing omni to facilitate analysts with running more complex triage tools remotely.

command: powershell.exe C:\Windows\temp\ExtractLogons.ps1 -DaysBack 14 -OutputFile $FILENAME$
file_name: $time$_LogonActivity.csv
merge: csv
id: LogonActivity
tags: [access, user, builtin]
dependencies: [utilities\ExtractLogons.ps1]

The dependencies block allows users to specify one or more files or directories that this directive requires exist on the target prior to execution – dependencies are always copied into a single directory (C:\Windows\Temp) and then removed once execution is complete. Dependencies can also specify an http file that will be retrieved during parsing configuration.

dependencies:[https://raw.githubusercontent.com/joeavanzato/Trawler/refs/heads/main/trawler.ps1]

Typically though, if your configuration requires some remote files for download, you will be better off using the Preparation section of the configuration – this allows for commands to be executed for preparing the analysis environment – usually this means downloading any necessary tools that you want to deploy to targets, such as Autoruns or the Eric Zimmerman parsing toolset. 

preparations:
  - command: powershell.exe -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/EricZimmerman/Get-ZimmermanTools/refs/heads/master/Get-ZimmermanTools.ps1'))"
    note: Download and execute Get-ZimmermanTools into current working directory
  - command: powershell.exe -Command "iwr -Uri 'https://download.sysinternals.com/files/Autoruns.zip' -OutFile .\Autoruns.zip ; Expand-Archive -Path Autoruns.zip -Force"
    note: Download and unzip Autoruns

This can be used to help ensure that required dependencies exist prior to executing your configuration.

When omni runs, it will create two folders – ‘devices‘ and ‘aggregated‘ – inside devices a directory is created for each target device that will contain all data collected for that target. Aggregated will store any merged files once collection is complete depending on configuration settings – for example, all running processes for all computers if using the first config specified in this post.

Devices folder contains individual results, Aggregated folder contains merged results from all devices

Keep in mind – omni is designed to facilitate rapid and light-weight network-wide hunting – although it is of course possible to execute and collect any type of evidence – for example, launching KAPE remotely and collecting the subsequent zips from specified targets, like below:

command: C:\windows\temp\kape\kape.exe --tsource C --tdest C:\Windows\temp\kape\machine\ --tflush --target !SANS_Triage --zip kape && powershell.exe -Command "$kapezip = Get-ChildItem -Path C:\Windows\temp\kape\machine\*.zip; Rename-Item -Path $kapezip.FullName -NewName '$FILENAME$'"
file_name: $time$_kape.zip
merge: pool
id: kape
add_hostname: True
dependencies: [KAPE]

Of course, doing this across thousands of devices would results in a massive amount of data, but for a more limited scope this could be a highly effective means of collecting evidence – choose your configurations and targets carefully.

Below are some common command-line examples for launching omni:

omni.exe -tags builtin
- Launch omni with all targets from .\config.yaml having tag 'builtin' with default timeout (15) and worker (250) settings, using Scheduled Tasks for execution and querying AD for enabled computers to use as targets

omni.exe -workers 500 -timeout 30 -tags quick,process
- Add more workers, increase the timeout duration per-target and only use configurations with the specified tags

omni.exe -targets hostname1,hostname2,hostname3
omni.exe -targets targets.txt
- Use the specified computer targets from command-line or file

omni.exe -method wmi
- Deploy omni using WMI instead of Scheduled Tasks for remote execution

omni.exe -config configs\test.yaml
- Execute a specific named configuration file

Ultimately, you can use omni to launch any type of script, command or software remotely at-scale on any number of targets. I’ve often found myself on engagements for clients who lack effective SIEM or EDR tooling, meaning that when we find something like a known-bad IP address, Process Name, Binary Path, Service/Task Name or some other IOC, we have no way to effectively hunt this across the network.

omni comes with a pre-built configuration file that contains directives for common situations such as collecting running processes, TCP connections, installed Services/Tasks, etc (https://github.com/joeavanzato/omni/blob/main/config.yaml). Prior to use, you should customize a configuration that meets your collection needs depending on the situation at hand. omni also includes some example configuration files for specific use-cases at https://github.com/joeavanzato/omni/configs.

Please consider omni during your next engagement in a low-posture network suffering a cyber incident. If you experience any bugs, issues or have any questions, please open an Issue on GitHub. I am eager to hear about feature requests, ideas or problems with the software.

RetrievIR: Forensic Artifact Retrieval in PowerShell

Whenever I can, I like to use PowerShell for DFIR tasks – it’s ubiquitous presence usually means less headaches when deploying tools in client environments. To that end, exploring what is available from an open-source perspective leads most people to a few options when it comes to common DFIR tasks and automations:

Let’s talk about each of these first.

Kansa is a highly-modular framework written in PowerShell that provides Incident Response teams the capability to easily query for common artifacts as well as perform some level of analysis on the results. It can be extended by writing custom PowerShell modules to retrieve evidence as required – unfortunately, it relies on PowerShell Remoting which in turn relies on Windows Remote Management (WinRM) – a feature that, in my experience, is frequently not enabled for Desktop endpoints in corporate environments.

PowerForensics is a library written in PowerShell and C# that exposes functionality allowing for users to, in their own tools, easily gather artifacts directly through parsing of the NTFS/FAT file system artifacts such as the $MFT. This is particularly useful when analyzing dead disks or otherwise locked data – it is not intended as a ‘live’ triage tool.

CyLR is a C# tool designed to aid front-line responders in the collection of common artifacts from live systems – unfortunately, the artifact selection is hard-coded into the tool rather than available via a configuration of any type. This makes it’s usefulness relatively limited in scope.

Finally, I would be remiss if I did not discuss Velociraptor – this awesome tool is great at helping teams gain visibility into endpoints at scale and comes packed with community-contributed modules for collecting evidence. Velociraptor is ALSO capable of generating offline evidence collection packages but these must be configured ahead of time via the GUI – often this can be overkill, especially if you are not already used to the tool or don’t have it deployed in an easily accessible location.

There are some other common closed-source tools such as KAPE but these are typically not allowed to be used in paid engagements or third-party networks unless an enterprise license is obtained, making it less useful for smaller teams that cannot afford such a license.

Each of these tools is great in their own right – but I felt a need to create something to fill what I perceived as a gap – a standalone evidence collection (and parsing) tool with flexible evidence specification based on easy to read and create JSON files.

Introducing, RetrievIR [https://github.com/joeavanzato/RetrievIR] – a PowerShell script capable of parsing JSON configuration files in order to collect files, registry key/values and command outputs from local and remote hosts. At it’s core, RetrievIR is relatively simple – it will hunt for files that match specified patterns, registry keys that match provided filters and execute commands either in-line or from a specified file. Additionally, I’ve created a follow-up script called ParseIR which is designed to parse RetrievIR output using common tools such as the Eric Zimmerman set of parsers as well as some custom utilities that are still evolving.

One of the main goals in creating this was to help provide DFIR teams the capability to specify exactly what evidence they want to collect along with tagging and categorizing evidence – this means that one or more configuration files can be used in multiple ways as the operator can tell RetrievIR to only collect evidence that contains a specific tag or is part of a specified category rather than always collecting everything in the configuration file. Evidence specification does not require an individual to know how to program – everything is based in the JSON configuration including what paths to search, recursiveness, what files to filter on, what commands to execute, what registry keys to inspect and so-on.

RetrievIR is intended for use in assisting the live triage of endpoints – it collects raw evidence that is typically then processed into machine-readable information which can then be fed into centralized data stores for investigation (Elastic, Splunk, SQL, etc). RetrievIR configurations are described as JSON objects with different properties available depending on whether the target is the filesystem, the registry or a command execution. An example of each type of configuration is shown below.

{
	"files": {
		"Avast": {
			"category": "AntiVirus",
			"filter": ["*.log"],
			"recursive": false,
			"paths": [
				"%HOMEDRIVE%\\ProgramData\\Avast Software\\Avast\\Log\\*",
				"%HOMEDRIVE%\\ProgramData\\Avast Software\\Avast\\Chest\\*",
				"%HOMEDRIVE%\\Users\\*\\Avast Software\\Avast\\Log\\*"
			],
			"tags": ["sans_triage"]
		}
	},
	"commands": {
		"CommandLineConsumers": {
			"category": "WMI",
			"command": "Get-WmiObject CommandLineEventConsumer -Namespace root\\subscription  -ErrorAction SilentlyContinue | Select-Object * | Export-Csv -NoTypeInformation -Path '#FILEPATH#'",
			"output": "CommandLineEventConsumers.csv",
			"tags": ["sans_triage", "light"],
			"type": "WMI-CommandlineConsumers",
			"parser": "CSVOutputCollector"
		}
	},
	"registry": {
		"DefenderExclusions": {
			"category": "Antivirus",
			"paths": [
				"HKLM\\SOFTWARE\\Microsoft\\Windows Defender\\Exclusions"
],
			"recursive": true,
			"keys": ["*"],
			"store_empty": true,
			"tags" : ["sans_triage"]
		}
	}
}

This example configuration will tell RetrievIR to do three distinct things:

  1. Look for any file ending in ‘.log’ non-recursively at the 3 specified paths.
  2. Execute the provided command and output it to the File Name specified in ‘output’.
  3. Examine the provided registry path recursively and record any key:value stored in any path, including the current.

There are some other things going on but at it’s core, this is all that is required to use RetrievIR effectively and obtain evidence to help your team analyze systems. I’ll be writing more advanced articles covering tagging, parsing and additional properties available – but hopefully this has been enough to pique your interest and maybe help your team more rapidly triage live systems!

Link: https://github.com/joeavanzato/RetrievIR

Please check it out and provide feedback so I can make it better!

Common Commandlines

Capture the standard ‘SANS Triage’ collection of artifacts as outlined in KAPE target files.

.\RetrievIR.ps1 -tags sans_triage

After evidence collection is complete, assuming default options were used – parse the evidence!

.\ParseIR.ps1

Capture artifacts relating to browsers and anti-virus tools

.\RetrievIR.ps1 -categories antivirus,browsers

Identify available categories or tags for use with -categories and -tags.

.\RetrievIR.ps1 (-tagscan | -categoryscan)