Skip to content
· 6 min read INFO @Sdmrf

Building a Detection Lab in 2025: My Setup for Threat Research

A practical walkthrough of setting up a home detection lab for security research - from hypervisor choices to logging pipelines.

On this page

I get asked about my lab setup constantly. So here’s the detailed breakdown of what I run, why I chose each component, and how you can build something similar without breaking the bank.

Fair warning: this is opinionated. Other setups work fine. This is what works for me.

Goals

Before building anything, know what you want to accomplish:

  • My goals: Malware analysis, detection rule development, testing attack techniques, learning new tools
  • Not goals: Production security monitoring, long-term data retention, high availability

This shapes every decision. A research lab has different requirements than a production SOC.

Hardware

You don’t need much to start.

My current setup:

  • Primary: Custom desktop, Ryzen 9 5900X, 128GB RAM, 2TB NVMe
  • Secondary: Old Dell Optiplex, i7, 64GB RAM, 1TB SSD
  • Network: Managed switch with VLANs, pfSense firewall on an old mini PC

Minimum viable lab:

  • Any machine with 32GB RAM and a decent CPU
  • Can run everything virtualized on one box

More RAM = more VMs. That’s the main constraint. Start with what you have.

Hypervisor: Proxmox

I switched from VMware to Proxmox a year ago and haven’t looked back.

Why Proxmox:

  • Free and open source
  • Excellent VM and container support
  • Web-based management
  • Good snapshot capabilities
  • Active community

Installation:

# Download ISO from proxmox.com
# Boot from USB, follow installer
# Post-install, access web UI at https://<ip>:8006

First things I do on a fresh Proxmox install:

# Remove enterprise repo nag
sed -i "s/^deb/#deb/" /etc/apt/sources.list.d/pve-enterprise.list

# Add no-subscription repo
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list

apt update && apt upgrade -y

Network Layout

Isolation is important when you’re running malware.

Internet


┌─────────┐
│ pfSense │
└─────────┘

    ├── VLAN 10 (Management) - Proxmox, security tools

    ├── VLAN 20 (Analysis) - Malware analysis VMs
    │   └── No internet by default, enable as needed

    ├── VLAN 30 (Detection) - Target VMs, logging

    └── VLAN 40 (Services) - SIEM, databases

Key principle: Analysis VMs (where malware runs) are isolated by default. I enable internet access temporarily when needed, with full packet capture running.

Core VMs

1. Security Onion (SIEM/IDS)

Security Onion bundles everything you need for detection:

  • Elasticsearch/Opensearch for storage
  • Kibana for visualization
  • Zeek for network analysis
  • Suricata for IDS
  • Fleet for endpoint collection
# Import Security Onion ISO to Proxmox
# Create VM: 8 cores, 32GB RAM, 500GB storage
# Follow setup wizard - choose "Standalone" for lab use

I configure it to capture from a mirror port on the switch, seeing all VLAN 30 traffic.

2. Windows 10/11 Target

For testing Windows attacks and running malware samples.

# Resources: 4 cores, 8GB RAM, 60GB disk
# Disable Defender (for controlled testing only!)
# Install Sysmon with a good config
# Configure Windows Event Forwarding to Security Onion

Sysmon config I use: SwiftOnSecurity’s config as a base, heavily modified for what I’m researching.

# Install Sysmon
Invoke-WebRequest -Uri "https://download.sysinternals.com/files/Sysmon.zip" -OutFile sysmon.zip
Expand-Archive sysmon.zip
.\Sysmon64.exe -accepteula -i sysmonconfig.xml

3. Windows Server (AD DC)

For testing Active Directory attacks:

# Windows Server 2022
# 4 cores, 8GB RAM
# Promote to Domain Controller
# Create some users, groups, GPOs

Having a DC makes the lab useful for testing:

  • Kerberoasting
  • DCSync
  • Group policy attacks
  • Lateral movement

4. Linux Target

Ubuntu Server for testing Linux-focused attacks:

# Ubuntu 22.04 LTS
# 2 cores, 4GB RAM
# Install auditd with good rules
# Configure rsyslog to forward to SIEM

5. Kali/Attack Box

For simulating attacker actions:

# Kali Linux
# 2 cores, 4GB RAM
# On a separate VLAN from targets
# Route through the network to generate traffic

Logging Pipeline

Getting logs from endpoints to the SIEM is crucial.

Windows endpoints:

  • Sysmon → Windows Event Log
  • Windows Event Log → Winlogbeat → Security Onion

Linux endpoints:

  • auditd → syslog
  • syslog → rsyslog → Security Onion (syslog input)

Network:

  • Switch mirror port → Security Onion sensor interface
  • Zeek + Suricata process the traffic

Making It Useful

A lab collecting dust is worthless. Here’s how I actually use mine:

Detection Rule Development

  1. Pick a technique from MITRE ATT&CK
  2. Execute it on a target VM
  3. Check what logs were generated
  4. Write a detection rule
  5. Test against normal activity for false positives
  6. Document and save

Malware Analysis

  1. Get a sample (MalwareBazaar, VirusTotal, etc.)
  2. Snapshot the analysis VM
  3. Run sample with monitoring enabled
  4. Capture network traffic, process activity, file system changes
  5. Revert snapshot
  6. Analyze captured data

Tool Evaluation

When a new security tool comes out:

  1. Deploy it in the lab
  2. Run known-bad activity
  3. See what it catches (and misses)
  4. Compare to existing tools
  5. Decide if it’s worth deploying in production

Cost

People assume this is expensive. It’s not.

My rough costs:

  • Primary server: $1,500 (built over time, used parts)
  • Secondary server: $200 (used Optiplex)
  • Network gear: $150 (used managed switch)
  • pfSense box: $100 (old mini PC)
  • Software: $0 (everything is free/open source)

Total: ~$2,000 over several years

You can start with a single machine and $0 in software. Upgrade as you go.

Common Mistakes

Things I did wrong that you can avoid:

  1. Over-engineering initially - Start simple, add complexity as needed
  2. Not taking snapshots - Snapshot before every malware run
  3. Poor network isolation - Assume malware will try to spread
  4. Not enough RAM - Get as much as you can afford
  5. Skipping documentation - Document your setup; future you will thank you

What I’d Do Differently

If starting from scratch today:

  • More RAM - 128GB isn’t enough for what I want to run
  • Dedicated NAS - Local storage fills up fast with PCAPs
  • Better switch - One with actual flow export capabilities
  • Physical separation - Separate box for malware analysis

Next Steps

If you’re inspired to build your own:

  1. Start with what you have
  2. Pick one use case (detection rules? malware analysis?)
  3. Build the minimum needed for that use case
  4. Expand as you learn what you need

Don’t try to build the perfect lab. Build a working lab, then improve it.

Resources


The best detection lab is the one you actually use. Start small, learn constantly, and iterate.

Related Articles