Home Lab 1 E2

My Journey: Setting Up Suricata IDS on OPNsense for Nmap Scan Detection (and the Challenges Along the Way!)

Hello fellow tech enthusiasts! If you’re looking to enhance your network security by detecting Nmap scans using Suricata on OPNsense, you’re in the right place. I recently went through this process in my virtual lab, and while the end goal is straightforward, the path can have a few twists and turns. This blog post documents not just the “how-to” but also the real-world troubleshooting steps we took, why certain problems occurred, and how we solved them. My hope is that sharing these experiences will save you some time and help you understand the “why” behind the “what.”

Our Lab Setup:

• OPNsense Firewall (Virtual Machine in VirtualBox)
• Kali Linux (Virtual Machine in VirtualBox, acting as a client on the LAN)

The Goal: Configure Suricata on OPNsense to detect Nmap scans initiated against or from our lab network.

Part 1: The Unplanned Adventure - Achieving Basic OPNsense Stability

Before we could even think about Suricata, we hit several foundational roadblocks. Getting OPNsense to just run stably and provide basic internet access to our Kali VM was the first major chapter.

Challenge 1: Choosing the Right VirtualBox Network Configuration

Initially, my OPNsense VM had its WAN interface set to Bridged Adapter and LAN to Host-only Adapter.

• Why this can be problematic:
  • Bridged Adapter (WAN): Connects the OPNsense WAN directly to my home network. If my home network and OPNsense’s default LAN network (192.168.1.x) use the same IP range, it causes an IP conflict.
  • Host-only Adapter (LAN): VirtualBox has its own DHCP server for host-only networks. This often “races” with the OPNsense DHCP server on its LAN, leading to Kali getting an IP from VirtualBox instead of OPNsense.
• Solution & Why: We switched to a more isolated and reliable setup for labs:
  1. OPNsense WAN (Adapter 1): Set to NAT in VirtualBox.
     • Why: This gives OPNsense easy internet access through the host machine without directly exposing it to the home network or risking IP conflicts on the WAN side with the home router. OPNsense WAN gets an IP like 10.0.2.15.
  2. OPNsense LAN (Adapter 2): Set to Internal Network (e.g., named opn-lab-net).
     • Why: An “Internal Network” is a completely isolated virtual switch. Only VMs explicitly connected to it can communicate. It has no built-in DHCP server, forcing clients to rely solely on OPNsense.
  3. Kali Linux Network (Adapter 1): Also set to the same Internal Network (opn-lab-net).
     • Why: This places Kali on the same isolated network as the OPNsense LAN port.

Challenge 2: The Blank LAN IP on OPNsense

After changing network adapter types in VirtualBox, OPNsense often gets confused about which virtual NIC is WAN and which is LAN.

• Symptom: The OPNsense console showed the LAN IP as blank.
• Solution: Manually assign interfaces from the OPNsense console.
  1. Boot OPNsense.
  2. Select Option 1: Assign interfaces.
  3. Answer n to VLANs.
  4. Assign the first network card (e.g., em0 or vtnet0) to WAN.
  5. Assign the second network card (e.g., em1 or vtnet1) to LAN.
  6. Confirm.
• Why: This tells OPNsense explicitly which virtual hardware maps to its logical WAN and LAN roles. We also learned that after assigning interfaces, a clean reboot from the OPNsense menu (Option 6) is crucial to make these assignments stick permanently.

Challenge 3: IP Conflict on the LAN (DHCP Hell!)

My OPNsense WAN was getting an IP from my home router that was on the same subnet (e.g., 192.168.1.x) as OPNsense’s default LAN IP (192.168.1.1). This caused the OPNsense DHCP server (dhcpd) to log errors like Multiple interfaces match the same shared network: em0 em1.

• Symptom: No internet on Kali, DHCP errors in OPNsense logs.
• Solution: Change the OPNsense LAN subnet.
  1. From the OPNsense console, select Option 2: Set interface IP address.
  2. Select the LAN interface.
  3. Answer n to “Configure IPv4 address LAN interface via DHCP?”.
  4. Set a new static IPv4 address: 192.168.10.1
  5. Subnet bit count: 24
  6. Enable DHCP server: y
  7. DHCP range: 192.168.10.100 to 192.168.10.200.
• Why: This moves the internal lab network to a completely different range, eliminating the conflict with the WAN side.
• Follow-up: Kali then had to be forced to get a new IP from this new range using sudo nmcli connection down 'Wired connection 1' && sudo nmcli connection up 'Wired connection 1' or by rebooting Kali. We even encountered “sticky” DHCP leases on Kali where it kept 192.168.56.102 (from a previous Host-Only attempt), requiring a Kali reboot to clear.

Challenge 4: OPNsense DNS Resolution Failures (SERVFAIL, timeout)

Even with correct IPs, Kali couldn’t resolve domain names. dig google.com would fail.

• Symptom: No internet Browse, dig commands to OPNsense DNS (192.168.10.1) resulted in SERVFAIL or timed out. Ping to 8.8.8.8 from Kali might have failed with “Destination Host Unreachable” if OPNsense itself had no route.
• Multiple Root Causes & Solutions:
  1. OPNsense had no upstream DNS servers: The System -> Settings -> General page had an empty DNS server list.
     • Fix: Manually added 8.8.8.8 and 1.1.1.1.
     • Why: OPNsense needs to know which DNS servers it should use to look up external domains. The WAN DHCP wasn’t providing them.
  2. Unbound DNS not listening on LAN or not forwarding:
     • Fix: In Services -> Unbound DNS -> General, ensured Network Interfaces was set to All (recommended).
     • In Services -> Unbound DNS -> Query Forwarding, checked Use System Nameservers (this is the newer equivalent of “Enable Forwarding Mode”).
     • Why: Unbound needs to listen for queries from the LAN and then forward them to the upstream DNS servers we set in System General.
  3. Time Zone & DNSSEC: An incorrect timezone (Etc/UTC instead of local Asia/Kolkata) can cause DNSSEC validation to fail, leading to SERVFAIL.
     • Fix: Set the correct timezone in System -> Settings -> General. Temporarily disabling DNSSEC in Services -> Unbound DNS -> General helped confirm this was part of the issue.
     • Why: DNSSEC relies on accurate time for signature validation.
  4. Unbound Internal SSL Certificate Error: Logs showed sslv3 alert bad certificate for Unbound remote control.
     • Fix: In Services -> Unbound DNS -> Miscellaneous (or Advanced in some versions), clicked Regenerate TLS Certificate. Or, from the shell: rm /var/unbound/unbound_*.pem && service unbound restart.
     • Why: The internal certificates Unbound uses for its control functions were corrupted.

Challenge 5: Hardware Offloading Issues (em1 drop mbuf...)

The OPNsense console was flooded with em1 drop mbuf that needs checksum of fload errors, causing instability.

• Symptom: System hangs, console errors.
• Solution: Disable hardware offloading.
  1. The error occurred immediately on boot, making GUI access hard. We first tried using boot-time tunables from the OPNsense loader prompt (Option 3 at boot) to temporarily disable offloading:

     set hw.em.tso_enable=0
     set hw.em.lro_enable=0
     set net.inet.tcp.tso=0
     set net.inet.tcp.lro=0
     set net.inet.tcp.recv_checksum_offload=0
     set net.inet.tcp.send_checksum_offload=0
     set net.inet.udp.recv_checksum_offload=0
     set net.inet.udp.send_checksum_offload=0
     boot
             
     

  2. Once stable enough to access the GUI: In Interfaces -> Settings, CHECKED the boxes for:
     • Disable hardware checksum offload
     • Disable hardware TCP segmentation offload
     • Disable hardware large receive offload
  3. Clicked Save and rebooted.
• Why: Emulated network cards in VMs (like the default Intel em driver) can have issues with hardware offloading features. Disabling them makes the CPU do the work but improves stability. The checkbox labels “Disable…” mean checking them turns off the feature.

Challenge 6: Suricata Initialization Failures (Device not configured for em0)

When trying to enable Suricata, it would fail to start, logging errors about em0 (WAN) not being configured for Netmap.

• Symptom: Suricata wouldn’t start, iflib_netmap_config messages on console (sometimes), errors in Suricata log.
• Solution (Highly Recommended for Suricata): Switch OPNsense VM network adapters in VirtualBox to Paravirtualized Network (virtio-net) for both WAN and LAN.
  1. Power off OPNsense.
  2. Change adapter types in VirtualBox VM settings.
  3. Power on OPNsense.
  4. Re-assign interfaces from the console (they will now be vtnet0, vtnet1, etc.).
• Why: virtio-net drivers are designed for virtualization, are more efficient, and handle advanced packet processing (like Netmap used by Suricata) much more reliably than emulated hardware drivers like em. This often resolves these specific initialization failures. (The user was also advised they could change Kali to virtio-net too, for consistency and potential minor benefits, though it’s less critical for a client.)

Phew! That was a lot of foundational work. With a stable OPNsense and Kali setup, we can finally move to Suricata.

Part 2: Setting Up Suricata for Nmap Scan Detection

Now that our OPNsense firewall is stable and Kali has reliable internet through it, we can configure Suricata.

Goal: Detect Nmap scans. We learned that relying on well-maintained, community-driven rulesets is usually better and easier than writing complex custom rules from scratch for common threats like Nmap scans.

Step 1: Install/Enable Suricata

1. In OPNsense GUI, navigate to System -> Firmware -> Plugins. Ensure os-suricata is installed.
2. Go to Services -> Intrusion Detection -> Administration.
3. On the Settings tab:
   • Check Enabled for the Intrusion Detection System.
   • Interfaces: Select the interface(s) to monitor.
     • To detect scans from the internet against your WAN, select your WAN interface (e.g., vtnet0).
     • To detect scans originating from your LAN (e.g., from Kali outwards), select your LAN interface (e.g., vtnet1). You can select both.
   • Home networks: This is crucial. It defines $HOME_NET. Ensure it includes:
     • Your OPNsense LAN: e.g., 192.168.1.0/24 (if you kept the default after fixing conflicts).
     • If your WAN is bridged and gets a private IP from your home router, include that network too (e.g., 192.168.0.0/24).
   • Click Save.

Step 2: Download and Select Rulesets

1. Go to the Download tab (still under Services -> Intrusion Detection -> Administration).
2. Enable desired rulesets. The ET open/... categories are excellent. OPNsense breaks “ET Open” into many sub-categories.
   • For Nmap/Scan detection, enable categories like:
     • ET open/scan or ET open/emerging-scan
     • ET open/emerging-recon
   • Also consider general threat categories like ET open/emerging-exploit, ET open/emerging-malware.
3. Click Download & Update Rules. Wait for it to complete.

Step 3: Apply Rule Policies

1. Go to the Rules tab.
2. For each interface you are monitoring (e.g., WAN, LAN), select it from the dropdown.
3. You’ll see the categories you enabled. Ensure their Action is set to at least Alert.
4. Click Apply at the bottom of the rules list for that interface.

Step 4: Globally Apply Suricata Configuration

1. Go back to the Settings tab (under Services -> Intrusion Detection -> Administration).
2. Click the main Apply button at the top. This restarts Suricata with all your selected rules and policies.

Step 5: Testing Nmap Scan Detection

1. To Test Scans Against WAN:
   • If your OPNsense WAN is Bridged (e.g., IP 192.168.0.108): From another machine on your physical home network (or your host PC), scan this OPNsense WAN IP: sudo nmap -sS 192.168.0.108.
   • If your OPNsense WAN is NAT (IP 10.0.2.15): Testing external scans is trickier. You could try scanning 10.0.2.15 from your host PC.
2. To Test Scans From LAN Outwards:
   • Ensure Suricata is monitoring your LAN interface.
   • From your Kali VM, scan an external target: sudo nmap -sS scanme.nmap.org or sudo nmap -sS 8.8.8.8.
3. Check Alerts: In OPNsense, go to Services -> Intrusion Detection -> Alerts. You should see alerts related to Nmap activity if everything is configured correctly and your scan matches a rule.

(We encountered an issue where an initial custom rule had syntax errors like unknown rule keyword 'fin', which highlighted the importance of correct Suricata rule syntax. The fix was to simplify the flag checks in the custom rule, e.g., just flags:S; and rely on thresholding.)

Key Takeaways from Our Journey

• Environment Matters: Virtual networking (especially with multiple adapters and different modes like NAT, Bridged, Host-Only, Internal) can be complex and lead to IP conflicts or DHCP races. Using NAT + Internal Network proved most stable for this isolated lab.
• Client-Side Issues: Sometimes the firewall is fine, but the client (Kali) holds onto old DHCP leases or has incorrect DNS. Forcing a lease renewal (nmcli connection down/up or reboot) is key.
• DNS is Fundamental: If the firewall itself can’t resolve DNS, nothing downstream will work. Manually setting upstream DNS servers in OPNsense (System -> Settings -> General) and ensuring Unbound forwards to them (Services -> Unbound DNS -> Query Forwarding -> Use System Nameservers) was critical.
• Hardware Offloading in VMs: Often a source of instability. Disabling CRC, TSO, and LRO in Interfaces -> Settings is a common fix for mbuf errors.
• virtio-net for Performance/Stability: For demanding services like Suricata in VMs, virtio-net (paravirtualized) network adapters are generally more stable and performant than emulated ones like Intel em drivers. Remember to re-assign interfaces in OPNsense after changing adapter types in VirtualBox.
• Read the Logs: OPNsense logs (System -> Log Files and service-specific logs like Suricata’s) provide invaluable clues when things go wrong. We saw dhcpd errors, Unbound bad certificate errors, and Suricata parsing errors, each pointing to a specific solution.
• Systematic Troubleshooting: When faced with complex issues, changing one thing at a time and verifying is crucial. Don’t assume!
• Default Rulesets are Powerful: For common tasks like Nmap detection, well-maintained public rulesets like ET Open are often all you need and are more robust than trying to write everything from scratch.

Conclusion

Setting up a sophisticated firewall like OPNsense with an IDS like Suricata in a virtual lab is a fantastic learning experience. While we faced numerous unexpected challenges, each one provided a deeper understanding of networking, virtualization, and firewall administration. The key is persistence and a systematic approach to troubleshooting.

I hope this detailed account of our journey helps you in your own lab setups! Happy firewalling!

To get in touch with me or for general discussion please visit ZeroDayMindset Discussion