Date of Award

Spring 2026

Rights

Access is available to all users

Date Available to Non-EWU Users

2026-06-15

Document Type

Thesis

Degree Name

Master of Science (MS) in Cyber Defense

Department

Computer Science

First Advisor

Sanmeet Kaur

Second Advisor

Stuart Steiner

Third Advisor

Lynnae Daniels

Abstract

Vulnerability identification during penetration testing relies on rigid string-matching to map network scan data to Common Platform Enumeration (CPE) identifiers and downstream Common Vulnerabilities and Exposures (CVEs). The approach frequently fails on physical Internet of Things (IoT) devices, which produce non-standard, irregular service banners that resist deterministic parsing. Large Language Models can reason through these fuzzy associations, but cloud-hosted models introduce cost, latency, and operational security concerns when processing reconnaissance data from live networks. This thesis asks whether locally-hosted open-weight Large Language Models (LLMs) can perform this task well enough to be useful, and how performance varies with model scale, family, and prompt design. I evaluate 36 LLMs across three deployment tiers (frontier hosted APIs, Ollama Cloud, and local vLLM), plus Nmap’s native CPE extraction as the no-LLM baseline, on a curated corpus of Nmap covering 16 analyzed IoT device entries drawn from an 18-entry testbed: 14 physical entries and 4 derived entries, with one physical and one derived entry excluded for unusable scan or run data. Three findings shape the result. Model family is the dominant source of variance, with a wider spread than prompt design or scan-suite choice produces. The best truly-local model approaches frontier accuracy on a multi-GPU workstation but does not on single-GPU consumer hardware. The most operationally dangerous failure is not a malformed string but a confident, well-formed CPE that retrieves the wrong vulnerability list.

Download

Included in

Cybersecurity Commons

Share

COinS