Machine Learning Anomaly Detection Reduces False Positives More Effectively Than Rule-Based Systems
Cybersecurity systems are constantly scanning for threats, but they often raise alarms for activities that turn out to be harmless. These incorrect alerts, known as false positives, can overwhelm security teams and lead to wasted effort. This paper investigates how Machine Learning-based anomaly detection can help reduce false positives compared to traditional rule-based threat detection methods. Every alert typically requires investigation by security analysts. When most alerts are false positives, analysts spend countless hours checking on non-issues instead of hunting real threats. Industry studies estimate that organizations collectively waste millions of dollars and tens of thousands of work-hours per year chasing false alerts. According to a 2015 survey by the Ponemon Institute of over 600 U.S. IT and security practitioners, organizations spent on average nearly 21,000 hours per year and about $1.3 million in labor costs investigating security alerts that ultimately turned out to be false alarms. This is time and money that could have been directed toward strengthening defenses or responding to actual incidents. A flood of false alarms can overwhelm security teams, a phenomenon often called “alert fatigue.” When analysts are inundated with thousands of alerts each day, many of them false, it becomes difficult to distinguish the real threats from the noise. This study compared a rule-based detector with a machine learning detector on a Kaggle URL dataset with more than 800,000 labeled links. Both methods use only URL-based features, including length, special characters, and common phishing keywords. The rule-based system reaches about 55 percent accuracy and misses most phishing URLs. The Random Forest model reaches about 89 percent accuracy and can rank truly malicious cases above normal ones in about 95 percent of comparisons, while greatly reducing both false positives and false negatives. This shows that machine learning can give a much stronger defense than traditional rules.
-
Title -> Thesis
-
Casual -> Scientific/formal
-
Change the source
-
explain ROC AUC
-
Captures attention