Abstract
This study investigates machine learning methods for detecting malicious network traffic data in consumer IoT devices using labeled data traffic statistics. Using 20 datasets of diverse attacks, including botnets, Trojans, Hide-and-Seek, and Linux attacks, the study evaluates various ensemble methods for classifying malicious and benign traffic. The ANOVA algorithm is applied for feature ranking, and features are selected for classification. In addition, a cost matrix that penalizes misclassifying malicious traffic more heavily is applied in its classification. Results show that reducing features via feature ranking not only improves classification accuracy but also significantly reduces computation load and overfitting. Reduction of features from 22 to 5 achieved validation accuracy of 98-99% and improved test accuracy by up to 40%. With compact feature sets, bagged trees, kernel naïve Bayes, and ensemble tree methods, the methods delivered 80 - 97.4% and achieved the highest accuracy of 95.7 - 97.4% test accuracy, while mitigating overfitting. The proposed method achieved an accuracy of 95.7 - 97.4%. Despite limitations in data volume and the simplification to binary classification, the findings show that this research provides a foundation for the development of a lightweight, feature-efficient IoT malicious traffic detection system, motivating further research into multi-class, device-independent, real-time detection.