Published research · IJSREM, NCFT 2025
Cyber Threat Detection and Profiling Using AI
Built to analyze open-source cybersecurity chatter on Twitter (OSINT) and turn it into structured, actionable threat intelligence. The pipeline pulls in raw social data, cleans and processes it with NLP, then classifies it against known attacker behavior.
Integrated the MITRE ATT&CK framework so detected threats map directly to recognized tactics and techniques, generating real-time threat profiles instead of just raw alerts.
Benchmarked four classification models: Random Forest, SVM, Logistic Regression, and Naïve Bayes - to find the best-performing approach, landing on an F1-score of 77%.
Overview
As the gap between vulnerability disclosure and real-world exploitation keeps shrinking, security teams need threat intelligence that arrives in real time rather than after the fact. ThreatLens addresses this by treating Twitter as a live OSINT feed to continuously scann posts for threat-related language, then running that language through an NLP and machine-learning pipeline to identify, name, and profile emerging threats automatically.
Rather than just flagging that something suspicious happened, the system profiles the threat's likely intent, origin, and potential impact, then maps it onto the MITRE ATT&CK framework so the output is a structured, analyst-ready threat profile instead of raw noise.
Key Features
Real-Time OSINT Monitoring
- Continuously ingests posts from key Twitter sources as a live threat-intelligence feed.
- Filters signal from noise across high-volume, unstructured social data.
NLP Preprocessing (spaCy)
- Tokenization, stemming, lemmatization, and stop-word removal to clean raw text.
- Feature extraction tuned to isolate threat-relevant language from everyday chatter.
MITRE ATT&CK Mapping
- Identified threats are mapped to known tactics and techniques in the ATT&CK matrix.
- Produces structured, actionable threat profiles instead of unlabeled alerts.
ML-Based Threat Profiling
- A layered classification model assesses intent and likely impact per threat.
- Generates risk-based alerts to shorten analyst response time.
Machine Learning Models Benchmarked
Six classifiers were evaluated for the profiling stage to find the strongest performer on this data: Decision Tree, Gradient Boosting, Logistic Regression, Naïve Bayes, Random Forest, and SVM. The profiling component reached a 77% F1-score in characterizing emerging threats.
Pipeline
Twitter (OSINT) ──► NLP Preprocessing (spaCy) ──► ML Classification ──► MITRE ATT&CK Mapping ──► Risk-Based Alerts
System Roles
- Service Provider : trains/tests models, reviews accuracy results, and manages registered users.
- Admin : reviews and authorizes registered user accounts.
- Remote User : registers, logs in, and requests threat/identity predictions against the trained models.
Published Research
Co-authored and presented at NCFT 2025, published in the International Journal of Scientific Research in Engineering and Management (IJSREM), Special Edition Volume 09 (ISSN: 2582-3930).