AI-DRIVEN SOCIAL ENGINEERING AWARENESS TRAINING MODEL

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 12 Issue: 07 | July 2025

p-ISSN: 2395-0072

www.irjet.net

AI-DRIVEN SOCIAL ENGINEERING AWARENESS TRAINING MODEL Lavanya K M1, Dr. H P Mohan Kumar2 1Department of MCA, P.E.S College of Engineering, Mandya, Karnataka, India

2Department of CS&E, P.E.S College of Engineering, Mandya, Karnataka, India

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract - In the rapidly evolving landscape of digital

audio files using MFCC-based feature extraction and deep learning models such as LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Networks). Text classification is based on converting message content into numerical vectors using TF-IDF and then feeding it to classifiers like Logistic Regression and SVM. For voice detection, MFCC features help in capturing speech characteristics which are then used to train the neural network models to distinguish between real and fake (AIgenerated or mimicked) audio.

communication, threats like social engineering have become a serious challenge. Social engineering attacks, particularly those conducted through telephonic conversations and text messages, aim to manipulate individuals into revealing sensitive personal, financial, or security-related information.

These attacks bypass traditional technical security measures by exploiting human psychology. The rise of fake calls, scam messages, and voice spoofing has necessitated the development of intelligent systems capable of detecting such malicious attempts in real-time.

2. LITERATURE SURVEY

This project presents a comprehensive system for the detection and classification of telephonic social engineering attacks, combining both textual message data and telephonic audio recordings. The system uses machine learning and deep learning models to classify text messages as either spam (social engineering attempt) or non-spam, and audio files as either fake (synthetic/manipulated) or real (genuine voice).

[1] In an early study, Breda et al. (2017) emphasized the psychological manipulation at the core of social engineering, where attackers exploit human trust, curiosity, and urgency to gain access to sensitive information. Although the study thoroughly discussed real-world incidents and human vulnerabilities, it lacked the development of technical or automated solutions to mitigate these attacks.

Key Words: Machine learning, deep learning, genetic

[2] Expanding on this foundation, Salahudin and Kabocha (2019) conducted a comprehensive survey categorizing social engineering methods, such as phishing, baiting, and pretexting. Their work was pivotal in identifying behavioural triggers and proposing countermeasures like access control and employee training. However, their approach remained reactive and did not suggest intelligent, real-time AI systems for proactive detection.

algorithm, LSTM, TF-IDF, KNN, SVM

1.INTRODUCTION Social engineering is a psychological manipulation technique wherein attackers deceive individuals into revealing confidential information. Unlike traditional hacking, which requires technical expertise, social engineering relies on exploiting human emotions such as fear, urgency, curiosity, or trust. These attacks often come in the form of fraudulent phone calls, messages, or emails. For instance, a caller may impersonate a bank representative and ask for OTPs, PIN numbers, or account details. In other cases, automated fake voices powered by AI technology may attempt to sound convincing enough to bypass voice authentication systems.

[3] With the growing role of social media in daily communication, Naz et al. (2024) explored the risks of social engineering through platforms like Facebook and Twitter.

The study showcased how attackers use publicly available information to engineer convincing scams and impersonation. Despite its depth in social media-focused threats, the work lacked any integration of speech-based attack vectors or voice-based detection systems.

Given this context, there is a dire need to develop intelligent systems that can help detect and prevent these manipulative attacks. This project, titled "AI-DRIVEN SOCIAL ENGINEERING AWARENESS TRAINING MODEL", is a step towards addressing this critical issue. It aims to build a detection system that can classify inputs—either in text or audio format—as either genuine or suspicious.

[4] In a more implementation-oriented study, Deepak Singh Malik (2023) analysed phone-based scams and proposed machine learning-based anomaly detection as a potential solution. His recommendation to combine user profiling and voice analysis laid the groundwork for multi-modal security systems. However, the study lacked concrete implementation or experimental validation of the proposed models.

The solution is two-fold: one part deals with text message analysis using natural language processing (NLP) and machine learning (ML), while the second part processes

© 2025, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 836

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 12 Issue: 07 | July 2025

p-ISSN: 2395-0072

www.irjet.net

[5] A holistic view of the landscape was offered by Tejal Rathod et al. (2025), who presented a layered AI-based approach combining voice authentication, NLP, and sentiment analysis for combating deepfake-enabled social engineering. They proposed a real-time fraud scoring system but acknowledged the lack of scalable, low-cost solutions for widespread public deployment—highlighting a key opportunity for this project.

organized into several key phases: dataset collection and preprocessing, text message classification, voice call classification, and integration through a web-based interface.

[6] In the audio domain, Ameer Hamza (2022) focused on detecting deepfake speech using MFCC feature extraction with machine learning classifiers like SVM and Random Forest. His models achieved over 95% accuracy in classifying synthetic audio. However, his models were trained on narrowly scoped datasets and lacked multilingual or realtime capabilities.

[7] Addressing linguistic diversity, Omair Ahmad (2025) investigated deepfake detection in Urdu using LSTM and CNN networks. His study achieved over 98% accuracy and emphasized the need for regional language datasets. This work highlights the importance of multilingual training, particularly for applications in countries like India.

[8] Further contributions by Hamza et al. (2022) reinforced the value of MFCC and delta features in deepfake detection, stressing the role of robust training under varied acoustic conditions. However, their models still struggled with generalization across datasets and languages—underscoring the need for scalable solutions.

Fig 3.1 System Architecture

3.1 Dataset Collection and Preprocessing

[9] Almutairi and Elgibreen (2022) reviewed a range of audio deepfake detection techniques, including FFT, ResNet, and GAN-based methods. Their survey discussed current challenges like compressed audio detection, adversarial attacks, and lack of explainable AI in this domain. While comprehensive, the study did not propose or test any unified platform combining both text and audio detection.

To begin with, two distinct datasets are prepared—one for textual data and the other for voice recordings. The text message dataset is composed of labelled samples categorized as either legitimate messages or scam/spam messages, including phishing attempts, impersonation, fraudulent banking requests, and OTP (One-Time Password) scams.

These messages are collected from publicly available repositories, open-source SMS datasets, and manually curated samples. To prepare the text for model training, several preprocessing steps are applied to ensure data uniformity and enhance learning. Each message is converted to lowercase to eliminate case sensitivity. Punctuation marks, symbols, and irrelevant characters are removed. Stop words—commonly occurring words that carry minimal semantic load—are filtered out. Next, tokenization is applied, which splits the message into individual words or tokens.

Finally, stemming or lemmatization is employed to reduce words to their base forms, thereby consolidating similar terms and reducing dimensionality. Once cleaned, the textual data is vectorized using TF-IDF (Term Frequency-Inverse Document Frequency). This technique transforms text into numerical representations that reflect the importance of terms relative to a document and the overall corpus, capturing the contextual weight of each word. This sparse matrix of TF-IDF features becomes the input to the machine learning classifiers.

[10] Ezer Osei Yeboah-Boateng & Priscilla Mateko Amanor (2024),Phishing, Smishing & Vishing: An Assessment of Threats against Mobile Devices Conducted exploratory interviews with around 20 mobile users to assess their awareness and behavior toward phishing, vishing (voice phishing), and SMS-based phishing (Smishing), Highlighted a taxonomy of persuasive (“alluring” or “decoying”) words used in mobile phishing attacks to inform user awareness.

3. METHODOLOGY The proposed system aims to detect and classify telephonic social engineering attacks by analysing both textual messages and voice-based calls using a dual-modality machine learning and deep learning framework. This hybrid approach enhances the robustness of the detection mechanism by combining natural language processing techniques with audio signal processing. The methodology is

© 2025, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 837

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 12 Issue: 07 | July 2025

p-ISSN: 2395-0072

www.irjet.net

Simultaneously, a voice audio dataset is curated containing two categories: real voice recordings, which represent genuine human speech from legitimate calls, and fake recordings, which include AI-generated (deepfake) voices, mimicry attempts, or pre-recorded scam messages. The dataset is compiled from open-source speech repositories, telephony datasets, and synthetic voice samples created using text-to-speech (TTS) tools. Each audio file undergoes a series of preprocessing steps to normalize and extract meaningful features. The files are first converted to mono channel and resampled at 16 kHz to standardize frequency.

Silence trimming is performed to eliminate pauses, and noise reduction techniques are applied to remove ambient disturbances. The core feature extraction process involves computing Mel-Frequency Cepstral Coefficients (MFCC), which represent the short-term power spectrum of sound and are widely used in speech recognition. Typically, 13 to 40 MFCCs are extracted per frame. These coefficients effectively capture the timbral and tonal qualities of human speech. To ensure uniform input size, feature vectors are padded or truncated to a fixed length. In some cases, MFCCs are visualized as spectrogram images for CNN-based models.

architecture includes 2–3 convolutional layers activated with ReLU, followed by max pooling layers to down-sample feature maps and extract spatial patterns. These are then flattened and passed through fully connected (dense) layers, ending with a softmax activation function for binary classification. This model is optimized using Adam optimizer and categorical cross entropy loss function.

In the LSTM-based model, MFCC features are treated as temporal sequences, which allow the model to capture dependencies over time. The architecture includes one or more LSTM layers capable of remembering long-term dependencies, followed by dense layers with dropout regularization to prevent overfitting. The final layer uses SoftMax for classification. Training is conducted with early stopping, learning rate scheduling, and batch normalization to enhance stability and convergence. Both models are trained using the same 80/20 train-test split, and model checkpoints are saved in .h5 format for Kera’s/TensorFlow deployment. Evaluation is conducted using metrics such as accuracy, ROC-AUC, F1-score, and confusion matrix.

3.4 System Integration and Deployment

3.2 Text Classification Process

To provide seamless interaction for end-users, both modules are integrated into a unified web application using the Flask framework. The web interface offers two primary functionalities: 1) text input box for message classification, and 2) file upload form for voice classification. Upon submission, the Flask backend routes the request to the appropriate model. For text messages, the input is preprocessed, vectorized with TF-IDF, and fed into the serialized ML model to return a prediction label and confidence score. For audio files, MFCCs are extracted onthe-fly, and the corresponding deep learning model classifies the input as either Real or Fake, also displaying a confidence level. Users can listen to the uploaded audio through an integrated audio playback feature.

The text classification module leverages traditional machine learning algorithms to categorize input messages as either Spam (social engineering attempt) or Non-Spam (benign message). The pre-processed TF-IDF feature matrix serves as input to a suite of classifiers, each evaluated for its predictive accuracy and generalizability. The models trained include Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), and K-Nearest Neighbours (KNN). Logistic Regression is selected for its simplicity, interpretability, and effectiveness in binary classification.

SVM is chosen for its ability to handle high-dimensional data and non-linear decision boundaries using kernel tricks.

Random Forest is employed due to its ensemble-based robustness and resistance to overfitting, especially in noisy datasets. KNN is used to assess performance based on similarity measures, albeit with higher computational cost during inference. The dataset is split into 80% training and 20% testing, and k-fold cross-validation is used to validate results. Models are evaluated using standard metrics: accuracy, precision, recall, F1-score, and confusion matrix.

The best-performing model—typically SVM or RF—is saved using joblib or pickle for later deployment.

The frontend is developed using HTML, CSS, and Bootstrap, ensuring responsiveness and ease of use across devices. The backend handles real-time inference, completing predictions in under three seconds for both modalities. All models are stored in appropriate formats: .pkl or .joblib for text classifiers, and .h5 for deep learning audio models. The application architecture is modular, supporting future expansion for multilingual detection, real-time streaming analysis, and integration with external APIs for fraud alerts.

3.3 Voice Classification Process

4. RESULTS AND DISCUSSION

The voice analysis module employs deep learning architectures, specifically Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks, to distinguish between real and fake audio samples. Two parallel model architectures are designed to explore performance trade-offs. In the CNN-based model, the MFCC features are treated as 2D spectrogram images. The

© 2025, IRJET

|

Impact Factor value: 8.315

The proposed system was evaluated on two primary tasks: classification of social engineering text messages and detection of fake versus real audio calls. The goal was to assess the performance of machine learning and deep learning models on relevant datasets using standard evaluation metrics such as accuracy, precision, recall, F1-

|

ISO 9001:2008 Certified Journal

|

Page 838

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 12 Issue: 07 | July 2025

p-ISSN: 2395-0072

www.irjet.net

score, and confusion matrix. The results from both classification modules are presented and interpreted below.

Discussion – Voice Models:

4.1Text Classification Results A dataset of labelled SMS messages—including phishing, impersonation, and genuine messages—was pre-processed and transformed using TF-IDF. Four machine learning models were trained: Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), and K-Nearest neighbours (KNN).

Model

Accuracy

ROCAUC

Precision

Recall

F1Score

CNN

94.5%

0.96

94.3%

93.8%

94.0%

LSTM

96.2%

0.97

95.8%

95.4%

95.6%

Table 4.1 Text Classification Result



The LSTM model demonstrated superior performance, particularly in capturing temporal speech dynamics critical for detecting manipulated audio.



CNN performed slightly lower but remained a strong contender due to its strength in spatial pattern recognition over MFCC spectrograms.



ROC-AUC scores of 0.96 and 0.97 for CNN and LSTM respectively indicate excellent class separability.



Misclassifications in CNN were mostly attributed to short-duration or heavily compressed audio clips, where spectral resolution is low.

5. SNAPSHOTS

Discussion – Text Models: 

SVM outperformed other models, particularly in detecting borderline spam messages that mimic real communication.



Logistic Regression and Random Forest offered slightly lower but still competitive performance, with Random Forest showing better robustness on noisy inputs.



KNN, though intuitive, lagged in accuracy due to its reliance on distance metrics, which are less effective in sparse TF-IDF vectors.

Snapshot 5.1 Login page

4.2 Voice Classification Results For the voice classification module, a dataset containing genuine and fake voice samples (including deepfake and synthetic speech) was pre-processed using MFCC feature extraction. Two deep learning models—Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM)—were trained and evaluated.

Model

Accuracy

Precision

Recall

F1-Score

Logistic regression

94.8%

94.5%

94.2%

94.3%

SVM

96.1%

95.9%

95.6%

95.7%

Random Forest

95.6%

95.2%

94.8%

95.0%

KNN

92.3%

91.8%

90.5%

91.1%

Snapshot 5.2 Text prediction page

Table 4.2 Voice Classification Result

© 2025, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 839

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 12 Issue: 07 | July 2025

p-ISSN: 2395-0072

www.irjet.net

The system was successfully deployed as a Flask-based web application, offering users an accessible platform to input messages or upload voice recordings for real-time classification. The application demonstrated strong responsiveness and usability, with prediction times under three seconds and a clear, user-friendly interface. This confirms the feasibility of deploying such a system in realworld environments like call centres, banking, government, and telecom sectors.

REFERENCES Snapshot 5.3 Voice prediction page

[1] Breda, Filipe & Barbosa, Hugo & Morais, Telmo. (2017).

SOCIAL ENGINEERING AND CYBER SECURITY. 4204-4211.

10.21125/inted.2017.1008.

[2] Salahdine, Fatima & Kaabouch, Naima. (2019). Social Engineering Attacks: A Survey. Future Internet. 11.

10.3390/fi11040089.

[3] Naz, Anam & Sarwar, Madiha & Kaleem, Muhammad & Mushtaq, Muhammad & Rashid, Salman. (2024). A comprehensive survey on social engineering-based attacks on social networks. International Journal of ADVANCED AND APPLIED SCIENCES. 11.139-154. 10.21833/ijaas.2024.04.016.

Snapshot 5.4 Prediction Result

[4] Deepak Singh Malik Social Engineering Attacks and their mitigation solution link https://ijaem.net/issue_dcp/Social%20Engineering%20Atta cks%20and%20their%20mitigation%20solutions.pdf

6.CONCLUSION The growing sophistication of social engineering attacks, particularly those exploiting human vulnerabilities through telephonic calls and fraudulent text messages, demands the development of intelligent, automated detection systems.

This paper presented a dual-modality AI-based system capable of detecting social engineering threats in both text and voice communication channels. The proposed framework integrates Natural Language Processing (NLP), classical Machine Learning (ML), and Deep Learning (DL) techniques to address both message-based phishing and audio-based impersonation attempts, including deepfake speech.

[5] Tejal Rathod, Nilesh Kumar Jadav, Sudeep Tanwar, Abdulatif Alabdulatif, Deepak Garg, Anupam Singh,A comprehensive survey on social engineering attacks, countermeasures, case study, and research challenges, Information Processing & Management, Volume 62, Issue 1,2025,103928, ISSN 0306-4573, https://doi.org/10.1016/j.ipm.2024.103928.

[6] AMEER HAMZA Deepfake Audio Detection via MFCC Features Using Machine Learning Received 14 December 2022, accepted 20 December 2022, date of publication 21 December 2022, date of current version 29 December 2022.

Digital Object Identifier 10.1109/ACCESS.2022.3231480

For textual classification, models such as Support Vector Machine (SVM), Logistic Regression, Random Forest, and KNearest Neighbours were evaluated using TF-IDF features.

Among them, SVM consistently delivered superior performance, achieving over 96% accuracy in identifying scam messages. On the auditory front, a robust pipeline using Mel-Frequency Cepstral Coefficients (MFCC) was implemented, followed by CNN and LSTM models for deepfake voice detection. LSTM proved more effective, reaching an accuracy of 96.2% due to its temporal sensitivity to speech patterns.

© 2025, IRJET

|

Impact Factor value: 8.315

[7] OMAIR AHMAD Deepfake Audio Detection for Urdu Language Using Deep Neural Networks Received 7 April 2025, accepted 7 May 2025, date of publication 19 May 2025, date of current version 10 June 2025. Digital Object Identifier 10.1109/ACCESS.2025.3571293 [8] Hamza, Ameer & Javed, Abdul Rehman & Iqbal, Farkhud & Kryvinska, Natalia & Almadhor, Ahmad & Jalil, Zunera & Borghol, Rouba. (2022). Deepfake Audio Detection via MFCC Features Using Machine Learning. IEEE Access. PP. 1-1.

10.1109/ACCESS.2022.3231480.

|

ISO 9001:2008 Certified Journal

|

Page 840

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 12 Issue: 07 | July 2025

p-ISSN: 2395-0072

www.irjet.net

[9] Almutairi, Z.; Elgibreen, H. A Review of Modern Audio Deepfake Detection Methods: Challenges and Future Directions. Algorithms 2022, 15,155. https://doi.org/10.3390/a15050155 [10] Ezer Osei Yeboah-Boateng & Priscilla Mateko Amanor (2024),Phishing, Smishing & Vishing: An Assessment of Threats against Mobile Devices.

BIOGRAPHIES Lavanya K M, received Bachelor’s Degree in Computer Applications from University of Mysore, India and she Currently pursuing MCA in P.E.S College of Engineering, Mandya.

Dr. H.P Mohan Kumar, Obtained MCA, MSc(Tech) and PhD from University of Mysore, India in 1998, 2009 and 2015 respectively.

He is Working as a Professor in Dept. of CS&E, P.E.S. College of Engineering, Mandya, Karnataka, India, His Area of Interest are Computer Vision, video analysis and Data Mining.

© 2025, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 841