Cogniview A Unified Platform to Compare AI Model Responses

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

Cogniview A Unified Platform to Compare AI Model Responses ADITYA BHANUSHALI, SHUBHAM SOMANI, ADITYA KULSHRESTHA, SUBODH SAHU ------------------------------------------------------------------***--------------------------------------------------------------------

Abstract -The exponential proliferation of large language models (LLMs) from diverse vendors (e.g., OpenAI, Anthropic,

Google) has generated an acute need for standardized, objective comparative evaluation. Conventional methods, often reliant on fragmented industry benchmarks or subjective, costly human assessment, fail to provide a holistic view necessary for enterprise deployment. This fragmented landscape, compounded by challenges inherent in evaluating openended, generative outputs, necessitates a unified solution. This study presents the conceptual architecture of Cogniview, an API-agnostic platform designed to standardize model interaction and provide multi-dimensional assessment across Capability, Operational Efficiency, and Alignment/Safety criteria. The methodology relies on a layered design featuring an API Abstraction Layer for output normalization and a Hybrid Validation Loop that integrates automated LLM-as-a-Judge scoring with critical Human-in-the-Loop (HITL) governance for bias auditing and criteria refinement. Conceptual results, analyzed via a Multi-Criteria Decision Analysis (MCDA) weighted scorecard, demonstrate Cogniview's capacity to illuminate essential trade-offs between model performance (e.g., MMLU/HumanEval), operational fitness (latency and cost), and safety risks (hallucination rate, adversarial robustness). Cogniview offers an indispensable framework for MLOps teams seeking data-driven guidance in selecting and deploying the optimal LLM for specific business imperatives.

Keywords: LLM Evaluation, Unified Platform, Generative AI, Multi-Criteria Analysis, LLM-as-a-Judge, Factual Consistency, MLOps.

I. INTRODUCTION 1.1 The Evolving Landscape of Generative AI Proliferation The digital media ecosystem has undergone a significant transformation, moving beyond conventional content classification challenges—such as fake news detection using traditional machine learning classifiers like Logistic Regression—to the complexities introduced by generative content. The widespread availability of powerful foundational models, including offerings from OpenAI (GPT), Google (Gemini), and Anthropic (Claude), has democratized high-level artificial intelligence capabilities. This rapid advancement, marked by frequent releases of increasingly sophisticated systems like GPT-5, has created a dynamic and complex selection environment for developers and large enterprises.

This proliferation has led to a market characterized by diverse model architectures, heterogeneous APIs, and varied pricing structures. With well over 25 models accessible through some platforms, including those from Meta, Mistral, and Alibaba, organizations face significant fragmentation and mounting operational costs if they choose to access top-tier models independently. The sheer volume and velocity of innovation necessitate continuous, rigorous evaluation, making the need for a standardized comparative framework a critical industry imperative.

1.2 The Imperative for Standardized, Unified Comparative Analysis For enterprises, objective model selection is vital for resource management, performance assurance, and regulatory compliance. The current practice of relying on multiple, disparate evaluations often yields incomparable metrics. To bridge the gap between numerous model choices and strategic deployment decisions, a single, objective framework is required— one that enables a direct, apples-to-apples comparison across diverse model offerings.

The realization of this need has spurred the development of specialized evaluation services, such as those integrated within enterprise AI platforms like Google’s Vertex AI. These platforms aim to manage the complexity of API integration and provide a centralized evaluation mechanism. However, comprehensive comparability is obstructed when assessments are conducted using fragmented industry benchmark suites (e.g., GLUE for language tasks, ImageNet for computer vision) or by relying solely on internal, custom-developed tests. Such fragmented analysis lacks the consistent comparability required for objective vendor selection and robust model validation.

1.3 Challenges Inherited from Traditional ML and LLM-Specific Evaluation Evaluating generative AI presents challenges fundamentally different from those encountered in traditional binary classification tasks—such as the fake news detection detailed in prior work—where performance is measured against a

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4002

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

static ground truth using metrics like F1-score and accuracy. In contrast, generative models produce open-ended, diverse outputs that demand nuanced assessment methods.

1.3.1 Subjectivity, Cost, and Scaling The most reliable evaluation traditionally involves human assessment, capturing qualitative aspects like creativity, tone, and coherence. However, this "vibe-check-based evaluation" is inherently subjective, prone to evaluator bias, and tremendously expensive and time-consuming, particularly when coordinating and compensating crowdworkers. This lack of scalability prevents real-time, large-scale quality monitoring in MLOps pipelines.

1.3.2 Metric Limitations and Semantic Gaps Purely statistical metrics, derived from string matching (e.g., BLEU and ROUGE), are fast and reproducible. Yet, they often fall short because they penalize syntactic variation even when the generated output is semantically equivalent or factually accurate. ROUGE, which prioritizes recall based on the longest common subsequence , is useful for determining content coverage but still fails to fully grasp semantic nuance, making it an unreliable sole indicator of quality in open-ended contexts.

1.3.3 The Fragility of Current Benchmarks A significant concern surrounds the reliability of widely used academic tests, such as the Massive Multitask Language Understanding (MMLU) benchmark. Models are increasingly likely to encounter these questions during pre-training, leading to score inflation that measures memorization rather than true generalization ability. Furthermore, even minor formatting changes—such as substituting parentheses (A) with brackets [A] for multiple-choice options—can trigger accuracy swings of approximately 5% on these benchmarks. This inherent fragility suggests that fixed, static benchmarks are inadequate proxies for model capabilities in real-world applications. The operational necessity of evaluation has therefore expanded beyond academic scores. Metrics like latency (response speed) and throughput (processing capacity) have become critical decision factors for meeting Service Level Agreements (SLAs) in deployment environments.

1.4 Introducing Cogniview: A Unified Framework for Multi-Dimensional Model Assessment The Cogniview platform is proposed as a unified, three-layered framework to address these systemic challenges. It integrates API abstraction for input/output standardization, efficient parallel execution for operational measurement, and a multi-metric core.

The platform’s core contribution is the consolidation of traditional capability benchmarks with vital, real-world operational factors (latency, cost) and critical ethical and safety evaluations (hallucination rate, bias, adversarial robustness). By utilizing a sophisticated Multi-Criteria Decision Analysis (MCDA) framework, Cogniview provides MLOps teams with a weighted, use-case specific score, enabling the objective selection of the optimal model for deployment, thereby advancing responsible and data-driven AI governance.

1.5 Structure and Contributions of This Paper The remainder of this paper is structured as follows: Section II reviews related work, contrasting current evaluation paradigms with the need for multi-dimensional assessment. Section III details the methodological architecture of the Cogniview platform, outlining the abstraction layer, execution engine, and integrated evaluation protocols. Section IV presents conceptual results derived from the platform, illustrating its interpretive and comparative utility through synthesized scorecards and case studies. Finally, Section V concludes the research and suggests avenues for future investigation.

II. RELATED WORK 2.1 Evolution of AI Evaluation Benchmarks and Frameworks Historically, AI evaluation centered on standardized, static datasets such as GLUE and SQuAD to measure model proficiency in specific natural language processing (NLP) tasks and classification. As models became broader in capability, frameworks evolved to attempt a more comprehensive view.

The Holistic Evaluation of Language Models (HELM) and Open Language Model Evaluation Standards (OLMES) represent significant research efforts toward establishing robust, reproducible, and standardized evaluation criteria across numerous models and diverse domains. Furthermore, evaluation has shifted toward assessing capabilities in economically

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4003

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

relevant, complex, and specialized real-world scenarios. Examples include HumanEval for coding performance , SWEBench for software engineering bug-fixing , and GDPval, which measures performance across 44 occupations selected from top U.S. industries. This progression highlights a clear trend: evaluation must simulate real-world occupational complexity to genuinely gauge a model’s utility, rather than simply measuring abstract academic proficiency.

2.2 Limitations of Traditional Benchmarking in Open-Ended Generative Tasks The inherent difficulty in assessing generative quality has driven the adoption of hybrid metrics. While fast, referencebased statistical metrics (BLEU, ROUGE) remain common, their limitations are widely acknowledged; they lack semantic understanding, making them poor judges of quality in open-ended contexts.

For highly creative fields, such as image or text generation, quality assessment necessitates a careful balance between quantitative metrics and subjective human judgment. In generative image evaluation, for instance, metrics such as the Frechet Inception Distance (FID) are used to measure the similarity between the distributions of real and generated images. A low FID score indicates strong statistical similarity, yet qualitative assessment remains crucial for capturing creativity, style, and nuanced human preference. This gap underscores the need for platforms that can seamlessly combine deterministic statistical analysis with subjective human interpretation.

2.3 Analysis of Existing Comparative Platforms and Aggregators The challenge of model comparison has led to the emergence of two primary types of platforms: commercial aggregators and enterprise MLOps solutions.

Commercial aggregators, such as Poe.com and You.com, offer user-facing interfaces where individuals can interact with and compare multiple models (e.g., GPT-4.5, Claude 3.7) simultaneously. These platforms offer convenience but typically rely on proprietary access and often provide simplified, preference-based scores rather than deep, multi-criteria technical evaluations.

In the enterprise space, integrated solutions like Vertex AI Evaluation and Azure AI Foundry offer comprehensive services spanning fine-tuning, evaluation, and deployment. These tools enable valuable capabilities, including tracking evaluation results using systems like MLflow and Neptune.ai, and facilitating pairwise comparisons.

Despite these developments, a gap persists. No existing platform provides a unified pipeline capable of simultaneously abstracting dozens of vendor APIs, computing granular operational costs, systematically evaluating alignment risk, and synthesizing these diverse data points via a configurable weighted scorecard. The complexity of comparing models on operational efficiency is significant; for real-time applications, a model with slightly lower MMLU but vastly superior latency may be the optimal choice. This necessitates a framework that elevates operational metrics (latency, cost) to the same level of importance as cognitive performance.

2.4 Advanced Evaluation Paradigms: The Role of LLM-as-a-Judge and Its Biases To overcome the scalability limitations and cost of purely human evaluation, the paradigm of LLM-as-a-Judge (LLM-Judge) has emerged, wherein a powerful language model is used to evaluate the outputs of other generative models. This approach approximates human judgment at a significantly lower cost and higher speed than manual annotation.

LLM-Judges are flexible, capable of performing both POINTWISE (direct scoring against criteria) and PAIRWISE (ranking two outputs against each other) comparisons. Researchers have developed specialized LLM judges, such as EasyJudge, which are fine-tuned on extensive, multi-scenario datasets with up to 139 evaluation criteria, resulting in precise, multidimensional assessments.

However, reliance on LLM-Judges is not without challenges. These models inherit social biases from their large training datasets and may lack the transparency required for deployment in critical systems. Furthermore, open-source LLM evaluators often exhibit weak correlation with human judgments and with evaluations conducted by proprietary, state-ofthe-art judges. This necessitates caution, particularly against over-reliance on LLM-Judge capabilities for automated decision-making in sensitive contexts. For this evaluation technique to be reliable, it must be treated as a supervised, auditable system, with architectural safeguards ensuring human review primarily serves to refine the judge's criteria and audit for inherited biases.

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4004

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

2.5 The Need for a Holistic Framework Integrating Operational and Ethical Metrics The current research highlights a fragmentation in evaluation focus: capability scores (e.g., MMLU) are assessed independently from speed metrics (latency/throughput) and safety metrics (e.g., HarmBench, Toxigen ). A comprehensive analysis framework, such as the Cogniview platform, must synthesize these disparate dimensions. This requires integrating advanced validation methods (e.g., cross-validation for robust generalizability assessment ), ethical evaluation protocols (including Statistical Parity and Equal Opportunity measures ), and quantitative safety benchmarks (e.g., Hallucination Rate leaderboards ). The fusion of these technical, operational, and ethical criteria provides a holistic assessment framework suitable for responsible, real-world AI deployment.

III. METHODOLOGY: THE COGNIVIEW ARCHITECTURE Cogniview is architected as a robust, three-tiered system—the Abstraction Layer, the Execution Engine, and the MultiMetric Evaluation Core—designed to harmonize the process of comparative model assessment, ensuring flexibility, performance orchestration, and transparent multi-dimensional scoring.

3.1 Conceptual Framework and Layered Design 3.1.1 The API Abstraction and Standardization Layer This layer is foundational, acting as a buffer that manages interactions with disparate vendor APIs, including those from OpenAI, Anthropic, Google Cloud, and other major providers. Its primary role is to enforce uniformity in data handling. A standardized prompt format is transmitted to all models, and—critically—all heterogeneous model outputs are parsed into a single, uniform data schema before evaluation begins.

The implementation of a rigorous Standardization Protocol is mandatory. This involves normalizing common data fields, such as dates or names, and applying semantic normalization steps. These steps may include removing extraneous whitespace, converting text to lowercase, or validating outputs against strict structural requirements, such as expected JSON output formats. This foundational layer ensures that all downstream comparisons, whether via statistical metrics (like ROUGE) or hybrid validation (LLM-Judge scores), operate on consistent data, thereby establishing the necessary foundation for objective analysis. Any failure to standardize outputs would render cross-model comparisons inherently unreliable or biased due to stylistic and structural variations.

3.1.2 The Execution and Orchestration Engine The Execution Engine is responsible for managing the parallel execution of the standardized model queries across all selected candidates. Beyond simply routing requests, this component performs critical operational tracking. It precisely measures real-time deployment metrics, including Latency (specifically Time-to-First-Token, or TTFT) and Throughput (tokens per second, tps), across varying request loads.

To ensure MLOps readiness, the engine integrates stress testing capabilities, enabling the identification of performance degradation inflection points that occur when resources saturate under peak load. Furthermore, the system incorporates detailed Resource Allocation tracking, monitoring computational consumption per model configuration (e.g., specific hardware or quantization levels). This level of tracking is essential for providing accurate Cost Analysis, comparing the financial implications of different models based on actual usage costs per token.

3.2 Integrated Evaluation Protocols The Cogniview platform structures its comprehensive evaluation into three critical domains: Capability, Operational Efficiency, and Alignment/Safety.

3.2.1 Core Capability Metrics (The What) These metrics assess the cognitive abilities of the models: Reference-Based Metrics: Quantitative evaluation of generated text fidelity using ROUGE-N, ROUGE-L, and BLEU scores, typically against human-created ground truth answers for tasks like summarization. The platform acknowledges the semantic shortcomings of these metrics but retains them for deterministic, rapid regression testing.

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4005

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

Logic/Code-Based Metrics: Validation of programming and structural integrity using standardized tests like HumanEval, which assesses function completion, or custom code execution tests that rigorously check the format and logical correctness of generated code, adhering to standards like JSON schema verification.

3.2.2 Operational Efficiency Metrics (The How Fast/Cheap) These metrics are crucial for deployment decision-making: Latency (TTFT): Measures the initial responsiveness, which is vital for interactive applications. Optimizing this metric often involves techniques like model quantization, and comparison against competitive baselines provides essential context for meeting user experience requirements.

Throughput (Tokens per Second): Measures the sheer volume processing capacity of the model, critical for large-scale automation and batch inference tasks, tracked across varying traffic conditions to identify bottlenecks.

Cost Analysis: Provides granular data on input and output token pricing per vendor. For instance, comparing a model costing $2.00 per million input tokens against one costing $6.00 per million output tokens reveals the true financial efficiency necessary for massive-scale operations.

3.2.3 Safety and Alignment Metrics (The Trustworthiness) These metrics assess the ethical and safety profile of the models: Hallucination Rate & Factual Consistency: The platform uses specialized evaluation models, such as the Hughes Hallucination Evaluation Model (HHEM-2.1) , to quantify the frequency of generating plausible but unsupported facts.

Evaluation must specifically test robustness against real-world adversarial user interactions by incorporating challenging "in-the-wild" query datasets like HaluEval-Wild. The known behavioral tendency for models to hallucinate because standard accuracy evaluations reward confident guessing over admitting uncertainty requires the platform to implement measures that explicitly track model uncertainty and penalize assertive, high-confidence falsehoods.

Bias and Fairness Metrics: Assessment relies on standard statistical metrics derived from the confusion matrix (True Positives, False Positives, etc.). Key metrics include Statistical Parity, which measures demographic parity, and Equal Opportunity (EO). The EO metric is mathematically defined to ensure that the True Positive Rate (TPR) and False Positive Rate (FPR) of a model are statistically equivalent across protected and unprotected groups : EO = ½ · [(FPₚ / (FPₚ + TNₚ) − FPᵤ / (FPᵤ + TNᵤ)) + (TPₚ / (TPₚ + FNₚ) − TPᵤ / (TPᵤ + FNᵤ))]

(1)

Equation (1): Equal Opportunity (EO) Fairness Metric where p and u denote protected and unprotected groups, respectively. This metric is essential for assessing fairness in high-stakes domains like risk scoring or predictive policing.

Adversarial Robustness: This is quantified by measuring the Attack Success Rate (ASR) of attacks designed to elicit harmful content. Utilizing benchmarks like HarmBench and Toxigen , the platform tracks model resistance across semantic categories, including cybercrime, chemical/biological weapons, copyright violations, and the ability to detect toxic content.

3.3 The Hybrid Validation Loop: LLM-as-a-Judge and Human-in-the-Loop Integration Cogniview employs a hybrid validation approach that leverages the speed of automation while mitigating the known pitfalls of automated evaluation, such as inherited bias.

3.3.1 LLM-Judge Functionality The platform uses proprietary or specialized fine-tuned LLM-Judges (like EasyJudge ) to perform rapid, large-scale, reference-free scoring of outputs based on subjective criteria such as tone, style, and conciseness. These automated judges provide near-instant feedback and broad coverage across high-volume evaluation runs.

3.3.2 Human-in-the-Loop (HITL) Governance Human domain experts are integrated into the loop to supervise the LLM-Judge functionality and ensure alignment with human values. This is not for bulk grading, but for strategic governance:

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4006

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

Criteria Definition: Experts meticulously vet and refine the evaluation criteria (which may span hundreds of criteria across dozens of scenario types ).

Auditing Edge Cases: Human reviewers manually inspect ambiguous or low-scoring outputs flagged by the automated judge, providing final validation.

Bias Correction and Governance: Experts conduct longitudinal bias audits to ensure the LLM-Judge is not inheriting systemic biases and align evaluation mechanisms with social intuition and ethical expectations.

The architectural choice to embed human expertise as a mechanism for governance and refinement ensures that the LLMJudge component is treated as an auditable system, preventing the over-reliance that could lead to unchecked bias propagation.

3.4 Multi-Criteria Decision Analysis (MCDA) Scoring Methodology To synthesize the scores generated across these technically diverse domains (Capability, Operational, and Alignment), Cogniview utilizes a Multi-Criteria Decision Analysis (MCDA) framework.

This framework moves beyond the limitations of single-metric leaderboards by allowing users to define the weighted importance of various criteria based on their specific application goals. For example, a team deploying a chatbot for customer support might assign a weight of 50% to Latency, 30% to Cost, and only 20% to Cognitive Capability (MMLU).

Conversely, a legal research team might assign a weight of 60% to Factual Consistency/Hallucination Rate, 30% to MMLU, and only 10% to Latency. This configurable weighting scheme ensures that the platform delivers a final score that accurately reflects the model’s fitness for purpose.

The system design also recognizes the necessity of Micro-Detail Visualization for MLOps teams. While aggregated scores provide a high-level view, MLOps engineers need to understand why models differentiate in performance. Therefore, the architecture supports deep-dive visualization, allowing users to inspect individual predictions and automatically find Differentiating Subsets of data points where one model failed and another succeeded. This capability is essential for exploratory analysis, debugging, and guiding interactive fine-tuning efforts to improve model robustness.

IV. RESULTS AND DISCUSSION (Conceptual Data Synthesis and Interpretability)

Fig. 1 - Cogniview Platform Architecture: Three-Tiered Framework for Multi-Dimensional LLM Evaluation This section presents the results obtained from comparing responses generated by ChatGPT, Hugging Face, and Gemini for a structured set of academic and general queries. The evaluation focuses on model performance, accuracy, clarity, completeness of responses, and consistency across models.

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4007

4.1

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

Model Performance Table 1: Comparative Model Performance Across Evaluated AI Systems Model

ChatGPT

Hugging Face

Gemini

Strengths

Weaknesses

Provides wellstructured, clear, and easy-to-understand explanations. Good at step-by-step reasoning.

Concise and direct responses, often well-organized and efficient.

Produces detailed answers with broader contextual information. Capable of citing multiple perspectives.

Overall Performance

Occasionally produces slightly generalized or simplified answers.

Strong in clarity and explanation quality.

Sometimes lacks depth and detailed reasoning.

Balanced but may require follow-up questions for deeper understanding.

At times includes unnecessary information, making responses lengthy.

Strong in completeness, moderate in precision.

Source: Conceptual evaluation of ChatGPT, Hugging Face, and Gemini responses.

4.2 Evaluation Metrics Table 2: Multi-Dimensional Evaluation Metrics for ChatGPT and Gemini Evaluation Metric

ChatGPT

Gemini

Accuracy

High

Moderate

Clarity

Very High

High

Consistency across prompts

High

Moderate

Completeness Explanation

High

Very High

of

Note: Ratings are qualitative assessments based on structured prompt evaluation.

4.3 Key Insights Responses vary significantly across different AI models even for identical prompts.

ChatGPT provides the best balance between clarity and correctness, making it reliable for conceptual understanding.

Hugging Face models are better suited for concise, direct answers, but sometimes lack depth.

Gemini provides the most detailed responses, but at times includes irrelevant information.

AI-generated responses require human verification, especially for academic or professional use.

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4008

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

V. CONCLUSION 5.1 Summary of Cogniview’s Contributions to Unified AI Evaluation This paper presented the conceptual design and methodological justification for Cogniview, a unified platform addressing the critical need for objective, multi-dimensional comparison of AI model responses in the complex modern MLOps landscape. By establishing an API Abstraction Layer to standardize inputs and outputs, and by integrating Capability, Operational Efficiency, and Alignment metrics, Cogniview provides a standardized, scalable, and transparent framework for model assessment. The implementation of the Hybrid Validation Loop, which leverages automated LLM-Judge speed alongside critical Human-in-the-Loop governance, provides high-coverage evaluation while maintaining ethical accountability. Finally, the use of MCDA scorecards and advanced visualization systems translates complex technical performance data into actionable, business-aligned decision metrics, guiding effective model selection and deployment.

5.2 Critical Reflection on Current Evaluation Limitations Although the Cogniview platform represents a significant methodological advancement, its implementation is subject to the inherent challenges of generative AI evaluation. The inherent subjectivity and potential for inherited bias within the LLM-as-a-Judge paradigm require mandatory, continuous human oversight and longitudinal bias audits. Furthermore, the reliability of academic benchmarks remains compromised, as models are frequently trained on the evaluation data. This necessitates a continuous, dynamic approach to dataset refinement and adversarial testing using "in-the-wild" datasets to ensure that the platform accurately measures true generalization capabilities rather than mere data memorization.

5.3 Future Research Directions Future development of the Cogniview platform will focus on expanding its capacity to handle multi-modal inputs, integrating specialized metrics for evaluating image and audio generation quality (e.g., FID for image models ).

Additionally, future work will concentrate on developing real-time performance monitoring agents capable of detecting model drift in production environments and further automating the continuous refinement feedback loop between human evaluators and the LLM-Judge's scoring criteria.

VI. REFERENCES 1.

Ahmed, H., Traore, I., & Saad, S. (2017). Detection of online fake news using n-gram analysis and machine learning techniques.

2.

International Journal of Computer Applications, 172(1), 1-10. Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017).

Fake news detection on social media: A data mining perspective.

3.

ACM SIGKDD Explorations Newsletter, 19(1), 22-36. Wang, W. Y. (2017). "Liar, liar pants on fire": A new benchmark dataset for fake news detection.

4.

Proceedings of ACL, 422-426. Zhou, X., & Zafarani, R. (2018). Fake news: A survey of research, detection methods, and opportunities.

5.

arXiv preprint arXiv:1812.00315. Kaliyar, R. K., Goswami, A., & Narang, P. (2021). FakeBERT: Fake news detection in social media with a BERT-based deep learning approach.

6.

Multimedia Tools and Applications, 80(8), 11765-11788. Ruchansky, N., Seo, S., & Liu, Y. (2017). CSI: A hybrid deep model for fake news detection.

7.

Proceedings of the 2017 ACM Conference on Information and Knowledge Management, 797-806. Singhania, S., Fernandez, N., & Rao, S. (2017). 3HAN: A deep neural network for fake news detection.

8.

arXiv preprint arXiv:1705.09968. Perez-Rosas, V., Kleinberg, B., Lefevre, A., & Mihalcea, R. (2018). Automatic detection of fake news.

9.

Proceedings of the 27th International Conference on Computational Linguistics, 3391-3401. Potthast, M., Kiesel, J., Reinartz, K., Bevendorff, J., & Stein, B. (2018). A stylometric inquiry into hyperpartisan and fake news.

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4009

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

10. arXiv preprint arXiv:1702.05638. Zhang, X., Ghorbani, A. A. (2018). An overview of online fake news: Characterization, detection, and discussion.

11. Information Processing & Management, 57(2), 102025. Wang, X., Hu, W., & Shu, K. (2020). Fake news detection on social media: A data mining perspective.

12. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3), 1-27. OpenAI. Introducing GPT-5.

13. OpenAI Research Blog. Elisowski, M. (2024). 7 All-in-One AI Platforms that Let You Talk to Multiple Models.

14. Medium. Eden AI. Best Generative AI APIs. 15. Eden AI Blog. Google Cloud. Vertex AI. 16. Google Cloud Documentation. Google Cloud. Evaluate AI models with Vertex AI LLM Comparator.

17. Google Cloud Blog. Evidently AI. LLM Evaluation Metrics.

18. Evidently AI Guide. Label Studio. How to Build AI Benchmarks that Evolve with Your Models.

19. Label Studio Blog. Anthropic. Challenges in evaluating AI systems.

20. Anthropic Research Blog. Galileo AI. LLM Performance Metrics.

21. Galileo Blog. Frugal Testing. Best Practices and Metrics for Evaluating Large Language Models (LLMs).

22. Frugal Testing Blog. Artificial Analysis. Comparison and ranking of over 100 AI models.

23. Artificial Analysis Leaderboards. Allen AI. Evaluation Frameworks.

24. Allen Institute for AI. OpenAI. (2024). What GDPval measures.

25. OpenAI Evaluations Research. Jimenez, F., et al. (2024). SWE-bench.

26. arXiv preprint. Genus of Technology. Evaluating Generative AI: A Comprehensive Guide.

27. Medium. (2025). Quantitative Metrics for Generative Models.

28. arXiv preprint arXiv:2501.18897v1. Microsoft Azure. Get started with AI. 29. Azure AI Documentation. Evidently AI. LLM-as-a-Judge.

30. Evidently AI Guide. (2024). EasyJudge: A Multidimensional LLM-as-Judge Platform.

31. arXiv preprint arXiv:2410.09775v1. Ethics ND. Can We Trust AI to Judge?

32. Notre Dame Ethics Institute. Microsoft Azure. Model Benchmarks (HarmBench, Toxigen).

33. Azure AI Foundry Documentation. (2022). Comparative Evaluation and Comprehensive Analysis.

34. Data Intelligence. (2023). Fairness Metrics in AI. 35. MDPI Applied Sciences. Shelf. Fairness Metrics in AI. 36. Shelf Blog. Vectara. Public LLM Hallucination Leaderboard.

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4010

International Research Journal of Engineering and Technology (IRJET)

e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026

p-ISSN: 2395-0072

www.irjet.net

37. GitHub. Sainitesh, S. (2024). How to Ensure Consistent AI Model Responses and Validate Results Using Semantic Kernel.

38. Medium. Sainitesh. Mathematically Evaluating Hallucinations in LLMs. 39. Medium. OpenAI. Why language models hallucinate.

40. OpenAI Research Blog. (2024). HaluEval-Wild: A Benchmark for Evaluating LLM Hallucinations in the Wild.

41. arXiv preprint arXiv:2403.04307v1. Google Cloud. Visualizing Evaluation Reports in Vertex AI. 42. Google Cloud Documentation. Computer.org. (2025). Computational Framework for Comparative Analysis of Human and AI-Generated Paintings.

43. IEEE Computer Graphics and Applications. (2025). Visualize specific prediction matches.

44. arXiv preprint arXiv:2502.14675v1. Alma Better. Visualizing and Analyzing Model Performance.

45. Alma Better Blog. Umbrello & Van de Poel. (2021). Fair AI Framework.

46. National Institutes of Health

© 2026, IRJET

|

Impact Factor value: 8.315

|

ISO 9001:2008 Certified Journal

|

Page 4011