Untangling Emotional Threads: Hallucination Networks of Large Language Models

Paper Link

Abstract

Generative AI models, known for their capacity to generate intricate and realistic data, have rapidly found applications across various domains. Yet, doubts linger regarding the full scope of their hallucinatory capabilities and the reliability of their outcomes. These concerns underscore the need for rigorous analysis and validation of generative AI models. This study employs network analysis to explore the inherent characteristics of generative AI models, focusing on their deviations and disparities between generated and actual content. Using GPT3.5 and RoBERTa, we analyze tweets, vocabulary, and emotion networks from their outputs. Although network comparison demonstrated hallucination, non-classification, and instability patterns in GPT-3.5 compared to RoBERTa as a baseline, both models exhibit promise and room for improvement.