Can AI fool people into believing fabricated or hallucinated information ?

Can artificial intelligence be used to deceive by fabricating or hallucinating convincing yet false information? Generative AI systems now produce text indistinguishable from human writing, a capability increasingly exploited in phishing, propaganda, and synthetic media.

Background

AI can produce convincing fabricated or hallucinated information that people often accept as true, especially when the output is tailored to sound authoritative or emotionally resonant. Systems like large language models can generate text indistinguishable from human writing, which has been exploited in phishing, misinformation campaigns, and deepfake content creation. Studies show that humans are prone to trusting AI-generated text despite its inaccuracies, particularly when it aligns with their existing beliefs or is presented with confidence. However, AI lacks true understanding, relying on patterns in training data rather than factual verification, which can lead to plausible but false assertions. Detection tools exist but are not foolproof, as adversaries continuously refine methods to bypass safeguards. Social media platforms and regulators have struggled to keep pace with the spread of AI-generated disinformation, which can erode public trust and influence real-world behavior. The risk is amplified when AI systems are fine-tuned on biased or low-quality data, further distorting outputs.