Skip to content

Is AI Information Trustworthy? I Asked ChatGPT

Picture of Eric Magill
The explosion of hype surrounding Artificial Intelligence systems since ChatGPT burst the technology into the national consciousness has clouded much of the reality of its capabilities.
 
So much so, that rather than waste time sorting through it all, I thought I'd start by asking ChatGPT itself, "How accurate is information from Artificial Intelligence?"
 
I could have cut and pasted the surprisingly candid response, as it matched my current opinion of using AI for research and information vs. other endeavors that to date it seems more suited for, such as coding and automation.
 
I had the same opinion myself when I last wrote about it in February.
 
ChatGPT highlighted seven factors that affect the accuracy of AI information:
 
  1. Data Quality -- based on the quality of the data it has been trained on
  2. Knowledge Cutoff -- whatever that cutoff date is, the AI system will not be aware of information since then (ChatGPT routinely tells me it doesn't know anything since its Knowledge Cutoff of September 2021, so more than two years ago)
  3. Context and Scope -- AI systems don't always understand context or scope
  4. Confirmation Bias -- can creep into AI answers if it has been introduced into the training data, either sub-consciously or purposefully.
  5. Subject Matter -- AI will be more accurate on some subjects than others but less so when asked for opinions or subjectivity
  6. Regular Updates -- a big factor in accuracy as information changes over time, particularly during and since COVID
  7. Verification -- verbatim from ChatGPT on the importance of verifying AI responses ... "It's always a good practice to verify information obtained from AI, especially for critical decisions or sensitive topics. Double-checking with reliable sources is essential."
ChatGPT concluded with this ... 
 
"In general, AI can provide accurate and reliable information for a wide range of topics, but it's important to use AI as a tool to supplement human decision-making and not as a sole source of information, especially when precision and context are critical."
 
ChatGPT even pops up a disclaimer when you log in:
"While we have safeguards, ChatGPT may give you inaccurate information. It's not intended to give advice."
 
I have felt from the beginning that while AI provides useful clues on where to start or where to look when information searches hit a dead-end, its responses can't be relied on for business decision-making ... yet.
 
You still need to do the leg-work for that.
 
For help with fact-checking and finding credible, updated, authoritative research and information that can be used in business decision-making or Vendor Risk Assessments, contact me at 302-537-4198, ericm@edminfopro.com or on our Contact form.
 

You can also request an online meeting.
, , , , , ,