top of page

ChatGPT - Please Write My Biography: Understanding the Limits of Large Language Models



Large Language Models (LLMs) such as ChatGPT have been getting a LOT of attention lately, and it is important to know what to make of all the hype. LLMs are a promising tool, but they are still searching for their niche. While it is important to pay attention to emerging technologies such as these, it is equally important to be realistic about their limitations.


I recently watched a fantastic explainer on LLMs by Raf Alvarado at the University of Virginia School of Data Science. Following his lead, I asked ChatGPT to write my biography:


Write a biography of William Russin (Russ) Clay, Senior Data Scientist in the University of Virginia Health System


I was flattered by the response. ChatGPT wrote a glowing biography which came across as the introduction to a Keynote address at the culmination of a distinguished career. However, as Alvarado also found, most of it was either overly general or completely fabricated. Here is the biography that ChatGPT wrote. I color-coded parts of it green if it indicated something that was true, and red if it was false. (I also added my own commentary in parentheses). Note that there is plenty in here that is not objectively true or false, but is definitely misleading (e.g., that I am a 'prominent' figure in the field of data science).


 

Title: William Russin (Russ) Clay: Pioneering Data Scientist in the University of Virginia Health System


Introduction:

William Russin Clay, known affectionately as Russ Clay, is a prominent figure in the field of data science, with a specific focus on healthcare analytics. With a distinguished career as a senior data scientist in the University of Virginia (UVA) Health System, Russ Clay has played a pivotal role in harnessing the power of data to transform patient care, optimize medical processes, and drive scientific advancements. His expertise in data analysis and innovative problem-solving has earned him recognition and admiration within the healthcare industry.


Early Life and Education:

Born and raised in Charlottesville, Virginia, (I was born and raised in Greensburg, Pennsylvania) Russ Clay developed a passion for technology and mathematics from a young age. He exhibited exceptional aptitude in both subjects throughout his academic journey. (I was always decent at tech and math, but I wouldn't say I excelled. I initially struggled with programming and advanced calculus. Statistics didn't come easy either. I think the fact that I had a passionate interest in these subjects, but not an intuitive knack for them helped me understand them more deeply in the long run) After completing high school with top honors, Russ Clay enrolled at the University of Virginia, where he pursued a degree in computer science and statistics. (I went to Carnegie Mellon for my undergraduate degree in Information Systems, and I did my PhD work in Experimental Psychology at Virginia Commonwealth University) His undergraduate years provided a strong foundation in data analysis, programming, and quantitative research methodologies. (True, but it didn't really happen the way the biography was written)


Career and Contributions:

Upon graduating from UVA, Russ Clay embarked on a career path that merged his technical skills with his passion for healthcare. He joined the University of Virginia Health System as a data analyst, where he quickly distinguished himself through his ability to extract valuable insights from complex datasets. Recognizing his potential, the organization offered him opportunities for professional growth, and he took on increasingly challenging roles.


As a senior data scientist, Russ Clay has been instrumental in transforming the University of Virginia Health System's data-driven approach to patient care. He has implemented cutting-edge analytics techniques, such as machine learning and predictive modeling, to improve clinical decision-making and optimize resource allocation. By leveraging large-scale datasets, Russ Clay has identified patterns, trends, and correlations that have paved the way for evidence-based interventions and improved patient outcomes.


Beyond his work within the UVA Health System, Russ Clay has actively contributed to the wider data science community. He has authored numerous research papers, presenting his findings at renowned conferences and symposiums. (True, but most of my academic publications are in experimental psychology, with a few contributions to health care research more recently) His work on predictive modeling for early disease detection has garnered significant attention, earning him accolades and invitations to collaborate with esteemed researchers from other institutions.


Russ Clay's contributions extend beyond academia. He is known for his passion for mentorship and fostering a collaborative environment. He has mentored and guided aspiring data scientists, helping them develop their skills and navigate the complex field of healthcare analytics. Russ Clay's commitment to fostering interdisciplinary collaborations has led to groundbreaking projects, bringing together experts from diverse fields to tackle complex healthcare challenges.


Personal Life and Philanthropy:

Outside of his professional endeavors, Russ Clay is known for his humility and compassionate nature. He is deeply committed to giving back to his community and has actively participated in various philanthropic initiatives. He has volunteered his time and expertise to nonprofit organizations dedicated to healthcare improvement and data literacy. (I do make a point to maintain involvement with nonprofits, but this hasn't historically involved healthcare and data literacy)


In his leisure time, Russ Clay enjoys hiking in the picturesque Blue Ridge Mountains surrounding Charlottesville. He is an avid reader and enjoys exploring topics beyond his field, further expanding his intellectual horizons.


Conclusion:

William Russin (Russ) Clay's unwavering dedication to data science and healthcare has established him as a leader in his field. Through his work as a senior data scientist in the University of Virginia Health System, Russ Clay has transformed the way data is utilized to enhance patient care and medical research. His innovative solutions, commitment to mentorship, and philanthropic contributions have positively impacted the healthcare industry and inspired countless individuals to pursue careers in data science. Russ Clay's tireless pursuit of knowledge and his passion for improving healthcare outcomes serve as a testament to his lasting impact on the field of data science and the University of Virginia Health System.


 

This exercise reveals what LLMs are currently good at: They are good at creating and writing language; language that sounds good, and language that is structurally correct based on lots of previous examples. However, they are not necessarily a good primary source of factual information. Unfortunately, we humans tend to think that if something is well written and sounds professional, it is probably true. Because LLMs can write so well, it will be increasingly important for us to verify information before we trust it. Advances in large language models are a clear breakthrough in the technological landscape. However, LLMs are still in search of their niche - and until they find it, we need to be careful not to assume that they are capable of things that they were not designed to do!


Russ Clay, PhD - Principal Data Scientist / Founder

120 views0 comments

Comments


bottom of page