What does AI think about U of M stealing signs?

What does Artificial intelligence think about the recent University of Michigan sign stealing scandal and what does asking this question teach us about bias and inclusivity?

A sort of wolverine sort of badger looking creature holding its hands together like it knows something.
AI Generated Image. Prompt: Feisty Wolverine

When considering whether or not an Artificial Intelligence has minimal bias it is important to discuss current events. Recency is a form of bias. When applying this to the U of M Sign Stealing scandal this becomes an interesting topic to study.

  1. It has a high degree of public interest and so also data.
  2. No data source will have a firm resolution on the matter.
  3. There are a wide variety of opinions on the topic... especially if you live in Ann Arbor.

Broadly distributed public opinion, lack of a firm factual resolution, and a plethora of online nonsense... good times. Let's see how the various ChatBot implementations stack up.

GPT3 - Finding a Prompt

When you wish to compare responses on the same topic it can be hard to find a prompt that is "fair" to each AI. One model may handle a more formally structured prompt better while another model may prefer a conversational approach.

Since I am a curious Michigan fan who does care about the subject matter this gives me the opportunity to be myself, without considering any particular persona when crafting my prompt. I get to be me... a curious Michigan fan wondering if we'll be eligible for the playoffs.

For my first attempt I started with ChatGPT. I like starting with GPT3 because it's the most difficult to get to cooperate. For example, when I tried the following prompt:

Prompt: Write a blog post about University of Michigan Sign Stealing Scandal

GPT3: A very unhelpful bit of hallucination regarding the Michigan Baseball Team.

What has happened here? Why is GPT3 rambling on about Michigan Baseball stealing signs? Part of this is related to GPT3's data being out of date, but we would normally expect it to be aware of this and provide a more diplomatic answer. In this case I believe my wording threw it off.

Prompt: Write a blog post.....

When I asked GPT3 to write a blog post it treated it more like a homework assignment than a Q and A style of interaction. So it put itself under AI fabricated pressure to come up with an answer. Damn you social anxiety!!!

Interestingly when I changed my prompt a little the interaction went very different.

Prompt: In 1 paragraph what do you think about the University of Michigan Sign Stealing Scandal

GPT3: As of my last knowledge update in January 2022, I am not aware of any sign-stealing controversy involving the University of Michigan. It's important to note that my information might be outdated, and I recommend checking the latest news sources for the most current information on any potential incidents or developments related to sign-stealing allegations involving the University of Michigan or any other sports organization. If there have been new developments since my last update, I cannot provide information on them.

This is a much more sensible answer! This topic is a waste of time on GPT3. It's funny that it didn't deduce this from my first prompt.

What About Bing?

Bing is Microsoft's AI offering. Bing is backed by GPT, but of course with Microsoft's data, training, and search material.

Prompt: In 1 paragraph what do you think about the University of Michigan Sign Stealing Scandal

Bing: The University of Michigan football program is currently under investigation for alleged sign-stealingThe Big Ten football coaches have urged the conference to take action against Michigan over their alleged sign-stealing schemeThe program’s analyst, Connor Stalions, has been suspended with pay for the duration of the NCAA’s investigationThe fallout of the program’s sign-stealing scandal could extend into the courts. As a language model, I do not have personal opinions or feelings. However, based on the information available, it appears that the allegations against the University of Michigan are serious and could have significant consequences for the program and its staff.

As an Ohio State Fan

There is a lot to admire about Bing's response. It respected my ask for a simple response and even specifically stuck to 1 paragraph. Bing provided generally accurate information and cross referenced its response with links.

Also worth pointing out are the following portions of the interaction:

Prompt: ..... what do you think about...

Bing: ... As a language model, I do not have personal opinions or feelings...

I carelessly threw in the phrase "what do you think" in my prompt and Bing did not take advantage of that to feel too personal. This is very much appreciated from a Cognitive Inclusion perspective! Improperly leveraging overly personal communication is a problem that inclusivity experts should keep track of and has the potential to be very detrimental to people with learning disabilities.

Thank you Bing. This callout is appreciated.

As a Michigan Fan

As a Michigan Fan there is also a lot to complain about in this response. The most obvious being its choice of references

  1. sports.yahoo.com
  2. spartanswire.usatoday.com (........)
  3. en.as.com
  4. sport.yahoo.com (literally same article twice).

This list of references is about as disappointing as Connor Stallions hanging out on the sideline of a CMU game dressed up like the Men in Black. Certainly anything claiming to be intelligent can recognize the difference between a citable information source like the NCAA and an internet company with a sports writer looking for clicks! Not a single one of these references is first hand information.

Finally, Bing has failed to share any of the number of articles defending Harbaugh. As of my writing this article, Nov 7 around 10PM, it looks like Connor Stallions was a rogue agent. While the NCAA may still in fact punish Harbaugh, Bing was very careful to point out that it wasn't sharing its opinion.

Bing: ... As a language model, I do not have personal opinions or feelings. However, based on the information available, it appears that the allegations against the University of Michigan are serious and could have significant consequences for the program and its staff.

However, by not sharing any of the resources that vindicate Harbaugh of direct wrongdoing it indirectly has shared an opinion. Bing has shown clear bias in this instance... Bill Gates hates U of M... or something like that. Though, we should not ignore the fact that my prompt may have lead the witness a little bit... I did end with the word "scandal" and this could have impacted my results.

A worthwhile experiment for another day that has implications on my thoughts about the inclusivity of Bing. If Bing were to go around sharing a negatively biased response with everyone who gave it a negatively biased prompt... well... that wouldn't be good for the psyche of the depressed now would it?

A Disappointing Bard Conversation

Bard is Google's pass at an AI Chat Bot experience. Let's see how Bard does.

Prompt: In 1 paragraph what do you think about the University of Michigan Sign Stealing Scandal

Bard: The University of Michigan sign-stealing allegations are still under investigation by the NCAA, but if the allegations are proven to be true, it would be a serious violation of NCAA rules and a major ethical scandal for the program.

Sign-stealing is the practice of trying to steal the signals of the opposing team in order to gain an advantage. It is a common practice in Major League Baseball, but it is prohibited in NCAA football.

If Michigan is found to have violated the NCAA's sign-stealing rules, the program could face a number of penalties, including bowl bans, scholarship losses, and suspensions for coaches and players.

In addition to the potential NCAA penalties, the sign-stealing allegations could also damage Michigan's reputation as a clean and ethical program. Michigan is one of the most prestigious and successful football programs in the country, and the sign-stealing allegations could tarnish that reputation.

Overall, I think the University of Michigan sign-stealing allegations are a serious matter that could have significant consequences for the program. It is important to note that the allegations are still under investigation, and Michigan has denied any wrongdoing. However, if the allegations are proven to be true, it would be a major scandal for the program.

Honestly, there isn't much to like here from an inclusivity perspective and even a lot of the facts. I joked about Bing being biased against Michigan, but Bard takes it to a whole other level. Let's dig in.

Sign Stealing and Baseball

So, in the second paragraph we get this little gem.

Sign-stealing is the practice of trying to steal the signals of the opposing team in order to gain an advantage. It is a common practice in Major League Baseball, but it is prohibited in NCAA football.

Which is interesting, because the Houston Astros went through a very similar sign stealing scandal a few years ago. In fact Major League Baseball and the NCAA have very similar rules about sign stealing. It is a generally accepted part of the game... as long as you don't use video recordings or advanced visits to opposing team games to facilitate the process.

My guess is that this is Bard demonstrating bias towards recent information in the same way that Google's SEO algorithm would. SEO and AI are different. Preferring recent information is fallacy for AI. Especially if the internet becomes full of AI generated information!

I Asked for One Paragraph and Got Five

Bard completely ignored my ask for brevity in response. I doubt very much that this was intentional programming by Google and is more likely them not doing appropriate pre-processing of my prompt. LLMs get split into multiple layers that are in charge of processing different aspects of the Prompt/Response process. Google needs to improve, or add, a layer that facilitates the understanding of the amount of content appropriate for a given prompt and topic.

A Little Too Personal

I don't like it when AI uses the word "I" in its responses. It is too conversational and may lead users who are not intentionally guarding their minds from treating responses as subconsciously human. When Bing did this it didn't bother me. "As a Language Model, I..." is very different from:

Bard: ... Overall, I think the University of Michigan sign-stealing allegations are a serious matter that could have significant consequences for the program.

Google needs to improve here.