I just did a social media experiment with some new Artificial Intelliegence (AI) technology – and the results were pretty shocking.
I’ve heard a lot recently about ChatGPT (https://chat.openai.com/chat), a new AI chatbot that churns out realistic responses to almost any kind of question. For example, you could ask it for suggestions about your ten-year-old’s birthday party or how to debug some code. However, it’s got plenty of teachers worrying about how students could also use it to produce work that isn’t theirs.
The cynic in me thought there was no way a chatbot could produce something good enough to fool teachers or examining boards. So I put this to the test with a couple of non-scientific experiments – and here are the results.
Experiment 1
I asked ChatGPT about how important wellbeing was for teachers. Within 10 seconds it had produced a 143-word answer that started like this:
Well-being is a big deal as it is closely linked to our overall health, happiness, and quality of life. When teachers look after their well-being, they are more likely to be engaged and motivated in their work, and they are also more likely to be able to manage stress and challenges in a healthy way.
But was that going to convince anyone? I decided to post it on my LinkedIn profile with an image I’d grabbed from Google and no hashtags. The whole process took less than a minute. Within 48 hours it had received 519 impressions and seven reactions. Nobody suggested that the post was anything out of the ordinary. I’m not prolific on LinkedIn so these impressions were about average; but it made me wonder if I could improve.
Experiment 2
This time I asked ChatGPT why inclusion was important in schools. Here, ChatGPT hit its first bump in the road. It described helping ‘students with disabilities’; not quite the tone or diction most teachers would use, so I changed that to ‘students with learning challenges’. I also added my own intro to set the post up, and then added some hashtags at the end. Everything else was down to ChatGPT. Again, it sounded quite convincing:
Inclusion in schools means educating students with learning challenges in regular classrooms, alongside their peers, to the fullest extent possible. There are several reasons why inclusion in schools is essential:
It promotes social and academic benefits: Students who are included in regular classrooms have the opportunity to interact with and learn from everyone else. This can help them develop important social skills, as well as benefiting academically from the diverse perspectives and experiences of their classmates.
Within 48 hours this post had received 1,536 impressions, 19 reactions and three comments. Admittedly, one comment did point out the definition of inclusion was limited (which I agree with) but there were no other dissenting voices.
At this point I started to feel guilty. Experienced teachers were responding to a post that was written almost entirely by a chatbot. To be fair, the chatbot made some good points that were worth supporting, but it was worrying to see how easily professionals could be duped. I felt bad that people were reacting to one of my posts knowing that it was all down to AI. But the question is would some students feel just as guilty?
What does it mean?
Of course, there are many other factors that could have influenced this. Is inclusion a hot topic that people simply want to engage with and discuss? Did the hashtags and my intro help draw people into the post? Did the catchy little image at the end help?
This experiment obviously has limitations, but it does show that ChatGPT can produce content that is passable and seemingly authentic.
At the moment, ChatGPT is far from perfect. At times it will produce complete gibberish, it can be pretty verbose, it’s hopeless at citing references and doesn’t ask for clarification (so can get the wrong end of the stick). However, it is learning, and learning fast.
The dangers are obvious. If students start using this for coursework, such as that used for IGCSEs or the IB Diploma, it becomes a huge headache. I’ve already seen online forums where students have tried to create part of a Theory of Knowledge essay using this. The usual plagiarism detector Turnitin, is useless to catch examples like this because the chatbot is creating original content. That means it’s going to be very hard to fight technology with more technology in order to expose problems.
What are the next steps?
AI may be good, but there are still plenty of ways to fight back against this rather blatant form of cheating. Firstly, many international school students do not possess good enough language skills to produce the kind of grammatical structures and syntax that the chatbot can come up with.
The folk who came up with this did at least also develop a GPT Detector (https://huggingface.co/openai-detector/). That does feel like someone opening the gates for a horse to run free and then calling after it to return when it’s happily cantering down the road.
Even this detector has its faults too – but I won’t detail these here in case it gives someone ideas.
There are some ways to combat this, but it is an ever-changing landscape where more ways will be needed very soon. For now, you can:
-
Run the question you set students through GPTChat. You won’t get the same answer each time, but it will give you an idea about what it’s ‘thinking’.
-
Check the work against the student’s usual level. Does it have a similar tone and voice?
-
Sit down with the student and ask them to explain specific points, or even just vocabulary.
At the moment, GPTChat is probably not good enough to produce IB-level content. But that doesn’t mean students won’t see if it can. If teachers now have to have one eye on all student work in case it comes from an AI source, it’s going to place a huge extra burden on educators.
Ultimately, the answer may be in having a strong Academic Integrity policy that creates an understanding in schools that student work must be truly original, in every sense. If schools can do this, then students will appreciate the value and importance of creating work that they are proud to say is their own. This kind of culture of excellence is hard to achieve and takes considerable monitoring and setting up, but it is achievable.
We may not catch everything, but we need to try as the stakes are high and the consequences are serious for students who take digital shortcuts. Teachers are constantly fighting an uphill battle against technology and plagiarism, and it seems the battleground just became even steeper.
