AI Insights for Independent schools,
July 2024

As an international consultant, author and speaker with over 15 years of experience in education, I’m excited to share my insights on the rapidly evolving world of Generative AI in K-12 education. I am currently pursuing my PhD on the implications of GenAI on writing instruction and education, and have worked extensively with Independent Schools in Victoria and across Australia in 2023 and 2024.

The purpose of this monthly newsletter is to keep you informed about the latest developments in GenAI, provide practical resources for implementation in your classrooms, and offer guidance on the ethical considerations that come with this transformative technology. Whether you’re a technology early adopter or just beginning to explore AI in education, this newsletter aims to provide valuable insights for all educators in Independent schools.

Each month, we’ll explore new tools, share effective prompts and methods, discuss recent research and highlight professional development opportunities. We’ll also address the challenges and opportunities that GenAI presents, always with a focus on enhancing teaching and learning in our schools.

In this issue

• Spotlight: The AI Assessment Scale (AIAS)
• Application of the month: Claude 3.5 Sonnet
• 3 levels of prompting
• Research roundup: AI detection tools
• Upcoming professional development
• AI Leader Network
• Q&A

This month, we’re putting the spotlight on a tool that’s gaining traction in both K-12 and tertiary education: the AI Assessment Scale.

Developed by Dr. Mike Perkins, Dr. Jasper Roe, Associate Professor Jason MacVaugh, and Leon Furze, the AI Assessment Scale (AIAS) offers a structured approach to incorporating AI into assessment practices. This scale provides a framework for educators to thoughtfully integrate AI into their assessment strategies, moving beyond the binary of ‘use AI’ or ‘don’t use AI’.

The AIAS consists of five levels:

1. No AI: Traditional assessment without AI involvement
2. AI Assisted Brainstorming: Using AI for idea generation
3. AI Editing: AI for refining and polishing work
4. AI + Human Evaluation: Working alongside AI throughout the process
5. Full AI: AI generates the entire response, or significant parts of the whole

By using this scale, educators can design assessments that gradually introduce AI, allowing students to develop critical AI literacy skills while maintaining academic integrity.

Implementing the AIAS in your school:

1. Start with faculty discussions about the role of AI in assessment
2. Redesign assignments to align with different levels of the scale
3. Clearly communicate expectations to students for each assessment
4. Provide guidance on ethical AI use at each level
5. Regularly review and adjust your approach based on outcomes

Beaconhills College has successfully implemented the AIAS as part of their recent published school GenAI Guidelines. They’ve encouraged student engagement through focus groups, as well as staff professional development focused on the AI Assessment Scale and the need to rethink assessment for Generative AI.

Beaconhills College also invited parents to an online evening information session where Leon Furze discussed Generative AI, including how the AI Assessment Scale will be used to support academic integrity.

Perkins, Furze, Roe and MacVaugh have created a Custom GPT using OpenAI’s GPT-4o which allows users to interact with the AIAS. The custom chatbot contains all of the academic publications related to the AIAS, as well as a number of additional resources, examples, of the AIAS in action across different subject areas, and support materials.

You can ask the AIAS GPT questions about the Scale, or ask it to develop assessment tasks aligned to any of the five levels. You can also upload documents and PDFs such as existing assessment tasks, and have the AIAS GPT make recommendations based on the Scale and the research.

Back to Top

This month, we’re also spotlighting a tool I’ve been particularly excited about: Anthropic’s Claude 3.5 Sonnet. As an AI assistant, Claude 3.5 Sonnet offers some impressive capabilities that can be particularly useful in educational settings.

• Free: Chats, image analysis and the new Artifacts feature are available in the free version.
• Paid: The Projects feature, and much higher rates (the number of messages you can send during a chat).

While Claude 3.5 Sonnet is a powerful tool, it’s crucial to use it responsibly in educational settings. Encourage students to use it as a learning aid rather than a replacement for their own critical thinking. Always verify information provided by AI, and use it in conjunction with human expertise and judgement.

In my own work, I’ve found Claude 3.5 Sonnet particularly useful for generating initial drafts of content, which I then heavily edit and refine. It’s also excellent for brainstorming and exploring different perspectives on complex topics.

Remember, the key to effectively using AI tools like Claude in education is to view them as supplements to, not replacements for, human teaching and learning. In our next issue, we’ll explore another tool that could be useful for both teachers and students.

Back to Top

Recently I wrote an article which looked at 3 levels of prompting for Generative AI, from the most basic to the most complex. As we get more familiar with these systems, I’m noticing a lot of educators move beyond simple prompts. Sometimes, a short, simple prompt in GPT or another model is all you need, but often you will need to add more details and contextual material to get what you want.

Based on that recent blog post, here are three levels of prompting for lesson planning using Generative AI:

Remember, the goal is to augment your expertise, not replace it. Always review and adapt AI-generated content to ensure it meets your students’ needs and aligns with your teaching philosophy.

Back to Top

A recent study by Perkins et al. (2024) investigated the effectiveness of AI text detectors in identifying machine-generated content, particularly when ‘adversarial techniques’ are applied to evade detection. The research, which involved testing 805 samples across six major Generative AI (GenAI) text detectors, revealed significant limitations in the accuracy and reliability of these tools.

1. Low baseline accuracy: Even without manipulation, AI detectors demonstrated a relatively low average accuracy rate of 39.5% in identifying AI-generated content (Perkins et al., 2024).
2. Reduced accuracy with adversarial techniques: When adversarial techniques were applied to AI-generated text, the average accuracy of detectors dropped further to 22.14%, representing a 17.4% reduction in accuracy (Perkins et al., 2024).
3. Variation in detector performance: The study found considerable differences in the performance of various AI detectors, with accuracy rates ranging from 4.5% to 58.7% when faced with manipulated content (Perkins et al., 2024).
4. Effectiveness of evasion techniques: Some adversarial techniques, such as adding spelling errors and increasing textual “burstiness,” proved highly effective in evading detection (Perkins et al., 2024).

The findings of this study raise important concerns about the use of AI detection tools in academic settings. Previous research by Liang et al. (2023), as cited in Perkins et al. (2024), highlighted potential biases against non-native English speakers in AI detection systems. This issue, combined with the low accuracy rates and vulnerability to evasion techniques demonstrated in the current study, suggests that these tools may create barriers to inclusive assessment rather than promote academic integrity.

Furthermore, the study by Weber-Wulff et al. (2023), also referenced in Perkins et al. (2024), corroborates the findings on the susceptibility of AI detectors to various manipulation techniques, including machine translation and paraphrasing.

The results of this study underscore the need for a cautious and critical approach to implementing AI detection tools in education. The potential for false accusations and the ease with which these tools can be manipulated raise serious questions about their reliability in maintaining academic integrity. As Perkins et al. (2024) suggest, while these tools may have a role in supporting student learning when used non-punitively, they cannot be recommended for determining academic integrity violations.

Educators and institutions should consider alternative approaches to assessment that recognise the growing ubiquity of GenAI tools while ensuring fairness and inclusivity. This may involve rethinking traditional assessment methods and developing strategies that integrate AI tools into learning and assessment processes in ethical and transparent ways.

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. arXiv. http://arxiv.org/abs/2304.02819

Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024). GenAI Detection Tools, Adversarial Techniques and Implications for Inclusivity in Higher Education. arXiv:2403.19148 [cs.CY]. https://doi.org/10.48550/arXiv.2403.19148

Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1). https://doi.org/10.1007/s40979-023-00146-z

Back to Top

Back to Top

Independent Schools Victoria’s Artificial Intelligence (AI) Leader Network is designed to foster collaboration and communication among those with key oversight of AI at their school.

The network serves as a forum for leaders to come together, share insights and work towards enhancing the educational experience for all school communities.

Key takeaways:

• boost your leadership vision
• fresh and effective perspectives and ideas
• connection with fellow AI leaders
• actionable ideas to implement right away.

The first meeting will be held online on Wednesday 11 September, from 3.45 pm to 4.45 pm. ISV Member Schools only.

Back to Top

Every month we’ll be inviting Independent school teachers and leaders to ask questions, which Leon will (try to) answer in the following issue. To send your questions, requests, or feedback, please contact Leon at leonfurze@gmail.com.

Stay tuned for the next edition in ISV’s Latest in Learning newsletter.

Back to Top