Case Overview
Can universities create question papers and assignments without the manual effort of going through the text books, and increase student engagement?
We made it happen.
ABOUT THE CLIENT
Renowned South East Asian University
BUSINESS CHALLENGE
Online learning platforms, universities and other learning centers need a way to automatically generate assignments/exam questions from textbook. This can help instructors efficiently generate assessments and provide a more engaging learning experience for students.
SOLUTION OVERVIEW
We developed a system to fine-tune a large language model (LLM) to generate relevant question-answer pairs from a given text source, such as a university textbook PDF.
- Data Preparation: The textbook PDF content was extracted, cleaned, and segmented into chunks of approximately 2000 words each. We tasked ChatGPT to generate question-answer pairs from these chunks, resulting in a dataset of 1800 pairs.
- Model Selection: After trying smaller models like Google T5 and GPT-2, we settled on the TinyLlama model with 1.1 billion parameters, as it showed better performance for our use case.
- Training Workflow: TinyLlama was fine-tuned on the question-answer dataset for 10 epochs, with checkpointing to capture model states at different stages. This aimed to improve the model's understanding of the textbook content and its ability to generate accurate question-answer pairs.
- Inference: The fine-tuned model can generate responses to text prompts (e.g., textbook sections) by formatting the input, specifying generation parameters, and decoding the output to obtain the final question-answer pairs. TinyLlama Model at Huggingface
IMPACT
1. Efficient Assessment Creation: Instructors can automatically generate relevant assessment questions from textbook content, saving time and effort.
2. Improved Learning Experience: Students receive customized and engaging assessments tailored to the course material, enhancing their learning outcomes.
3. Scalability: The automated system can scale to handle large volumes of textbook content and generate assessments for numerous courses and subjects.
4. Continuous Improvement: The model can be further fine-tuned on additional data, improving its performance over time.
By leveraging large language models and fine-tuning techniques, this solution streamlines the assessment creation process for online learning platforms, ultimately benefiting both instructors and students.
Output/Process screens
AT A GLANCE
CHALLENGE
Online learning platforms require automated tools to generate assignment and exam questions from textbook content, streamlining assessment creation for instructors and enhancing student engagement.
IMPACT
A fine-tuned LLM custom built for the specific use-case of being able to generate question answer pairs.
This LLM represents a professor's assistant that reduces effort for the professor teaching the subject and rely on the LLM to generate questions that allow better assessment of the students.
This LLM represents a professor's assistant that reduces effort for the professor teaching the subject and rely on the LLM to generate questions that allow better assessment of the students.
