ContextVis: Envision Contextual Learning and Interaction with Generative Models
Accepted to the 26th International Conference on Human-Computer Interaction (HCII'24) (Paper).
Accepted for Oral Presentation and awarded full scholarship at the UAE Graduate Students Research Conference 2024 (UAE GSRC'24).
Abstract
ContextVis introduces a workflow by integrating generative models to create contextual learning materials. It aims to boost knowledge acquisition through the creation of resources with contextual cues. A case study on vocabulary learning demonstrates the effectiveness of generative models in developing educational resources that enrich language understanding and aid memory retention. The system combines an easy-to-use Dashboard for educators with an interactive Playground for learners, establishing a unified platform for content creation and interaction. Future work may expand to include a wider range of generative models, media formats, and customization features for educators.
The ContextVis System
We present the ContextVis system, which leverages generative AI to develop contextual learning materials and experiences. The workflow comprises three essential components: the back-end platform for processing user input and generating contextual data using Large Language Models (LLMs), as well as generating multi-modal assets through generative models; the database, where the metadata and generated resources are stored; and the front-end platform, featuring a Dashboard for educators and a Playground for learners. It is worth noting that our vision for this system encompasses the ability to receive inputs and generate outputs in multiple media formats, as the models employed can be substituted to accommodate diverse tasks.
Case Study: Contextual Vocabulary Learning
Acquiring English vocabulary can be challenging due to its inherent complexity characterized by contextual nuances, semantic diversity, and an extensive lexicon. Understanding and memorizing grammar rules alongside vocabulary demands a coherent and persistent learning strategy, often requiring prolonged exposure to a variety of linguistic stimuli. This extended engagement with both language inputs and outputs is crucial for reinforcing linguistic knowledge.
The process of vocabulary acquisition can be effectively integrated into the proposed workflow, where the integration of contextual factors plays a crucial role in fostering language development. To this end, we conducted a case study for vocabulary learning which exemplifies the significance of the ContextVis system. In the workflow, with selected vocabulary in a unit and an optional theme input in the Dashboard, the generative models in the back-end automatically generates a contextually coherent a story script and stickers for each word as output. The generated data and assets are stored in the database on the server for learners to access and explore in the Playground.
Dashboard for Educators
To facilitate seamless integration of vocabulary with generative contents, we developed a Dashboard for educators, with multiple panels allowing educators to input and select the vocabulary, then generate, preview and refine the generated educational resources. By assigning a theme as contextual cue, the generated assets of scripts and stickers are stored in the server and can be exported as teaching materials for classroom use. The Dashboard serves as a pivotal interface within the ContextVis system, offering user-friendly tools and features that empower educators to effectively create tailored educational content.
Playground for Learners
In addition to the generated learning contents, the Playground serves as a platform for learners to engage in interactive exploration. This interactive space offers learners the opportunity to actively review with the concepts and vocabulary introduced in the generated content and to explore more by themselves. It consists of four panels: Vocabulary, Script, Selected and the interactive visualization in the center. The Playground is seamlessly synchronized with the Dashboard as the two platforms are interconnected by the data and assets they share in the database on the server. This integration ensures that the learning materials sophisticatedly prepared by educators are available for learners to review and explore extra contents aligning with the theme of the original materials.
Discussion
In this article, we presented the ContextVis system, a workflow that leverages generative models for creating contextual learning materials and experiences. Through a case study on contextual vocabulary learning, we have demonstrated the utility of generative models in resource creation and the effectiveness of the system in fostering language development.
The user-friendly Dashboard for educators and an interactive Playground for learners together provide a comprehensive platform for both content creation and engagement. The integration of generative models, contextual cues, and interactive visualizations offers unique opportunities for learners to understand, explore, and retain knowledge. During the development stage, we found that providing competent information such as theme and context to generative models is essential to generate high-quality learning materials.
Future directions for the ContextVis system include expanding the range of generative models and media formats, as well as more in-depth customization options for educators. Overall, the ContextVis system represents a promising approach to enhance contextual and exploratory learning using generative models and interactive visualizations.