Share Email Print

Proceedings Paper

ULearn: understanding and reacting to student frustration using deep learning, mobile vision and NLP
Author(s): Lynne Grewe; Chengzhi Hu
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

ULearn is a system that uses deep learning, computer vision and NLP to assist students with the task of web-based learning. ULearn’s goal is to detect when the student is experiencing higher levels of frustration and then present alternative meaningful alternative content. The ULearn app features a web-brower, though the intention is to have ULearn assist students in learning scenarios it is equally applicable to other web-based tasks. While the user is browsing, ULearn monitors them using the front-facing camera and when negative emotions are detected the user is presented with a set of “tips”. The first step in ULearn is to perform face detection which returns an ROI that is fed into an emotion detection system. A deep-learning CNN is used to perform the emotion detection yielding one of anger, fear, disgust, surprise, neutral, and happy. If a significate negative emotion is detected ULearn generates a set of alternative content called “tips” which are a set of links to similar content web pages to the current one being viewed. These links can be found through scraping the currently viewed web page for content that is used directly in a search or first passing this information to an NLP stage. The NLP stage gives the saliency of the most prominent entities in the current web page content. Real test results are given, and the success and challenges faced by ULearn are presented along with future avenues of work.

Paper Details

Date Published: 7 May 2019
PDF: 13 pages
Proc. SPIE 11018, Signal Processing, Sensor/Information Fusion, and Target Recognition XXVIII, 110180W (7 May 2019); doi: 10.1117/12.2518262
Show Author Affiliations
Lynne Grewe, California State Univ., East Bay (United States)
Chengzhi Hu, California State Univ., East Bay (United States)

Published in SPIE Proceedings Vol. 11018:
Signal Processing, Sensor/Information Fusion, and Target Recognition XXVIII
Ivan Kadar; Erik P. Blasch; Lynne L. Grewe, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?