Once again, the Provost's Office has provided funding through the Learning Analytics Fellowship program to faculty and staff for the 2025-26 academic year. Details of the new fellows’ proposals are provided below:

Sarah Bass(Chemistry) will reprise (and dive deeper into) a Spring 25 DoIT & FDC panel presentation on her use of NotebookLM to develop "Virtual Prof. Bass," an AI-driven tool that guides students in her CHEM 101 course to proactively engage with course materials. Specifically, she will focus on at-risk students she studied further this summer in Carnegie Mellon’s LearnLab: Compared to students who take the CHEM 101 & 102 sequence in Fall and Spring, “off-track” students who do the opposite tend to underperform on standardized American Chemical Society (ACS) concept inventory exam questions embedded in her unit, midterm, and final exams. In addition to her 24/7 “Virtual Prof Bass,” she will market to and support “Off-track” students by offering focused practice and success strategies during the CHEM 101 common exam time on Friday afternoons from 4-6 p.m. Note: In addition to her LA CoP renewal, Bass’ research and practice are supported by a second year of funding from Digital Promise ChemCore, to support faculty use of the REAL Chem courseware developed by Arizona State University and Carnegie Mellon University. 

Neha Raikar(Chemical, Biochemical, and Environmental Engineering) will test whether anonymous grading can reduce bias and student anxiety. The study asks three things: (1) Does hiding names lower real or perceived bias? (2) Does the effect change by assignment type—auto-graded vs. instructor-graded, multiple-choice vs. free-response? (3) Does anonymity help students take risks, stay engaged, and continue in the course? Assignments will be randomly split into two groups: one graded with names, one without. Graders will use the same scoring guides and check that their scores match, and the team will compare results with basic statistics. Short interviews will gauge the time needed, benefits, costs, and any side effects. Students will take brief surveys before and after assessments about fairness and why they feel that way. Reports will share group trends only, not names. The expected result is fairer grading, higher student confidence, and clearer insight into what shapes scores.

Liang Zhu (Mechanical Engineering) is using developed conceptual questions at the start and end of an undergraduate heat transfer course to see where students begin strong, where they struggle, and how their understanding evolves across key course concepts. Keeping the same instructor ensures a consistent view, and early results show noticeable gains. The team will study patterns in the answers to spot common stumbling blocks, judge whether any questions are unclear or too easy, and confirm that the set fairly represents the course. Those insights will drive concrete improvements: clarifying or replacing weak questions, adding timely examples and practice where misconceptions persist, and fine-tuning pacing and emphasis. The question set will be refreshed into a balanced collection that better distinguishes levels of understanding and yields clearer feedback for teaching. The revised questions will be tried in a future offering of the course and accompanied by a brief student survey to gather perceptions and suggestions. Results will be shared with the funding office and at local education venues, align with departmental goals for cost-effective, evidence-based teaching, and inform similar updates in related courses with potential to scale more broadly.

These one-year learning analytics awards are renewable pending receipt of a final report, paper submitted for publication or conference presentation. In addition to use of UMBC’s Report Exchange (REX) data warehouse, and a Tableau “viewer” license, faculty recipients can consult with staff from Analytics and Business Intelligence, Instructional Technology and Institutional Research and Decision support (IRADS). 

For more information about this year’s workshops, speakers and another learning analytics mini-grant call for proposals to be announced in Spring 26, please visit doit.umbc.edu/analytics/community. 

By John Fritz & Tom Penniston

As part of UMBC’s data warehouse migration to HelioCampus, the Data Management Committee’s “Training & Outreach” working group has published a brief (8 min) video orientation on why and how to explore analytics @ UMBC. 

Developed, in part, as a response to the 2006 Middle States Self Study, UMBC’s Report Exchange (REX) data warehouse was designed to help collect, integrate, report on, analyze and (importantly) act on data from a variety of campus systems such as Blackboard, PeopleSoft, and myUMBC. Not surprisingly, the current 2025-26 Middle States Self Study and UMBC 3.0 Strategic Planning initiative will also focus on continuous improvement through curated, campus-wide data to inform decision making.

In addition to the migration to HelioCampus, which constitutes REX 2.0, the orientation video also describes:

• New and (likely) soon-to-be-required training on the Family Educational Rights and Privacy Act (FERPA) to access campus systems like PeopleSoft and REX itself.

• How to find and use free training videos about Tableau’s “viewer” role, which will be used to access predefined (“canned”) reports largely migrated from REX 1.0 that used Microsoft Reporting Services (RS) reports.

• A new “Analytics @ UMBC” Blackboard community where anyone can learn more about REX, “actionable intelligence,” and (eventually) even do some hands-on, self-paced data analysis with dummy data and AI.

• How to get involved with UMBC’s analytics community of practice, including the REX developers’ and users’ groups.

Earlier this semester, Taylor & Francis published Data Analytics & Adaptive Learning: Research Perspectives (2023), which includes a UMBC case study about Dr. Tara Carpenter's use of "spaced practice" in her high-enrollment, introductory course in general chemistry. 

In "Banking on adaptive questions to nudge student responsibility for learning in general chemistry," Carpenter and DoIT staff co-authors John Fritz and Tom Penniston describe Carpenter's course redesign efforts using learning science, analytics, and adaptive learning technology to nudge students into deeper, long-term learning, application and retention instead of cramming and short-term memorization to prepare for her exams.

The campus community can view the chapter here (UMBC account login req'd) or view Carpenter's 2022 presentation of related findings as part of the Learning Analytics Community of Practice.

By John Fritz

DoIT has a new addition to our suite of data visualizations supporting course engagement/instruction and teaching: The Blackboard course "waterfall" report (UMBC vpn req'd if using off-campus). 

As illustrated below, the waterfall dashboard summarizes student interactions within our Blackboard Learning Management System (LMS) on a week-by-week basis. Each row represents a student and each column represents a week in the semester, with the density of an individual cell color representing the student's time spent in the Blackboard course that week. Each course waterfall is organized by the student's final course grade on the left.

To encourage, identify, and share effective practices in the use of data to inform teaching and learning, the Provost's Office has once again awarded learning analytics mini grants to six faculty and two staff for the 2023-24 academic year (the three new grants and three renewals are described below):

Katie Birger (Public Health): "Analyzing Factors Affecting Student Success in Public Health Courses." This new proposal aims to standardize PBHL 300: Research Methods in Health, to address observed inconsistencies that potentially impact students' success in the subsequent PBHL 420 course (Epidemiology). Birger plans to investigate these discrepancies by analyzing data from Blackboard sections of both courses, to identify variables such as instructor differences, textbook usage, attendance, homework submission, and access to Panopto recordings that might affect student success rates. The findings will help standardize effective learning activities and enhance the learning assessment plan. To do so, Birger aims to implement and test evidence-based curriculum changes in PBHL 300 and then monitor student performance in PBHL 420. The overall goal is to improve student knowledge and skills in public health, better preparing them for the workforce or graduate education and enhancing the continuous improvement of the Public Health curriculum.

Joby Taylorand Emily Passera (Shriver Center): "Knowing the student, knowing the staff: How a reciprocal relationship alongside data-informed insights improves student outcomes in community engaged learning."This new proposal will delve into the Shriver Center's student data to help improve practitioners' understanding of what interventions and environments best scaffold success. Additionally, the researchers will investigate the potential benefits of incentivization through micro-credentialing. Through these approaches, Taylor and Passera hope to support both student growth and improve their pathways to academic success.

Rebecca Williams(CS/EE): "Improving Student Visual Literacy & Critical Thinking in CMSC 436/636 (Data Visualization)."This new proposal focusing on course redesign as an intervention to address student misconceptions about visualizations, such as the misunderstanding that simple, easy-to-read, charts are always effective and the belief that data visualizations are entirely objective. Williams' proposed interventions include having students search for, annotate, and critique visualization examples, to supplement those she provides (and models) herself. As such, students are more likely to develop these skills and the process is likely to lead to the long-term retention of these same skills.

Sarah Bass and Tara Carpenter(Chemistry): "Nudging CHEM 101 Students with myUMBC Personal Posts."This is a renewal proposal building off of Carpenter's prior efforts to advance student metacognition and responsibility for learning as well as conducting focus groups about student motivation in moving from general to organic chemistry. Specifically, Bass and Carpenter will use myUMBC's "personal post" functionality to nudge students in their initial exposure to and use of course resources for practice and exam preparation in CHEM 101 "Principles of Chemistry I," which they are co-teaching this semester. DoIT staff have also worked with Math and Psychology faculty to use myUMBC personal posts in this way to scale course-based nudging of students. As such, Bass and Carpenter will customize messaging focused on nudging students about their engagement and preparation for key course milestones and assessments.

Karen Chen(Information Systems): "Learning Analytics by Students for Students." In this renewal proposal building on her 9/30/22 LA community of practice presentation of the same name, Chen intends to further explore student agency and literacy in the development and use of learning analytics. She's particularly interested in leveraging de-identified UMBC data to inform student analytics projects in two data science courses, IS 428 "Data Mining Techniques and Applications" and IS 733 "Data Mining" in Fall 2023. These include the use of "Spaced Practice" in General Chemistry courses as well as student use of and anonymous feedback about the myUMBC Check My Activity (CMA) feedback tool for students.

Cody Goolsby-Cole(Physics): "How Does Student Performance on Ungraded Practice Questions Correlate to Performance on Exams?"In this renewal proposal, Goolsby-Cole seeks to increase student use of a growing CNMS practice of using online questions banks to prepare for high-stakes exams. Specifically, he has used ungraded practice questions, in terms of points contributing to a final grade, but including correct and incorrect answers displayed to students. Students in his PHYS 122 "Introductory Physics II," who use the practice questions clearly perform better on unit, midterm and final exams, but he'd like to see more students do so. He plans to continue development of his practice questions, but also do more proactive nudging with myUMBC personal posts (similar to Carpenter and Bass in Chemistry described above) to see if and how more students might use and benefit from the practice question bank.

"On behalf of Interim Provost Dauwalder, we're pleased to continue offering these learning analytics mini grants to help faculty use institutional data that might inform their course designs and instruction to the benefit of our students," says Robert Carpenter, associate provost and deputy CIO. "We are excited by these interesting projects and look forward to what the faculty may find."

These one-year learning analytics awards are renewable pending receipt of a final report, paper submitted for publication or conference presentation. In addition to use of UMBC's Report Exchange (REX) data warehouse, and a Tableau "viewer" license, faculty recipients can consult with staff from Analytics and Business Intelligence, Instructional Technology and Institutional Research and Decision support (IRADS). 

For more information about this year's workshops, speakers and another learning analytics mini-grant call for proposals to be announced in Spring 24, please visit doit.umbc.edu/analytics/community.

By John Fritz

• Image "Alt" tag: "2023-24 UMBC Learning Analytics Mini Grant Recipients Katie Birger, Emily Passera, Joby Taylor, Rebecca Williams, Sarah Bass, Tara Carpenter, Cody Goolsby-Cole, and Karen Chen"

DoIT's Coordinator of Learning Analytics, Tom Penniston, recently presented his paper (UMBC login req'd), Toward a New Paradigm: Learning Analytics 2.0, at the 25th International Conference on Human-Computer Interaction, HCI International 2023, in Copenhagen, Denmark. The conference focused on adapting design to meet the needs of our evolving world, from the rise of deep fakes and battling the spread of extremist propaganda, to the legal and social implications of large language model AI, to supporting the needs of the neuro- and ability- diverse populations we serve. 

Penniston's paper analyzed ongoing UMBC efforts to leverage predictive machine learning models to inform empathetic behavioral nudges to support student success. He argues that through intentional, rigorous course design we can align intervention strategies with direct measures of learning at scale and thereby best support our students' individualized needs, while simultaneously improving institutional measures of success, such as reduced DFW and 6-year graduation rates.

Attending the conference was a great professional opportunity for Penniston, who was able to represent his department and the university, learn more from experts within the community, and network with peers. "It's really amazing and humbling to be surrounded at work by folks who encourage and support me to pursue these kinds of opportunities!" said Penniston.

The success of students in a Blackboard course is often dependent on their level of engagement and their ability to stay on track with course content and assignments. As an instructor, it's crucial to have the tools and resources to monitor students' progress and ensure they are staying engaged and motivated throughout the course.

An Ultra course provides progress tracking tools to help instructors monitor how much time students are spending in the course, what content they are engaging with, and what their current grades are. This information is invaluable in helping instructors reach out to students who may be struggling or falling behind, and it also helps in retaining students who might otherwise drop out of the course. Progress Tracking is enabled by default in all Ultra courses as of SU2023.

One of the key benefits of tracking students progress is the ability to identify potential issues early on. If a student is not engaging with course content as much as they should be, the instructor can reach out and provide support, resources, or motivation to help the student get back on track. This proactive approach can help students stay engaged and motivated, and it also helps them to understand the importance of their participation and progress in the course.