To encourage, identify, and exchange effective practice in the use of data to inform teaching and learning, the Provost's Office has once again awarded learning analytics mini grants to three faculty for the 2022-23 academic year (the three faculty and their proposed topics are included below):

Tara Carpenter (Chemistry): "Modeling Effective Learning Strategies in Introductory Chemistry." This is a renewal of Carpenter's 21-22 mini grant, focused on students' understanding and use of "spaced practice" as an alternative to cramming for exams in her CHEM 102 "Principles of Chemistry II" course. Specifically, she is focused on how students can take their spaced practice "lessons learned" with them to CHEM 351 "Organic Chemistry," which requires CHEM 102. Carpenter presented her initial findings last spring, and has already offered a "spaced practice" workshop this fall for her Spring 22 CHEM 102 students currently enrolled in CHEM 351. She is also working with the FDC and DoIT to conduct a series of focus groups to better understand the challenges students face in setting up spaced practice on their own.

Karen Chen (Information Systems): "Bumpy Journey: Exploring Gateway Courses Failures and Major Switch - a Case Study with Information Systems Majors." This is a renewal of Chen's 21-22 mini grant focused on what is sometimes referred to "curricular analytics," particularly in CoEIT. She presented her initial findings at Indiana University's 2022 Learning Analytics Summit in May, but also leveraged this work to look at how she teaches data science by engaging students as creators, not just consumers, of learning analytics insights and interventions. Her presentation, "Analytics by Students for Students," was the first in UMBC's Fall 2022 data science and learning analytics series.

Cody Goolsby-Cole (Physics): "How Does Student Performance on Ungraded Practice Questions Correlate to Performance on Exams?"This is a new proposal emerging from Goolsby-Cole's participation in a grant to promote "Equity & Digital Education" by the Every Learner Everywhere Network. Building on a growing CNMS practice of using question banks and calculated question types for online exams in Blackboard, Goolsby-Cole also created practice questions that were ungraded, in terms of points contributing to a final grade, but including correct and incorrect answers displayed to students. During the first six units of his Spring 22 course, PHYS 122 "Introductory Physics II," he found students earning a 70% on the practice questions earned an average of 92% on the exam that followed. Students who did not use practice questions earned an average exam grade of 77%. He plans to build out and assess this practice environment further, including practice exams, to see if and how more students might use and benefit from it.

"On behalf of the Provost, we're pleased to continue offering these learning analytics mini grants to help faculty use institutional data that might inform their course designs and instruction to the benefit of our students," says Robert Carpenter, associate provost and deputy CIO. "We are excited by these interesting projects and look forward to what the faculty may find."

These one-year learning analytics awards are renewable pending receipt of a final report, paper submitted for publication or conference presentation. In addition to use of UMBC's Report Exchange (REX) data warehouse, and a Tableau "viewer" license, faculty recipients can consult with staff from Analytics and Business Intelligence, Instructional Technology and Institutional Research and Decision support (IRADS). 

For more information about this year's workshops, speakers and another learning analytics mini-grant call for proposals to be announced in Spring 23, please visit doit.umbc.edu/analytics/community. 

by John Fritz

For the Fall 2022 semester, DoIT's Analytics & Instructional Technology groups will continue to offer student success and learning analytics-related workshops for the UMBC community. 

All workshops will start at noon, and be available via video conference (the virtual meeting URL be announced on each myUMBC event registration site the morning of the event). To register and view more information, check the links below:

"Learning Analytics By Students For Students?" (9/30, Noon). In this session, Dr. Karen Chen, assistant professor of Information Systems, will make the case for why and how students can and should play a more active role in the design and development of learning analytics to improve student success. 

"Using Analytics to Predict & Improve Sophomore Retention" (10/14, Noon). In this session, student success colleagues Robert Carpenter, Delana Gregg and Len Mancini, will share how UMBC has worked with HelioCampus to build very accurate models that predict second year retention at the beginning of their third semester. In addition to explaining how the models work, key inputs into them, and where to find the results, they will share how we have begun to act on them. 

"Tips & Tricks for Telling Your Story with Data" (11/10, Noon). In this session, Mike Sharkey, an analytics thought leader, teacher, consultant and long-time friend of UMBC, will share his lessons learned using data, narrative and visualizations (including Tableau) to raise awareness and inspire change. 

"Mapping Student Pathways Through the Curriculum" (12/2, Noon). As institutions continue to identify barriers to student success one area to explore is how we engage faculty and departments in using learning analytics to take a fresh look, along with a greater understanding, of student pathways toward graduation. In this talk, Dr. Martha Oakley, Assoc. Vice Provost for Undergraduate Education and professor of chemistry at Indiana University Bloomington, will highlight recent developments by IU's Bloomington Assessment and Research (BAR) team to provide faculty and administrators with process maps that show how students actually navigate through curricula. 

To view prior data science and learning analytics meetings (and recordings), please visit doit.umbc.edu/analytics/community.

~ By John Fritz

I am pleased to announce that Tom Penniston has accepted a new position as DoIT’s Coordinator of Learning Analytics. 

In his new role, Tom will continue and extend his work to leverage institutional, academic, and learning analytics to inform course redesigns and improve student engagement and success. He will collaborate closely with DoIT’s Analytics and Business Intelligence team, the Office of Institutional Research and Decision Support (IRADS), and of course faculty, including recipients of learning analytics “mini-grants” (the next CFP is due 6/3/22), part of the UMBC’s Learning Analytics Community of Practice.

We are also pleased to announce that Tom has been accepted into this year’s competitive Learning Analytics in STEM Education Research (LASER) Institute hosted at NC State. More than 260 applications were received, and successful recipients also receive a $1,500 travel award and stipend. Before the pandemic, Tom had also completed the 2017 “Learn Lab”  summer institute at Carnegie Mellon University.

In addition to writing several analytics-related DoIT News stories, Tom recently co-authored a chapter, “Scaling Course Design as Learning Analytics Variable” (UMBC Box version), and has presented at both the EDUCAUSE and American Educational Research Association (AERA) annual conferences.

Please join me in congratulating Tom on his new role.

John Fritz
Assoc. Vice President, Instructional Technology
Division of Information Technology

The Data Science Team recently released its set of predictions for the 6-year graduation probabilities of the 2020 Freshman cohort. Below is a snapshot of the current cohort's predictions, in blue, with the 2019 Freshman cohort's predictions, in red. 

The vertical axis represents the end of 2nd semester prediction probability; the horizontal axis is the end of the 3rd semester prediction. 

The black diagonal line indicates the point on the graph at which the second semester and third semester predictions are equal. If a dot, representing an individual student, rests on the line there was no change in the student's predicted probability of graduating between their second and third semester. Points above the line mean a student's predicted 6-year graduation probability decreased from semester 2 to semester 3. 

The group of students above the line, particularly in the Moderate and Low probability color bands, are of concern as their predicted probabilities of graduation have declined over their last two completed semesters. 

The 2020 Freshman cohort was the first freshman class during COVID and one of the concerns is the effect COVID may have on student performance in later semesters. From the predictions chart there seems to be reasons for optimism. Looking at the dispersion of points, there is a greater concentration of students in the High and Very High likelihood buckets compared to the 2019 cohort, as shown by the clustering of blue dots in the upper right corner of the chart. 

However, some concern may remain. Students whose predictions fall in the Low and Moderate buckets have experienced sharper declines in predicted likelihood to graduate than students from the 2019 cohort (because the blue dots, 2020 cohort, are much further above the diagonal line than the red dots, 2019 cohort). 

The diagram we've shown in this post is produced by a tool developed by the Data Science Team.  The tool allows staff with broad student success responsibilities to identify groups of students who may need additional support to achieve their goals.  The tool is available at analytics.umbc.edu in the student success section.  Access to it requires permission.  

(NOTE: Please join us Friday, March 18th from 12-1pm, for the fourth in our series of our Spring 2022 Data Science Workshops. This predictive model will be the topic of the session. Go to this link - https://my3.my.umbc.edu/groups/instructional-technology/events/101281 to register)

Len Mancini

Data Scientist and Statistical Specialist

The Analytics, Business Intelligence, and Student Success Technologies Group is part of the Provost;s Office and the Division of Information Technology.  Our component parts include the Business Intelligence/Data Warehouse Team, the Analytics/Data Science Team, the PeopleSoft SA/Salesforce Team, and since late 2021, the myUMBC/Campus Information Portal Team.  

We're very pleased to provide you with our "annual" report (delayed by the pandemic, like so many other things).  The report details selected projects and initiatives from 2019 through mid-year 2021, our goals, and our projects going forward.  As the report is part of our assessment activities, we welcome your comments.

We are grateful to our many collaborators and partners across the campus.  Success is never final, but it is also surely a joint effort.

First-year retention is UMBC’s single biggest point of loss during the lifetime of a freshman cohort and a key focus area of the university’s student success efforts. To support this work the Data Science Team (from DoIT and the Provost's Office) and its campus partners have worked with our HelioCampus to produce the first of a series of predictive analytics reports or “playbooks.” 

The first playbook, just deployed on analytics.umbc.edu, predicts the likelihood of a first-time full-time freshman returning for their second year.  It is being used by the Academic Advocates in Undergraduate Academic Affairs to help identify students who may be at risk of not returning next fall. (Access to the playbook is role-based.)

On February 25th, at noon, the REX Analytics Group will meet to explore the factors that make these predictions change the most, how to read the dashboards, and how to interpret and use the playbook.  Please join us! Contact lmancini@umbc.edu or kjoseph@umbc.edu for meeting details.

Len Mancini

Data Scientist and Statistical Specialist