How do admins handle disputes from students about their Panopto viewing statistics?
Andrew Borts
Tyro
Hi everybody,
At UNLV, we've had a recent uptick of students who say they watched certain Panopto videos they were assigned to watch. We have instructors who are assigning grades to students based on how much of the videos they watch. However, instructors are assigning them grades of "0" because the statistics are showing that they didn't watch or complete them.
We haven't been able to replicate any disputes yet, though. No matter what device we use, or how we watch the same content the students are asked to watch, Panopto logs our number of views and minutes delivered perfectly.
Does anyone in higher ed have a blanket type response for these cases, or specific information you ask from the students to supplement their dispute? We've basically said "your instructor has the same stats we do, so it's up to them to interpret the stats and assign you a grade".
Also, can anyone from Panopto speak to the accuracy of the analytics? We feel like a lot of our students are trying to say "I watched it" to their instructor ,(even though the stats say they didn't), and then they call IT to see if they get a different answer. Kind of like asking mom for ice cream and, if she says no, going right to dad. :) Thank you all for any feedback and help!!
Sincerely,
Andy Borts, Senior Instructional Technology Support Specialist, UNLV
Answers
Hi Andy,
At the end of January, we had claims from two students that they had watched Panopto videos but that statistics were not recorded. Here's the information from the Panopto Support Case (00820535) that I opened:
"Student using a MacBook Air running macOS Catalina version 10.15.1 and using Safari 13.0.3 (15608.3.10.1.4) watched three Panopto videos; however they did not appear at all in the 'Top Users" report.
I visually saw this with my own eyes. I had the student in a Zoom session and the video was playing, but the Top Users did not show anything! I then had the user switch to Google Chrome (a recent version 97.??) and they went through the same steps to watch the same video. Their name and data then showed up in the Panopto stats. So it appears there is an issue with outdated browsers recording Panopto viewing statistics."
The second student: "was using macOS (10.13.4) and Chrome (79.0.3945.130). The same issue was occurring. "
After 4.5 months of going back and forth with support, the main relevant comments from Panopto Support are:
Panopto Support: "Loss of viewer statistics data is considered a serious issue, and I've escalated your case to our developers for further review. I'll let you know as soon as I have more information from them."
Panopto Support: "I've heard back from Dev. According to them, the expected behavior any time stats can't be gathered is for the viewer to receive a popup warning. Do you know (or can you find out) if either of these users received any kind of message indicating there was a problem when viewing in the older browsers?"
Me: No, the users never received any kind of message or warning. I asked what did the warning message supposed to look like.
Panopto Support: "... same warning/error as seen in this article: https://support.panopto.com/s/article/Why-Am-I-Seeing-an-Extension-Warning-in-Panopto"
Panopto Support: "I've just confirmed that the warning dialog is not displayed on either of those browser versions. In fact, while I'm able to play recordings if I go directly to them, I'm not able to fully load any other portion of the website in either of them. The site partially loads, but won't list any recordings, or allow me to access most of the pages on the site. I'm going to re-escalate this to our developers as I believe I now have the answers to their questions."
Panopto Support: "Thank you for your patience while we investigated this. As you noted, the affected users were using outdated versions of Safari/Chrome. Our developers have confirmed this is the reason for the stats not being logged. They have committed to handling this better in the future (likely by disabling the viewer for unsupported browsers) however at this time there is no ETA for when this might be done. They will continue to track the issue in their internal system. At this time I am provisionally closing the case. "
Me: So currently, how do end-users know that their statistics are NOT being recorded? Can Panopto provide us a report of our users that are accessing our Panopto system with outdated browsers that would cause their stats to not be recorded?
Panopto Support: "Outdated browser versions prevent the site from loading completely, including the logging and statistics portions of the code, as well as the bit that checks compatibility and provides an error.. I'll look into whether it's possible to provide a report of users who are using outdated browsers, however I suspect that data isn't recorded anywhere due to these same limitations."
Panopto Support: "Thank you for your patience while I dug deeper into this. Unfortunately my previous suspicion was correct; We do not log the browser version anywhere that users on an outdated version would have their data collected. The only site activity which is recorded for those users is their login, which doesn't record the browser version. Users who are on an outdated browser version will have limited access to the site. Any pages with a list of recordings will fail to load completely, and even pages that do load will have silent errors in the browser's console log. As I previously stated, an internal work item has been filed by our developers to handle this scenario better, and warn users, when an unsupported browser version is detected. "
------------------------------------------------
I know that's long, but hopefully something in it was helpful to you!
Thanks,
Jeffrey Ries, Senior eLearning System Administrator, University of Northern Iowa