I am part of the Learning Analytics team responsible for Blackboard Analytics at Curtin University in Perth, Western Australia. We have recently done a deep-dive into the data to investigate some issues that we have had with activity not being tracked on some items. I have made a summary of these issues for our staff to be aware of, but as these issues likely impact other customers of Blackboard Analytics, I thought I’d share with the community as well.
These issues are a priority for us, as it can impact the trust users have with the data, and the value that analytics can provide.
Blackboard have an open case to investigate these issues, but I’m hoping that if other institutions also think these issues are a priority it might agitate for a fix sooner!
Happy to discuss, and would also be interested in any other data issues your institutions have discovered as well.
Item Activity Issues Summary
This page summarise the issues with when activity in Blackboard sites is (and isn't) tracked by Blackboard Analytics.
Blackboard Analytics users are recommended to review this page before taking actions based on Blackboard Analytics data (such as contacting students).
- Students tend to access Blackboard via the mobile app much more frequently than via a browser. Therefore, generally speaking, missing mobile activity will impact data on individual students with heavy usage of the mobile app, than it will impact aggregate numbers for the Blackboard site or particular item.
- Mobile activity occurs when users access Blackboard via the mobile app, accesses via a web browser on a mobile device are treated as browser activity.
- Non-Interactive Items
- Attachment Types
- 'Content Folder/Content Area' and 'Assignment' Items
'Non-interactive' items, i.e. items that students can't click on, are not tracked by Blackboard Analytics. An example of a 'non-interactive' item is the 'Item' type item, created via the menu in the screenshot below, that also does not have an attachment. Adding an attachment to an item would make it interactive, and therefore trackable. There are some exceptions to this, explained later on the page.
Units with any 'non-interactive' items which have been viewed (i.e. been visible on in a student's browser or on the app screen).
- Views on 'non-interactive' items will not contribute to the activity totals for the site or students in the reports.
- 'Non-interactive' items will not appear on the 'Item Activity' report or 'Student at-a-Glance' report.
- Place the item in its 'Content Folder/Content Area', and look at the activity against the folder instead. Note: this will only work for browser activity, see 'Content Folder/Content Area items' for details.
- Add the item within a 'Learning Module', which which forces item to be viewed on its own page. This will work for both browser and mobile activity.
Activity on attachments (such as in the screenshot below), are not tracked directly and are instead tracked against the parent item. Note: some attachment types are not tracked at all, see 'Attachment Types' for details.
Units with any items in the site with attachments that have been accessed by a student.
- Attachments will not appear in the 'Item Activity' report or 'Student at-a-Glance' report. However you will be able to see the activity for an attachment against the parent item. In the screenshot above, clicks on 'Test Attachment.txt' can be seen against the 'Test Item' item in the reports.
- For items with multiple attachments, as in the example above, you will not be able to tell which attachments were accessed, as all activity is recorded against the parent item.
- Add content as a 'File' rather than as an attachment to another item. This will give the content its own identity, which can be tracked.
- Avoid adding more than one attachment per item, so when looking at the item in the 'Item Activity' report, you know the activity could only been on the one attachment.
Some types of content, when added as an attachment are not tracked, in particular, items which open within the browser (rather than being downloaded separately). This is known to affect .pdf files and images, such as .png, but may affect others as well.
The exception to this is for mobile accesses, which will track activity on these attachments.
Units with affected content types.
- Non trackable attachments (such as .pdfs) will not have their activity contribute to the activity against an item
- Items with an image can have their mobile activity tracked via the mobile app (because the image will open on its own page), but not via the browser (which does not open the image on its own page).
- Add content as a 'File' rather than as an attachment. This will ensure it is trackable.
'Content Folder/Content Area' and 'Assignment' Items
Due to the way the Blackboard mobile app works, activity on 'Content Folder/Content Area' and 'Assignment' items (including top-level folders, such as 'Unit Materials') are not tracked when students access them via the mobile app.
Units with any student(s) who accessed the site via the mobile app.
- Overall 'interactions' and 'minutes' shown on all 'Unit Analytics' reports, will be an under-representation of the total.
- Total activity for 'Content Folder/Content Area' and 'Assignment' items on the 'Item Activity' report will be an under-representation of the total.
- For any student who has accessed any 'Content Folder/Content Area' or 'Assignment' item via mobile, that student's overall 'interactions' and 'minutes' shown on all reports will be an under-representation of the total.
- For any student who has accessed 'Content Folder/Content Area' or 'Assignment' item(s) via mobile, that student's activity against those item(s) on the 'Item Activity' report or 'Student at-a-Glance' report, will be an under-representation of their total activity against that item.
- For any student who has accessed 'Content Folder/Content Area' or 'Assignment' item(s) via mobile, and has also never accessed those items via the browser, that student will not appear against those item(s) on the 'Item Activity' report, and the item(s) will not appear on that students 'Student at-a-Glance' report, as though they have never accessed the item at all.
- Use 'Learning Modules' rather then 'Content Folder/Content Areas' to organise the parts of your site that you want complete tracking on (Learning Modules can be tracked via both the mobile app, and a browser.
- As the majority of Blackboard accesses occur via the browser rather than via the Mobile app, it can be confusing for items which are not trackable via the browser, but are trackable via the mobile app to be displayed on the reports. It will appear as though an item has very few accesses, when in reality, it is just that the browser activity is not being included. A customisation is under consideration which would hide items which have only been accessed via the mobile app, to avoid this confusion.
- Blackboard is also working on a fix which will improve the track-ability these items. Case link: https://blackboard.secure.force.com/btbb_articleview?id=kA3390000004I1U (Behind the Blackboard Access may be required to access this link).
Data and Learning Analyst