A Year Later: Feedback. Focus. Results.

Blog Post created by tjatkin on Oct 4, 2018

Almost exactly a year ago, I shared the results of our 2017 Annual Client Satisfaction Survey via my blog.  We used your feedback to inform our planning and our efforts this past year and dedicated ourselves to continuous improvement with your views as guideposts for this work.


Ease of Doing Business with Blackboard

You told us that our post-sale delivery efforts were uncoordinated and made it difficult for you to work with Blackboard, and that we needed to focus on improved responsiveness in our work to support you.


We established a new organization under the leadership of Lee Blakemore to focus on providing an improved post-sale client experience -- the Global Client Operations & Success organization. We brought together all of the client delivery and support functions across the company into the new organization, along with our infrastructure and cloud services teams, to better integrate and coordinate our work.  


Over the last year we enhanced and formalized our executive and team focus, prioritization and coordination in responding to incidents and challenges that surfaced for our clients. 

  • Our leadership team is informed immediately of outages through an alert system, and we receive a consolidated daily Major Incident Management Report to ensure visibility into occurrences and resolution.
  • Lee Blakemore and I co-chair a weekly hot issues call, with leadership from our support, client success, infrastructure, cloud services and technical teams, to ensure tight coordination and track longer-term corrective actions.
  • We conduct monthly program reviews with our CEO and the entire leadership team that assess the trends and the actions we are taking to improve performance.


We have worked hard to strengthen our focus on continuous improvement as an integral part of our culture and work ethic.  We expanded the scope and strengthened the efforts of our work to understand the root causes of incidents that impact you.  Through our Incident Review Boards (IRBs), we assess the technical, process and human aspects of every incident. We validate the actions taken to resolve the incident were appropriate and as timely as possible. And, we focus on what we can do to improve the “system” to avoid future incidents of a similar nature.  The work of our IRBs is linked to our weekly hot issues calls so we can ensure the appropriate coordination and attention to all incidents.


Product Quality and Stability

You told us product quality and stability were critically important and needed attention. Building upon the outputs and insights from our IRBs as mentioned above, we placed a significant emphasis on improved testing in order to ensure higher quality.


We use an agile scrum process as our software development lifecycle which means our engineers and quality assurance resources approach the work collectively as a team whether it’s work on a bug fix or a feature change. The team reviews requirements from an engineering and QA perspective, looks at the code, and develops a test strategy. They perform a risk analysis to determine what testing needs to be done.


We've improved our automated testing across four areas: unit level testing, integrated testing, API and REST testing, and User Interface accepting testing.  In addition, the team does manual testing to validate all the changes being made are appropriate and typically includes:

  • Manual validation of the bug or feature change
  • Visual test across browsers, tablets, and mobile devices
  • Test using assisted technologies
  • And exploratory testing of critical use cases that may be impacted by the change.


Learn SaaS Clients using Continuous Delivery told us they would like to have more time for User Acceptance Testing (UAT) to ensure quality in their environment and manage the change (they were getting 7 to 8 business days - and one weekend, depending on time zone). Our product management and development teams identified process improvements allowing for more time for client UAT.  Our process improvements enable us to modify the Continuous Delivery Learn SaaS release schedule to give clients more time to conduct user acceptance testing and to manage change; we now provide 12-13 business days (and two weekends, depending on time zone).


We measure ourselves by your direct feedback, and through hard results of our performance. While learning is a year-round endeavor these days, there remains a very busy “Back to School” season in the July through September time period.  While we continue our efforts to improve product stability and quality, I’m proud of the work my colleagues and peers have done through the efforts mentioned above.

  • While the number of SaaS clients has doubled in the last year, the year over year comparison of outages between the 2017 and 2018 back to school season has shown a dramatic improvement – we have seen a 60% reduction in incidents.
  • For managed hosting clients, we similarly saw a significant decrease in outages – a 50% reduction in outages during back to school season (year over year).

And while I’m encouraged by the improvements, we know we are not done. Every outage has a direct impact on student success and we will continue to focus our work to improve stability and quality.


Proactive Communications

You told us proactive communications are important – here are just a few examples of our efforts to be more timely and transparent:

  • We introduced a new feature on Behind the Blackboard that provides a pop-up window announcing any current and impacting issues to enable you to know quickly what’s going on. The pop-up window shows up on your screen once you log into BtBb.
  • We redesigned and made improvements to  in order to keep you updated on planned system maintenance activities and service interruptions as they occur.  You can now receive alerts pushed to your iOS or Android device.
  • Quarterly Teaching & Learning Roadmaps: We offer quarterly roadmap sessions in 3 different time zones highlighting the upcoming features for each Learn, Collaborate and Blackboard Open LMS.


Keeping with our commitment to be transparent, as I did last year, I’ll use my blog next week to share a summary of the results of our 2018 Client Satisfaction Survey.


The Year in Context

Our work to be responsive to your feedback this past year was done while we continued to see a great deal of momentum and movement to Learn SaaS and Ultra.

  • We have seen a significant increase in SaaS deployments. As of this week, we have 428 clients on a Learn SaaS deployment. With the addition of China this summer and South Korea this week, we’ve deployed Learn SaaS to 6 regions plus GovCloud. This gives us a worldwide footprint to meet more and more client’s desires to transition to SaaS.
  • As of this week, we have 91 institutions that have enabled Learn Ultra (that’s more than a 45% increase since BbWorld).


I’m proud of my Blackboard colleagues around the world who are committed to improving how we serve as a partner to you. While we made progress in a number of areas, we know more remains to be done and we are focused on our culture of continuous improvement.


Thank you for your feedback, your partnership, and the work you do.