Skip navigation

 RSS Response from the back of the covered wagon

RSS posted on behalf of Blackboard Blog 4 years ago

    I was very happy to see today’s Chronicle article about Candace Thille’s take on analytics in higher education (As Big-Data Companies Come to Teaching, a Pioneer Issues a Warning). She is definitely a pioneer in the space, and given that, I’d like to chime in from my seat behind the driver in the back of the covered wagon. First, I need to point out that while I haven’t met Candace first hand, she is linked to the development of my analytics career. In 2013, I was on a higher ed analytics panel at a conference in Las Vegas. Candace was on the panel before me and Larry Summers was the keynote after me. Looking through the time machine lens, it was interesting. Summers was two years removed from his role in “The Social Network”…made for some good laughs at the keynote. Candace was on a panel with Andrew Ng from Coursera, and the panel description started off with this gem: “MOOC madness is here!”. Ahhh…2013…we hardly knew ye.

    Now, back to the topic at hand. In the article, Candace talks about her warnings towards the higher ed analytics space. While I detect a bit of sensationalism in the article (“Ms. Thille has begun to have darker thoughts about an industry she helped spark”), I think it’s a great piece to keep the conversation and constructive criticism alive. Her core thesis is that rapid commercialization is the wrong way to foster innovation in the analytics space and that higher ed is making a mistake by letting companies take the lead in shaping the market. I’d like to approach each of her points separately and add my commentary:

    • Colleges should have more control over the field   
      • I agree 100%. As a former analytics person at an institution, a founder of an analytics startup, and a current decision maker for a large analytics software company, I am in full agreement. I wish all of my clients took the initiative to lead. I think vendors (like what we do) should be seen as partners and specialists. Using the home building analogy—it’s your house, you designed it, and you’re going to live in it. However, if you need a roofer to work on the roof and that’s not your forte, it makes sense to call in a trusted vendor to finish the job (while focusing on quality and doing it for a reasonable price).
    • Companies aren’t as well equipped to develop and test new teaching algorithms as colleges are   
      • I “sorta” agree on this but not 100%. Obviously, a college can test classroom tools/approaches and a company can’t. However, that’s a single test bed. A company that has data from many different institutions has the ability to observe disparate data (different students, modalities, pedagogical approaches, uses of technology, etc.). Ideally, there’s a partnership that allows the two groups to work together. I’ve been involved in a number of such partnerships and I can say first-hand that there are wonderful collaborations going on in the space. I don’t see it as an either/or proposition.
    • Companies “black box” the learning technology decisions from colleges   
      • While I agree that there ARE companies that do this, I wholeheartedly disagree with this as a blanket statement. It’s easy to understand the genesis of this (robot tutors in the sky are referenced twice in this article….thanks, Knewton!). That being said, I am 100% against this proprietary approach to models. It doesn’t do anyone any good… it’s a short-term win and one that creates mistrust. I wrote all about it two years ago here (Your Secret Sauce Isn’t So Secret). I sincerely hope that this “black box” trope doesn’t dominate the space. It’s real easy to figure out who is who. Talk to the vendor about what modeling techniques they use and ask what some of the features/weights are for models they’ve built. Like Einstein said, “If you can’t explain it simply, you don’t understand it well enough.”
    • The demands of the VC market will inhibit innovation rather than foster it   
      • Again, I think it’s a matter of good actors and bad actors (wait…haven’t we heard that in higher ed before?). I’ll agree that it’s hard to discern one from the other, but that doesn’t mean that institutions should stay away from any commercial analytics vendor. Yes…I get the subtext here…I work for a commercial analytics vendor. Hopefully I’ve earned a bit of street cred to be able to make the statement, though.

    Later in the article, Norman Bier from CMU talks about the need for openness and transparency with analytics and adaptive learning. The author then states, “But the trend is going in the other direction. More and more colleges are turning to the commercial market for their adaptive-learning products.” There’s an overt assumption that “commercial” is antithetical to “open.” I get it, but I don’t think it’s a simple dichotomy. There’s a lot of effort and many resources that go into these tools. They’re difficult to do the right way. As I look through my lens of ed tech history, I see some institution-driven initiatives that worked, some that didn’t, and some where the jury is still out (here are some names that fall into one of these buckets—Sakai, LTI, Caliper, Moodle, OAAI, Unizin, and a host of other names/acronyms that you may not have heard of). My point is that some things require the resources and coordination that an open or community-sourced effort just can’t do.

    To summarize, I don’t like the “us vs. them” dichotomy. I think it’s patently obvious that there’s a need for a partnership. Learning technology companies can’t develop closed tools in a vacuum, and institutions don’t have the resources/continuity to bring an effective tool to market (IMHO). The article does a good job of showing both sides. There are comments from ALEKS and input from the always prescient George Siemens. However, the closing quote from Candace fuels the us vs. them flames:


    “To commodify it at this point in time, when we’re still doing very active research in it, almost assures that we’ll get less innovative, suboptimal products.”

    I disagree. If Goldie Blumenstyk talked to me for this article, I’d close with something like this:


    “Learning analytics is still a developing field. Joint exploration between the professors who are in the field and the commercial researchers who have the data and resources will help evolve the state of the market for all.”

    If you disagree with me, go to an ELI or LAK conference and listen to some of the work that’s being presented and discussed by individuals…regardless of whether their name tag has a .edu or .com on it.


    The post Response from the back of the covered wagon appeared first on Blackboard Blog.

    shared via RSS