ahulme

Simple Project Using New Q2 Beta Rest API In a B2 With IBM Watson Tone Analyzer

Blog Post created by ahulme on Aug 18, 2016

Working for Blackboard Support I get to learn something new everyday.  The new Rest API gives much simpler access to course information that is consumed/created externally from Blackboard or wrapped in a B2 and delivered as a course tool.     What you chose to do with the data is the interesting part.

 

This small project came from a desire to learn the new Rest interface in order too better support it. Along with my Wifes input that you should not just stop once you have the data, you need to do something with it.  My wife, Adele works on a Peer Mentor Project for People with Drug and Alcohol challenges and often discuses the way language can have an effect on the groups she teaches.  I mentioned that IBM Watson now has an Emotional Tone Content Analyzer and we though it would be good to run some of her teaching materials into it to see how she scored on the openness and analytical scales.

 

 

Text from Watson Site Tone Analyzer | IBM Watson Developer Cloud

 

The IBM Watson™ Tone Analyzer Service uses linguistic analysis to detect three types of tones from written text: emotions, social tendencies, and writing style. Emotions identified include things like anger, fear, joy, sadness, and disgust. Identified social tendencies include things from the Big Five personality traits used by some psychologists. These include openness, conscientiousness, extraversion, agreeableness, and emotional range. Identified writing styles include confident, analytical, and tentative.

 

This was a useful exercise so I though we would try this with Blackboard content and use the new Rest API to get the content out.

 

NB: I am not a developer so please do not consider any of this as best practice.

 

 

 

 

 

Step 1 Using the new rest API:

 

The new documentation is very easy to use and can be found here:  https://developer.blackboard.com/portal/displayApi

 

Rest Process:

 

  • Register as a developer in https://developer.blackboard.com/
  • Register the Rest Application on https://developer.blackboard.com/  You will get the application ID, Rest Key and Secrete
  • In your Blackboard environment create a Rest integration using the application key
  • Create the application using the REST API Key and Secrete in Eclipse or other IDE.
  • Test the application.

 

pastedImage_5.png

 

 

 

Register the Application here: https://developer.blackboard.com/portal/applications

 

rest Reg.png

Setup the API Integration on the Blackboard server:  System admin > REST API Integration's > Create Integration

 

image2016-7-20 14-31-20.png

The Rest API in Q2 2016 can now be used by an external application so lets start  with a simple java project to connect to the Rest API.

 

Using the examples from  https://github.com/blackboard/BBDN-REST-Demo-Java-Webapp I created 3 classes to manage the token, coursepk1 and content items.

 

getToken: This contain a method that takes the user, password and URL and provides the OAuth Token.

 

getcoursePK1: This contains a method that takes the URL, Token and Course BatchUID and return the course PK1

 

getContent: This has 2 methods

 

  • ParentContentItems: This method takes the URL, Token and Course Pk1 and returns an Object containing the Parent Content details.  This uses jackson-core-asl to convert the json to POJO.  This lists the top level content areas in a course.
  • ChildContenItemsJSON: This method takes the URL, Token and Parent Content Pk1 and returns an Object containing the Parent Content details.  This uses jackson-core-asl to convert the json to POJO.  This lists folder, assignment, items etc that are held in the parent content object.

 

TestMethod: This is to test the above methods.  Set the user, Pass, URL.  Run this to get the top level content and the get the text from the child content items into one string.  This Content of each content item has had the HTML stripped out of it using the Jsoup HTML parser https://jsoup.org/

 

Example course for testing.

 

Capture.PNG

Full project is attached.  GetCourseContent.zip

 

public class TestMethods {
     
    //Setup the user and passwords for the Rest Integration.
    public final static String USER = "Rest User";
    public final static String PASS = "Pass Auto Generated on Rest Registration";
    public final static String VIP = "Https://testserver.blackboard.com/";
     

    public static void main(String[] args) {
        // TODO Auto-generated method stub
         
        //Test the Get Token Method.
        String AccessToken = getToken.GetToken(USER, PASS, VIP);
        System.out.println("Token from getToken class: "+AccessToken);

         
        //Get Course Pk1 using the getCoursePK1 method
        String coursePK1 = getCoursePK1.coursePK1(VIP,  AccessToken, "newcourse");
        System.out.println("course Pk1: "+coursePK1);
         
        //Setup a String to use to populate the Body of the content items.
        StringBuilder stringBuilder = new StringBuilder();
         
                 
        //Get JSON content objects in a list of the parent content.
        List<Result> results = getContent.ParentContenItems(VIP, AccessToken, coursePK1);
        for(int i =0; i < results.size();i++){
             
            //Print the Parent content Pk1
            System.out.println("Parent ID: "+results.get(i).getId());
             
            //Get Child Content into results object
            List<Result> childresults = getContent.ChildContenItemsJSON(VIP, AccessToken, coursePK1,results.get(i).getId());
            for(int ic =0; ic < childresults.size();ic++){
                 
                //Only user content items that contain text in the body
                if (childresults.get(ic).getBody() != null && !childresults.get(ic).getBody().isEmpty()){
                 
                //Print the Child PK1 and body
                System.out.println("Child Item: "+childresults.get(ic).getId()+"  Body: "+Jsoup.parse(childresults.get(ic).getBody()).text());

                //Add the text to the String stringBuilder
                stringBuilder.append(Jsoup.parse(childresults.get(ic).getBody()).text()+"  ");
                }
                 
            }
             
             
             
            }
        //Print out the full text of all item in one text line
        System.out.println("Full text from course: "+stringBuilder);
         
        }

}

 

Output of this test method for an example course.

 

Https://mhtest3.blackboard.com/learn/api/public/v1/oauth2/token<grant_type=client_credentials,{Authorization=[Basic M2FkODI5NDQtYTY0MC00OWUwLWEyOTAtOGY5NzAwOTUxZGZhOnR4SkdIeG5JMzB5YVlLcktCcE1qNkRNOGd5TFRtV01K], Content-Type=[application/x-www-form-urlencoded]}>
Https://mhtest3.blackboard.com/learn/api/public/v1/courses/externalId:eng101
Https://mhtest3.blackboard.com/learn/api/public/v1/courses/_5366_1/contents
Https://mhtest3.blackboard.com/learn/api/public/v1/courses/_5366_1/contents/_24537_1/children
Https://mhtest3.blackboard.com/learn/api/public/v1/courses/_5366_1/contents/_24538_1/children
William Shakespeare (/??e?ksp??r/;[1] 26 April 1564 (baptised) – 23 April 1616)[nb 1] was an English poet, playwright, and actor, widely regarded as the greatest writer in the English language and the world's pre-eminent dramatist.[2] He is often called England's national poet, and the "Bard of Avon".[3][nb 2] His extant works, including collaborations, consist of approximately 38 plays,[nb 3] 154 sonnets, two long narrative poems, and a few other verses, some of uncertain authorship. His plays have been translated into every major living language and are performed more often than those of any other playwright.[4] Shakespeare was born and brought up in Stratford-upon-Avon, Warwickshire. At the age of 18, he married Anne Hathaway, with whom he had three children: Susanna, and twins Hamnet and Judith. Sometime between 1585 and 1592, he began a successful career in London as an actor, writer, and part-owner of a playing company called the Lord Chamberlain's Men, later known as the King's Men. He appears to have retired to Stratford around 1613, at age 49, where he died three years later. Few records of Shakespeare's private life survive, which has stimulated considerable speculation about such matters as his physical appearance, sexuality, and religious beliefs, and whether the works attributed to him were written by others.[5] Shakespeare produced most of his known work between 1589 and 1613.[6][nb 4] His early plays were primarily comedies and histories, and these are regarded as some of the best work ever produced in these genres. He then wrote mainly tragedies until about 1608, including Hamlet, Othello, King Lear, and Macbeth, considered some of the finest works in the English language.[2] In his last phase, he wrote tragicomedies, also known as romances, and collaborated with other playwrights. Many of his plays were published in editions of varying quality and accuracy during his lifetime. In 1623, however, John Heminges and Henry Condell, two friends and fellow actors of Shakespeare, published a more definitive text known as the First Folio, a posthumous collected edition of his dramatic works that included all but two of the plays now recognised as Shakespeare's.[7] It was prefaced with a poem by Ben Jonson, in which Shakespeare is hailed, presciently, as "not of an age, but for all time".[7] In the 20th and 21st centuries, his works have been repeatedly adapted and rediscovered by new movements in scholarship and performance. His plays remain highly popular, and are constantly studied, performed, and reinterpreted in diverse cultural and political contexts throughout the w  Watch Video Introduction to William Wordsworth User: n/a - Added: 7/18/12 William Wordsworth was a major English Romantic poet who, with Samuel Taylor Coleridge, helped to launch the Romantic Age in English literature with their joint publication Lyrical Ballads.    

 

 

Step 2 Deploy as a B2 Course Tool and Integrate with Watson.

 

Capture2.PNG

 

B2 Template

I used the amazing starting template  B2 from the All the Ducks team. https://github.com/AllTheDucks/atd-b2-stub

 

This B2 uses the Stripes framework and is an ideal starter B2.  Clone from GitHub  using Git bash for windows.

 

 

$ git clone https://github.com/AllTheDucks/atd-b2-stub.git BBS-AHRestContent

Cloning into 'BBS-AHRestContent'...

remote: Counting objects: 528, done.

remote: Total 528 (delta 0), reused 0 (delta 0), pack-reused 528 eceiving objects:  98% (518/528), 148.01 KiB | 286.00 KReceiving objects:  99% (523/528), 148.01 KiB | 286.00 KiB/s

Receiving objects: 100% (528/528), 167.01 KiB | 286.00 KiB/s, done.

Resolving deltas: 100% (225/225), done.

Checking connectivity... done.

 

ahulme@AHULMEP50 MINGW64 /c/Temp/AHRestContent (master)

$ cd BBS-AHRestContent/

 

ahulme@AHULMEP50 MINGW64 /c/Temp/AHRestContent/BBS-AHRestContent (master)

$ ./gradlew initB2

:initB2

Defaults found for generating building block

> Building 0% > :initB2

What is your Vendor Id (e.g. usq, unsw, qut, swin, etc)? (No spaces)  [BBS] BBS

What is your Vendor Name (e.g. University of Antarctica)? [Blackboard Support]

What is your Vendor Website (e.g. http://www.myu.edu.au/)? (No spaces)  [http://community.blackboard.com/]

What is your B2 Handle (e.g. jshack, autosignon, etc)? (No spaces)  AHRestContent

What is your B2 Name (e.g. Student View)? My RContent

What is the base java package of your project? (e.g. au.edu.uni.myproject) ? (No spaces)  com.ah

Do you want to use the course event listener? (Y/N) N

Do you want to use the Schema.xml? (Y/N) N

Do you want a system tool? (Y/N) Y

Do you want a course tool? (Y/N) Y

Initialising Building Block BBS-AHRestContent

Initialized empty Git repository in C:/Temp/AHRestContent/BBS-AHRestContent/.git/

 

BUILD SUCCESSFUL

 

Total time: 1 mins 13.789 secs

 

 

Setup the Stripes Controller for the Course tool.

 

This uses the 3 class file we created instep 1  with a new method in the getContent and the POJO classes for object have been moved to there own package.

 

getToken: This contain a method that takes the user, password and URL and provides the OAuth Token.

getcoursePK1: This contains a method that takes the URL, Token and Course BatchUID and return the course PK1

getContent: This has 3 methods

 

  • Content: This method takes the URL, Token and Course Pk1 and returns a String containing the text of the content items in the course. This only gets the text form the Parent level and one child level at the moment. .
  • ParentContentItems: This method takes the URL, Token and Course Pk1 and returns an Object containing the Parent Content details.  This uses jackson-core-asl to convert the json to POJO.  This lists the top level content areas in a course.
  • ChildContenItemsJSON: This method takes the URL, Token and Parent Content Pk1 and returns an Object containing the Parent Content details.  This uses jackson-core-asl to convert the json to POJO.  This lists folder, assignment, items etc that are held in the parent content object.

 

 

Setup the Watson service:  Login to bluemix and select the Tone Analyzer.

 

 

 

Stage 1: Getting your service credentials

Before you can work with a Watson Service, you need service credentials in Bluemix. If you already have credentials for this service, you can skip this stage.

To get your service credentials, follow these steps:

  1. Log in to Bluemix at console.ng.bluemix.net.
  2. Create an instance of the service:
    1. In the Bluemix Catalog, select your service.
    2. Under Add Service, type a unique name for the service instance in the Service namefield. For example, type tutorial-<username>. Leave the default values for the other options.
    3. Click Use.
  3. Copy your credentials:
    1. On the left side of the page, click Service Credentials to view your service credentials.
    2. Copy username and password from these service credentials.

 

This is from my account

 

Watson.png

 

 

You can test this from the command line using curl. This example is taken from the IBM documentation.

 

curl -k -u "c5fxccc-xzxcc-43b3-a95f-6edbebaa346a":"aJxFVbhXbY2c" -H "Content-Type: application/json" -d "{\"text\": \"Hi Team, I know the times are difficult! Our sales have been disappointing for the past three quarters for our data analytics product suite. We have a competitive data analytics product suite in the industry. But we need to do our job selling it! \"}" "https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2016-05-19"

 

C:\Users\ahulme\Downloads\curl_749_1_ssl>curl -k -u "c5f6ce24-1ae0-232323-a95f-6edbebaa346a":"aJxF233323bY2c" -H "Content-Type: application/json" -d "{\"text\": \"Hi Team, I know the times are difficult! Our sales have been disappointing for the past three quarters for our data analytics product suite. We have a competitive data analytics product suite in the industry. But we need to do our job selling it! \"}" "https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2016-05-19"

 

 

KeyValue
Document_tone{"tone_categories":[{"tones":[{"score":0.455891,"tone_id":"anger","tone_name":"Anger"},{"score":0.156707,"tone_id":"disgust","tone_name":"Disgust"},{"score":0.17315,"tone_id":"fear","tone_name":"Fear"},{"score":0.190073,"tone_id":"joy","tone_name":"Joy"},{"score":0.291627,"tone_id":"sadness","tone_name":"Sadness"}],"category_id":"emotion_tone","category_name":"Emotion Tone"},{"tones":[{"score":0.459,"tone_id":"analytical","tone_name":"Analytical"},{"score":0.0,"tone_id":"confident","tone_name":"Confident"},{"score":0.0,"tone_id":"tentative","tone_name":"Tentative"}],"category_id":"language_tone","category_name":"Language Tone"},{"tones":[{"score":0.03,"tone_id":"openness_big5","tone_name":"Openness"},{"score":0.188,"tone_id":"conscientiousness_big5","tone_name":"Conscientiousness"},{"score":0.405,"tone_id":"extraversion_big5","tone_name":"Extraversion"},{"score":0.879,"tone_id":"agreeableness_big5","tone_name":"Agreeableness"},{"score":0.962,"tone_id":"emotional_range_big5","tone_name":"Emotional Range"}],"category_id":"social_tone","category_name":"Social Tone"}]}
Sentences_tone[{"sentence_id":0,"text":"Hi Team, I know the times are difficult!","input_from":0,"input_to":40,"tone_categories":[{"tones":[{"score":0.340698,"tone_id":"anger","tone_name":"Anger"},{"score":0.132219,"tone_id":"disgust","tone_name":"Disgust"},{"score":0.183509,"tone_id":"fear","tone_name":"Fear"},{"score":0.32062,"tone_id":"joy","tone_name":"Joy"},{"score":0.306103,"tone_id":"sadness","tone_name":"Sadness"}],"category_id":"emotion_tone","category_name":"Emotion Tone"},{"tones":[{"score":0.847,"tone_id":"analytical","tone_name":"Analytical"},{"score":0.0,"tone_id":"confident","tone_name":"Confident"},{"score":0.0,"tone_id":"tentative","tone_name":"Tentative"}],"category_id":"language_tone","category_name":"Language Tone"},{"tones":[{"score":0.035,"tone_id":"openness_big5","tone_name":"Openness"},{"score":0.354,"tone_id":"conscientiousness_big5","tone_name":"Conscientiousness"},{"score":0.787,"tone_id":"extraversion_big5","tone_name":"Extraversion"},{"score":0.585,"tone_id":"agreeableness_big5","tone_name":"Agreeableness"},{"score":0.927,"tone_id":"emotional_range_big5","tone_name":"Emotional Range"}],"category_id":"social_tone","category_name":"Social Tone"}]},{"sentence_id":1,"text":"Our sales have been disappointing for the past three quarters for our data analytics product suite.","input_from":41,"input_to":140,"tone_categories":[{"tones":[{"score":0.437543,"tone_id":"anger","tone_name":"Anger"},{"score":0.206324,"tone_id":"disgust","tone_name":"Disgust"},{"score":0.137719,"tone_id":"fear","tone_name":"Fear"},{"score":0.180326,"tone_id":"joy","tone_name":"Joy"},{"score":0.272556,"tone_id":"sadness","tone_name":"Sadness"}],"category_id":"emotion_tone","category_name":"Emotion Tone"},{"tones":[{"score":0.0,"tone_id":"analytical","tone_name":"Analytical"},{"score":0.0,"tone_id":"confident","tone_name":"Confident"},{"score":0.0,"tone_id":"tentative","tone_name":"Tentative"}],"category_id":"language_tone","category_name":"Language Tone"},{"tones":[{"score":0.164,"tone_id":"openness_big5","tone_name":"Openness"},{"score":0.372,"tone_id":"conscientiousness_big5","tone_name":"Conscientiousness"},{"score":0.399,"tone_id":"extraversion_big5","tone_name":"Extraversion"},{"score":0.759,"tone_id":"agreeableness_big5","tone_name":"Agreeableness"},{"score":0.807,"tone_id":"emotional_range_big5","tone_name":"Emotional Range"}],"category_id":"social_tone","category_name":"Social Tone"}]},{"sentence_id":2,"text":"We have a competitive data analytics product suite in the industry.","input_from":141,"input_to":208,"tone_categories":[{"tones":[{"score":0.308401,"tone_id":"anger","tone_name":"Anger"},{"score":0.298622,"tone_id":"disgust","tone_name":"Disgust"},{"score":0.254084,"tone_id":"fear","tone_name":"Fear"},{"score":0.161348,"tone_id":"joy","tone_name":"Joy"},{"score":0.224839,"tone_id":"sadness","tone_name":"Sadness"}],"category_id":"emotion_tone","category_name":"Emotion Tone"},{"tones":[{"score":0.0,"tone_id":"analytical","tone_name":"Analytical"},{"score":0.0,"tone_id":"confident","tone_name":"Confident"},{"score":0.0,"tone_id":"tentative","tone_name":"Tentative"}],"category_id":"language_tone","category_name":"Language Tone"},{"tones":[{"score":0.628,"tone_id":"openness_big5","tone_name":"Openness"},{"score":0.926,"tone_id":"conscientiousness_big5","tone_name":"Conscientiousness"},{"score":0.031,"tone_id":"extraversion_big5","tone_name":"Extraversion"},{"score":0.526,"tone_id":"agreeableness_big5","tone_name":"Agreeableness"},{"score":0.54,"tone_id":"emotional_range_big5","tone_name":"Emotional Range"}],"category_id":"social_tone","category_name":"Social Tone"}]},{"sentence_id":3,"text":"But we need to do our job selling it!","input_from":209,"input_to":246,"tone_categories":[{"tones":[{"score":0.182246,"tone_id":"anger","tone_name":"Anger"},{"score":0.101415,"tone_id":"disgust","tone_name":"Disgust"},{"score":0.26452,"tone_id":"fear","tone_name":"Fear"},{"score":0.486167,"tone_id":"joy","tone_name":"Joy"},{"score":0.217572,"tone_id":"sadness","tone_name":"Sadness"}],"category_id":"emotion_tone","category_name":"Emotion Tone"},{"tones":[{"score":0.0,"tone_id":"analytical","tone_name":"Analytical"},{"score":0.0,"tone_id":"confident","tone_name":"Confident"},{"score":0.0,"tone_id":"tentative","tone_name":"Tentative"}],"category_id":"language_tone","category_name":"Language Tone"},{"tones":[{"score":0.013,"tone_id":"openness_big5","tone_name":"Openness"},{"score":0.018,"tone_id":"conscientiousness_big5","tone_name":"Conscientiousness"},{"score":0.833,"tone_id":"extraversion_big5","tone_name":"Extraversion"},{"score":0.952,"tone_id":"agreeableness_big5","tone_name":"Agreeableness"},{"score":0.975,"tone_id":"emotional_range_big5","tone_name":"Emotional Range"}],"category_id":"social_tone","category_name":"Social Tone"}]}]

 

 

 

Adding the service to a Building Block using the Java SDK for Watson

 

Instructions can be found here: https://www.ibm.com/watson/developercloud/tone-analyzer/api/v3/?java#authentication

 

Step 1. Set the REST and Watson User/Pass in the B2 setting page.

 

Add the following into the course stripes control Action.  CourseToolActoin.java

 

@SpringBean

privateConfigurationService<Configuration> configService; //injected with Spring

    //Setup Rest URL.  Get Hostname and Protocol from the BB Config service

    String Hostname = blackboard.platform.config.ConfigurationServiceFactory.getInstance().getBbProperty("bbconfig.frontend.fullhostname");

    String protocol = blackboard.platform.config.ConfigurationServiceFactory.getInstance().getBbProperty("bbconfig.frontend.protocol");

    String VIP = protocol+"://"+Hostname+"/";

   

    //Get REST AND Watson user And Pass from B2 Config

    String USER = configService.loadConfiguration().getSettingOne();

    String PASS = configService.loadConfiguration().getSettingTwo();

    String WATUSER =configService.loadConfiguration().getWatsonUser();

    String WATPASS =  configService.loadConfiguration().getWatsonPass();

  

 

Edit the Configuration.java to add in the new settings. This file was generated by the B2 template and already contained 2 settings.  We add 2 more for thew Watson user/pass.

 

package com.ah.config;

 

import net.sourceforge.stripes.validation.Validate;

/**
 * Created by wiley on 19/11/14.
 */
public class Configuration {

    private String settingOne;
    private String settingTwo;
    private String watsonUser;
    private String watsonPass;
    
    //Rest User Name
    public String getSettingOne() {
        return settingOne;
    }

    public void setSettingOne(String settingOne) {
        this.settingOne = settingOne;
    }
    
    //Rest Pass
    public String getSettingTwo() {
        return settingTwo;
    }

    public void setSettingTwo(String settingTwo) {
        this.settingTwo = settingTwo;
    }
    
    //Watson User
    public String getWatsonUser() {
        return watsonUser;
    }

    public void setWatsonUser(String watsonUser) {
        this.watsonUser = watsonUser;
    }
    
    
    //Watson Password
    public String getWatsonPass() {
        return watsonPass;
    }

    public void setWatsonPass(String watsonPass) {
        this.watsonPass = watsonPass;
    }
    
                   
}

 

 

Using the Watson SDK.

 

Add the  following to your gradle build file to get the SDK

 

compile 'com.ibm.watson.developer_cloud:java-sdk:3.0.1'

 

You can use this service by using the following code:

 

 

ToneAnalyzer service = new ToneAnalyzer(ToneAnalyzer.VERSION_DATE_2016_05_19);
        service.setUsernameAndPassword(User, Pass);

        String text = contenttext;

        // Call the service and get the tone
        ToneAnalysis toneAnalysis = service.getTone(text, null).execute();

     

       

  

 

Edit the config.JSP to add the new Form fields.

 

 

Config.png

 

BB Manifest changes. in order for Blackboard to be able to use the Watson service the following permissions are needed.

 

 

<permission type="java.lang.RuntimePermission"name="getenv.VCAP_SERVICES"/>

<permission type="java.net.NetPermission"name="getProxySelector"/>

Outcomes