Skip navigation
All Places > Blackboard Developer Community > Blog

Hello, I recently just had to run a report via Open DB and supply some Learn Information to a vendor that will be creating a PoC. I was tasked on getting this information but the information of course would violate FERPA. So I had to come up with a way to mask the user information to still supply an accurate model but protect the user information at the same time. So I decided to use an old python script that converted CSV files to XLSX and make some modifications.


 

I used openpyxl to create the Excel XLSX files, Docopt to make the CLI creation easy, and some built in core modules: csv, json, and random.


 

The main challenge was that I had two reports from Open Db that correlated between Activity and Gradebook Data by the USER_ID. I ended up doing some Python wizardry and use a global (gasp!! what globals in Python, your fired!) Anyways, yes the use of a global in Python is frowned upon, but hey it got the job done! That’s what re-factoring is for...right?


 

Anyways, I thought I would share this application and share my work and hopefully see how useful this can be. I plan on implementing a way to connect directly to Open Db (Already have it done in another project) and then add in some common functionality: Mongo DB Storage, Better Configuration Methods, etc.


 

You are all welcome to use, share, clone, hack, and destroy this code here:

https://github.com/elmiguel/CSV2XLSX


 

You will just need to query and save your own data sets… Luckily for you, I will share my queries:

 

Activity Accumulator:


 

SELECT AA.PK1
,AA.EVENT_TYPE
,U.USER_ID
,CM.BATCH_UID
,AA.GROUP_PK1
,AA.FORUM_PK1
,AA.INTERNAL_HANDLE
,AA.CONTENT_PK1
,AA.DATA
,AA.TIMESTAMP
,AA.STATUS
,AA.MESSAGES
,AA.SESSION_ID
FROM BBLEARN.ACTIVITY_ACCUMULATOR AA
INNER JOIN BBLEARN.USERS U
ON AA.USER_PK1 = U.PK1
INNER JOIN BBLEARN.COURSE_MAIN CM
ON AA.COURSE_PK1 = CM.PK1
WHERE CM.COURSE_ID IN (‘LIST COURSE_IDS HERE’
)
ORDER BY AA.TIMESTAMP DESC;







 

 

 
  Gradebook Data:


 

SELECT
CM.COURSE_ID,
U.USER_ID,
U.FIRSTNAME,
U.LASTNAME,
GM.TITLE,
GM.DUE_DATE,
A.SCORE,
CASE A.STATUS
WHEN 1 THEN 'NOT_ATTEMPTED'  
WHEN 2 THEN 'ABANDONED'
WHEN 3 THEN 'IN_PROGRESS'  
WHEN 4 THEN 'SUSPENDED'
WHEN 5 THEN 'CANCELLED '
WHEN 6 THEN 'NEEDS_GRADING'  
WHEN 7 THEN 'COMPLETED'
WHEN 8 THEN 'IN_MORE_PROGRESS'  
WHEN 9 THEN 'NEEDS_MORE_GRADING'
ELSE 'NO STATUS'
END AS STATUS,
A.ATTEMPT_DATE,
A.FIRST_GRADED_DATE,
A.LAST_GRADED_DATE,
A.DATE_ADDED,
A.DATE_MODIFIED,
A.LATEST_IND
-- A.STUDENT_SUBMISSION
-- A.PK1 ATTEMPT_PK1,
-- A.QTI_RESULT_DATA_PK1,
-- GG.GRADEBOOK_MAIN_PK1,
-- GG.PK1 GG_PK1
FROM GRADEBOOK_MAIN GM  
INNER JOIN GRADEBOOK_GRADE GG
ON GG.GRADEBOOK_MAIN_PK1 = GM.PK1
INNER JOIN ATTEMPT A
ON A.GRADEBOOK_GRADE_PK1 = GG.PK1
INNER JOIN COURSE_USERS CU
ON GG.COURSE_USERS_PK1 = CU.PK1
INNER JOIN COURSE_MAIN CM
ON CU.CRSMAIN_PK1 = CM.PK1
INNER JOIN USERS U
ON CU.USERS_PK1 = U.PK1
WHERE CM.COURSE_ID IN (‘LIST COURSE_IDS HERE’);







mkauffman

ALL FILES No More

Posted by mkauffman Apr 18, 2017

I'm writing this to announce an upcoming requirement for all Building Blocks (B2s). Currently Blackboard allows the following in a B2's bb-manifest.xml file.

<permission type="java.io.FilePermission" name="&amp;lt;&amp;lt;ALL FILES&amp;gt;&amp;gt;" actions="read,write,delete"/>

The above allows the B2 to write to anywhere on the host file system.

 

Because of the security implications,  Blackboard is asking all B2 developers to remove that permission from bb-manifest.xml by Q4 2017. We'll be communicating this out here on our Community site, via our Partner Newsletter, in an Announcement on Behind the Blackboard, etc.

 

Below is a set of permissions that opens up everything in the Blackboard directories. These are almost as 'bad' as <<ALL FILES>> in that they let you overwrite other Blackboard files and content, for example anything in the vi directory, but are far better than allowing changes to any file in the file system. /- indicates every file and directory beneath the specified folder. Once you get your Building Block functioning with the following, then we recommend reducing the set of permissions to only those directories/files that it needs to access, and only those actions that are necessary.

 

<permission type="java.io.FilePermission" name="${java.home}/-" actions="read"/>

<permission type="java.io.FilePermission" name="BB_HOME/-" actions="read,write,delete"/>

<permission type="java.io.FilePermission" name="BB_HOME/apps/tomcat/temp/-"  actions="read,write,delete" />

<permission type="java.io.FilePermission" name="BB_CONTENT/-" actions="read,write,delete"/>

 

Here is an option for logging only that is less promiscuous, if your B2 doesn't need to write elsewhere in the BB directories.

<permission type="java.io.FilePermission" name="BB_HOME/logs/-" actions="read,write,delete"/>

 

Following is a link to sample code that uses logback to write to blackboard/logs/custom and blackboard/logs/plugins/<vendor_id>-<handle>/

 

blackboard/logs/plugins/<vendor_id>-<handle> is the directory that B2s should be writing to from here on out. It's the only B2-specific directory that makes the B2 specific logs available to the Kibana log visualizer in SaaS.  GitHub - mark-b-kauffman/bbdn-bblogbackb2: Demo the use of Logback to create log files.

 

Please plan to address this soon so that your Building Block will be able to be installed in Q4 2017 and later.

Now we have a working Building Block lets add some more information to the course tool as a form.

 

CourseToolConfig.java

 

//Create a LIST filled with Group Objects using the Course ID.

 

  private static List<Group> GroupObjects(String course_id) throws PersistenceException{
        GroupDbLoader groupDbLoader = GroupDbLoader.Default.getInstance();
        List<Group> groups = groupDbLoader.loadAvailableByCourseId((Id.generateId(new DataType(blackboard.data.course.Course.class), course_id)));

        List<Group> G1 = new LinkedList<>();

            for (Group group : groups){
                
                //Get group titles. Add to Linked list g1.
                G1.add(group);
                System.out.println("Group Size"+group.getGroupMemberships().size());
    
                
            }
            
            return G1;
        
    }

 

 

Now we need to add some Getter's and Setter's for the List we will pass to the JSP page.

 

 

private List<Group> groupTitles; 
public List<Group> getGroupTitles() {return groupTitles; }
public void setGroupTitles(List<Group> groupTitles) { this.groupTitles = groupTitles; }

 

 

In our Default Handler   @DefaultHandler   public Resolution displayPage() we can populate this list

 

groupTitles = GroupObjects(course_id);

 

To display this in the JSP we use the actionBean.groupTitles List and then can call out the getTitle and getGroupMemberships().size() from the Group Object in the List

 

<bbNG:form action="coursetool" method="post">

        <bbNG:dataCollection>

            <bbNG:step title="Groups">

         

                        <bbNG:inventoryList description="Group Names"  className="Group" collection="${actionBean.groupTitles}" objectVar="title" emptyMsg="No data to display">

                        <bbNG:listElement  name="Group" label="Group" isRowHeader="true">${title.getTitle()}</bbNG:listElement>

                        <bbNG:listElement  name="Group Enrollments" label="Group Enrollments"  isRowHeader="false">${title.getGroupMemberships().size()}</bbNG:listElement>

                        <bbNG:listCheckboxElement  title="Title" name="${title.getTitle()}" id="ID" value="Y"></bbNG:listCheckboxElement>

                </bbNG:inventoryList>

 

                        <input type="hidden" name="course_id" value="${actionBean.course_id}" />

                     

                     

                 

            <bbNG:stepInstructions text="    ${message}  <BR>

               Course Title =  ${actionBean.course_title} <BR>

               Course Pk1 =  ${actionBean.course_id}  <BR>

               User Name  = ${actionBean.username}

               "/>

            </bbNG:step>

        <bbNG:stepSubmit  cancelUrl="Cancle" />

        </bbNG:dataCollection>

    </bbNG:form>

In part one we build a basic course tool.  In this part we will use the config utility to set dome value for our B2.  These can be used for things like Global user/pass.

 

The B2 stub uses ATD's Configuration Utilities Library to provide configuration services to the building block. This building block handles the saving, loading and caching of a configuration bean.

 

 

The Configuration.java contain the list of Getters and Setters for this page. More can be added of modified.

 

package com.devday.config;

import net.sourceforge.stripes.validation.Validate;

/**
 * Created by wiley on 19/11/14.
 */
public class Configuration {

    private String settingOne;
    private int settingTwo;

    public String getSettingOne() {
        return settingOne;
    }

    public void setSettingOne(String settingOne) {
        this.settingOne = settingOne;
    }

    public int getSettingTwo() {
        return settingTwo;
    }

    public void setSettingTwo(int settingTwo) {
        this.settingTwo = settingTwo;
    }
}

 

 

Stripes config.action

 

@LoginRequired
@EntitlementRestrictions(entitlements = "bbs.yourb2name.admin.MODIFY", errorPage = "/error.jsp")
@UrlBinding("/config")
public class ConfigAction implements ActionBean {

    private BlackboardActionBeanContext context;

    @ValidateNestedProperties({@Validate(field = "settingOne", required = true),
            @Validate(field = "settingTwo", required = true, minvalue = 0, maxvalue = 100)})
    private Configuration config;

    @SpringBean
    private ConfigurationService<Configuration> configService;

    @Before(stages = LifecycleStage.BindingAndValidation)
    public void loadConfiguration() {
        config = configService.loadConfiguration();
    }


    @DefaultHandler
    @DontValidate
    public Resolution displayConfigPage() {
        return new ForwardResolution("/WEB-INF/jsp/config.jsp");
    }

    public Resolution saveConfiguration() {
        configService.persistConfiguration(config);
        return new RedirectResolution(PlugInUtil.getPlugInManagerURL(), false);
    }

    @Override
    public ActionBeanContext getContext() {
        return context;
    }

    @Override
    public void setContext(ActionBeanContext context) {
        this.context = (BlackboardActionBeanContext)context;
    }

    public Configuration getConfig() {
        return config;
    }

    public void setConfig(Configuration config) {
        this.config = config;
    }
}

 

 

Note the Form Validation, if you make a change to the methods for example making the int a String you will need to update this validation to match the type.

 

@ValidateNestedProperties({@Validate(field = "settingOne", required = true),     @Validate(field = "settingTwo", required = true, minvalue = 0, maxvalue = 100)})

 

Default Values:

 

If you want to pre-populate these configs with default values then edit the defultconfig.xml under src/main/resources

 

<com.devday.config.Configuration>
  <settingOne>A default string value</settingOne>
  <settingTwo>42</settingTwo>
</com.devday.config.Configuration>

 

Config JSP.

 

This uses a simple Form to collect the data and pass via the Stripes Action

 

 

 

<stripes:form beanclass="com.devday.stripes.ConfigAction">

        <stripes:param name="saveConfiguration"/>

 

        <bbNG:dataCollection>

            <bbNG:step title="${toolSettingsStepTitle}" instructions="${toolSettingsStepInstructions}">

                <bbNG:dataElement isRequired="true" label="${settingOneLabel}">

                    <stripes:text name="config.settingOne"></stripes:text>

                    <stripes:errors field="config.settingOne"></stripes:errors>

                </bbNG:dataElement>

                <bbNG:dataElement isRequired="true" label="${settingTwoLabel}">

                    <stripes:text name="config.settingTwo"></stripes:text>

                    <stripes:errors field="config.settingTwo"></stripes:errors>

                </bbNG:dataElement>

            </bbNG:step>

            <bbNG:stepSubmit></bbNG:stepSubmit>

        </bbNG:dataCollection>

    </stripes:form>

For those of you joining the DevCon in Milan we will be running a Get you started session for Building Blocks using the fantastic Building Block template from the 'All the Ducks' Team.  This uses Stripes as its framework and Gradle to Build the project.

 

This will be a Multi-part series of Blog posts designed to get your up and running with your first B2.

 

If you would like to join the session then it will be helpful to run though this guide before hand and install Eclipse and Git Bash.

 

We will cover in the session:

 

  1. Getting you started with Eclipse
  2. Language pack Bundles
  3. API calls
  4. Data Handling

 

 

Install Eclipse:

 

Download and install Eclipse from Eclipse - The Eclipse Foundation open source community website.

 

Download Git bash for Windows Git - Downloading Package

 

Getting Started.

 

Launch git bash and CD to your Work space. I am using

 

ahulme@AHULMEP50 MINGW64 ~

$ cd workspace/devday/

 

Clone the template to your local Workspace.  Note the First part of the directory name shroud be you Vendor ID. In my case I am using bbs  (Use lowercase chars) and then the name of the B2.  This example uses bbs-yourb2name

 

 

ahulme@AHULMEP50 MINGW64 ~/workspace/devday

$ git clone https://github.com/AllTheDucks/atd-b2-stub.git bbs-yourb2name

Cloning into 'bbs-yourb2name'...

remote: Counting objects: 712, done.

remote: Total 712 (delta 0), reused 0 (delta 0), pack-reused 712 eceiving objects:  99% (705/712)

Receiving objects: 100% (712/712), 206.33 KiB | 0 bytes/s, done.

Resolving deltas: 100% (323/323), done.

Checking connectivity... done.

 

Change into the project directory

 

ahulme@AHULMEP50 MINGW64 ~/workspace/devday

$ cd bbs-yourb2name/

 

Run the initB2 task and answer all the questions.  Make sure you set the Vendor ID to match the one used above and the B2 handle as the same name as above.

 

ahulme@AHULMEP50 MINGW64 ~/workspace/devday/bbs-yourb2name (master)

$ ./gradlew initB2

:initB2

Defaults found for generating building block

> Building 0% > :initB2

What is your Vendor Id (e.g. usq, unsw, qut, swin, etc)? (No spaces) [BBS] bbs

What is your Vendor Name (e.g. University of Antarctica)? [Blackboard Support]

What is your Vendor Website (e.g. http://www.myu.edu.au/)? (No spaces) [http://community.blackboard.com/]

What is your B2 Handle (e.g. jshack, autosignon, etc)? (No spaces)  yourb2name

What is your B2 Name (e.g. Student View)? Dev Day2016         *Note I created this in January and forgot we moved on a year, its just to cold in Jan to think....

What is the base java package of your project? (e.g. au.edu.uni.myproject) ? (No spaces) com.devday

Do you want to use the course event listener? (Y/N) Y

Do you want to use the Schema.xml? (Y/N) Y

Do you want a system tool? (Y/N) Y

Do you want a course tool? (Y/N) Y

Initializing Building Block bbs-yourb2name

Initialized empty Git repository in C:/Users/ahulme/workspace/devday/bbs-yourb2name/.git/

 

BUILD SUCCESSFUL

 

Total time: 1 mins 59.188 secs

 

Import the Project into Eclipse.

 

File > Import > Gradle Project

 

Browse to the Directory your created and Select Build model.  Check the Project and finish.  This has now imported this into your work-space.

 

 

 

This is the project as it looks in Eclipse.    There are 3 stripes action classes in src > com.devday.stripes .  These control the actions for the Course Tool, System Tool and Config tool.  The JSP files for these tools can be found in src > main > webapp > WEB-INF > jsp.  The language bundles are in  src > main > webapp > WEB-INF > bundles

 

 

Build the B2.

 

Without making any changes we can Build this B2 and Deploy it to Blackboard.  This will create a course tool, a system tool, Some Database Schema objects and a Config tool for the B2.

 

Right click gradlew > run as > Gradle STS Build

Set the task to build > apply and then run

 

Console Output:

 

[sts] -----------------------------------------------------

[sts] Starting Gradle build for the following tasks:

[sts]      build

[sts] -----------------------------------------------------

:compileJavawarning: [options] bootstrap class path not set in conjunction with -source 1.7

1 warning

 

:processResources

:classes

:war

:assemble

:compileTestJava UP-TO-DATE

:processTestResources UP-TO-DATE

:testClasses UP-TO-DATE

:test UP-TO-DATE

:check UP-TO-DATE

:build

 

BUILD SUCCESSFUL

 

Total time: 1.584 secs

[sts] -----------------------------------------------------

[sts] Build finished succesfully!

[sts] Time taken: 0 min, 1 sec

[sts] -----------------------------------------------------

 

Deploy To Blackboard.

 

System admin GUI > Building Blocks > installed tools > Upload Building block

 

If you see this error in Red you will need to enable B2's to add custom objects. Building Blocks > Global Settings

 

Error: Could not install plugin.name. There are associated database changes and the global setting has been set to prevent any Building Block from creating custom database objects.

 

 

 

The war file can be found in the build\libs\ directory of the project space.

 

There will be a new tool now in the System admin panel highlighted below.

 

 

Click on this new link and you will see the tool.  As we have not configured this tool to do anything as yet a simple message will show to confirm it is working.   Note the URL used for this tool is https://Yoururl.com/webapps/bbs-yourb2name-BBLEARN/systemtool

 

 

The course tool is set by default to be unavailable courses so you can either set this globally or set on a per course bases for testing.  In a course  Customization > Tool Availability  > find the tool called Dev Day2016 and enable it.  You can then find this tool under tools and select it.

 

 

 

 

The B2 also has a config page for setting Global Configurations.  This can be access though the settings page and has buy default a String and an Integer value. We can change these and add more if required.

 

 

 

 

 

Course Tool Action.

 

This controls the Behavior when the course tool link is selected.  The URL https://finn.pd.local/webapps/bbs-yourb2name-BBLEARN/systemtool is mapped to this CourseToolActoin.java and the default action is to rerun the JSP /WEB-INF/jsp/course-tool.jsp

 

package com.devday.stripes;

import com.alltheducks.bb.stripes.BlackboardActionBeanContext;
import com.alltheducks.bb.stripes.EntitlementRestrictions;
import com.alltheducks.bb.stripes.LoginRequired;
import net.sourceforge.stripes.action.*;
import net.sourceforge.stripes.integration.spring.SpringBean;

@LoginRequired
@EntitlementRestrictions(entitlements = "bbs.yourb2name.course.VIEW", errorPage = "/error.jsp")
@UrlBinding("/coursetool")
public class CourseToolAction implements ActionBean {

    private BlackboardActionBeanContext context;

    @DefaultHandler
    public Resolution displayPage() {
        return new ForwardResolution("/WEB-INF/jsp/course-tool.jsp");
    }

    @Override
    public ActionBeanContext getContext() {
        return context;
    }

    @Override
    public void setContext(ActionBeanContext context) {
        this.context = (BlackboardActionBeanContext)context;
    }
}

 

 

A simple course Tool Project:

 

lets use our basic course tool to display some information about the course.  The FULL URL called for the tool link is https://finn.pd.local/webapps/bbs-yourb2name-BBLEARN/coursetool?course_id=_36_1  and this contain the course Pk1.  We can use this to get the course Pk1 and use the context to get the Course Title

 

Add the following to the CourseToolActoin.java on line 32 to setup our Getters and Setters.

 

private String course_title;

public String getCourse_title() {return course_title; }

public void setCourse_title(String course_title) { this.course_title = course_title; }

   

private String course_id;

public String getCourse_id() {return course_id; }

public void setCourse_id(String course_id) { this.course_id = course_id; }

 

The course Pk1 will be automatically mapped to the action bean course_id as this is provided in the URL.  We can set the Course Title using the context Stripes Action Bean.

 

package com.devday.stripes;

import com.alltheducks.bb.stripes.BlackboardActionBeanContext;
import com.alltheducks.bb.stripes.EntitlementRestrictions;
import com.alltheducks.bb.stripes.LoginRequired;
import net.sourceforge.stripes.action.*;
import net.sourceforge.stripes.integration.spring.SpringBean;

@LoginRequired
@EntitlementRestrictions(entitlements = "bbs.yourb2name.course.VIEW", errorPage = "/error.jsp")
@UrlBinding("/coursetool")
public class CourseToolAction implements ActionBean {

    private BlackboardActionBeanContext context;

    @DefaultHandler
    public Resolution displayPage() {
        
        //Set the actionbean.course_id to the  Course Title from the context
        course_title = context.getCourse().getTitle();
        
        
        return new ForwardResolution("/WEB-INF/jsp/course-tool.jsp");
    }

    @Override
    public ActionBeanContext getContext() {
        return context;
    }

    @Override
    public void setContext(ActionBeanContext context) {
        this.context = (BlackboardActionBeanContext)context;
    }
    
    
    private String course_title; 
       public String getCourse_title() {return course_title; }
    public void setCourse_title(String course_title) { this.course_title = course_title; }
    
    private String course_id; 
       public String getCourse_id() {return course_id; }
    public void setCourse_id(String course_id) { this.course_id = course_id; }
    
}


 

in the course_tools.jsp add in the actionBeans to display the Course Title and Pk1

 

 

mkauffman

Upgrade Your DVM

Posted by mkauffman Dec 28, 2016

Scott Hurrey mentioned this the other day during our Technical Office hours... You can upgrade a 3000.x DVM using the same installer package that you use for a self-hosted system. This blog post will show you how easy that is, after a brief plug for our office hours. If you've not attended, office hours a great way to get answers to your questions and to network with other developers. The schedule and link to join are on the upper right of this page.

 

Prequel: Before doing following, you may need to upgrade the DVMs Java version. See the required technologies for self-hosted environments for the release you are installing on the help.blackboard.com pages. Post Q4 2018/3500  you will also need to upgrade Postgres to 9.6. It's beyond the scope of this document to explain how to upgrade Postgres from 9.4 or 9.5 to 9.6. It's a standard Ubuntu install. Google on +upgrade +postgresql +9.5 to +9.6 +ubuntu +16.04 and that will get you close the right pages.

 

Here's what you do to upgrade your DVM.You will do all of your work as the vagrant user. Do not use root or bbuser. (Note: See addendum following these instructions for Q4 2018 and later for the resolution to the Access Denied dialog on the login page.)

  1. Stop Learn
  2. Download the installer for the version you want to upgrade to from Behind the Blackboard. Note this only works for Q2 2016 and later.
    1. For this example, I upgraded my DVM from 3000.1.1 to 3000.1.3 (Q2 2016 CU1 to Q2 2016 CU3)
  3. As the vagrant user, create the recommended installation directories.
    1. mkdir /usr/local/bbinstaller
    2. mkdir /usr/local/bbinstaller/3000_1_3/
  4. Create an installer.properties file. I've placed the full text for one that works for a DVM below.
    1. Place the installer.properties file in the /usr/local/bbinstaller directory.
  5. Make a copy of the license file. You MUST do this because the install will mess everything up if the installer.properties is pointing to a license file inside the blackboard directory that is being upgraded.
    1. cp /usr/local/blackboard/config/license/blackboard-license.xml /usr/local/blackboard-license-copy.xml
    2. chown vagrant /usr/local/blackboard-license-copy.xml
  6. Move the downloaded .zip installer file to the /usr/local/bbinstaller/3000_1_3/ directory
    1. The trick here is that on the DVM the /vagrant directory is the same directory as the host directory where you ran the vagrant up and vagrant ssh command. You place the .zip installer file into that directory, then on the DVM guest you can move it from into the installation directory.
  7. cd /usr/local/bbinstaller/3000_1_3/
  8. unzip learn-installer-3000.1.3-rel.70+214db31.zip
  9. Run the upgrade
    1. ./installer.sh -c /usr/local/bbinstaller/installer.properties
  10. Run PushConfig, which will also start the server.
      1. cd /usr/local/blackboard/tools/admin
      2. ./PushConfigUpdates.sh

 

That's it! Below are the contents of the installer.properties file that works for upgrading a DVM:

 

# DVM installer.properties for upgrade. First attempt was from Q2 2016 CU1 to CU3.

# Kauffman 2016.12.28

bbconfig.basedir=/usr/local/blackboard

bbconfig.java.home=/usr/lib/jvm/java-8-oracle

 

# cp /usr/local/blackboard/config/license/blackboard-license.xml /usr/local/blackboard-license-copy.xml

# chown bbuser /usr/local/blackboard-license-copy.xml

bbconfig.file.license=/usr/local/blackboard-license-copy.xml

 

## these properties should not be modified manually ##

antargs.default.vi.db.name=BBLEARN

antargs.default.vi.stats.db.name=BBLEARN_stats

 

# just like the Windows upgrade, the Linux upgrade fails if you don't have these 3 dummy antarg lines

antargs.default.users.administrator.password=passworddoesnotmatter

antargs.default.users.integration.password=passworddoesnotmatter

antargs.default.users.rootadmin.password=passworddoesnotmatter

 

############## properties listed here can be modified ###############

antargs.default.vi.db.password=postgres

antargs.default.vi.stats.db.password=postgres

antargs.default.vi.report.user.password=password

 

Addendum for Q4 2018 and Later:

With Q4 2018 and later the bb-cookie-disclosure makes it impossible to login. so you have to upgrade through the PushConfigUpdates as described above. Check that the system is up and running and that you do get stuck because of the cookie disclosure issue - you see Access Denied on the login page. Then use the ServiceController.sh to stop Learn,

cd /usr/local/blackboard/cache/plugins

rm -rf bb-cookie-disclosure

cd /usr/local/blackboard/content/vi/BBLEARN/plugins

rm -rf bb-cookie-disclosure

and finally use PushConfigUpdates.sh to start Learn. Ignore the errors in the stdout-stderr log you get because of the missing bb-cookie-disclosure plugin. At this point, I was able to login and use my upgraded DVM.

 

NOTE: If you use vagrant destroy to stop the system you will need to repeat this process. Using vagrant halt allows you to stop and start the DVM without doing so. However, with halt you may run into other odd behavior like the DVM hangs for a very long time without stopping, etc. I've had to fiddle with the machine when using halt, but have been able to keep using the same box. It's your choice on whether to use destroy and just know that you need to remove the bb-cookie-disclosure every time, our use halt and fiddle.

Overview

The goal of this blog is to provide a kind of walk-through of the steps needed to build the Blackboard Learn .NET web services library and to construct a very basic and simple application (client) through which to pull and display user information to the console.

 

Provided will be a few base classes which can be used as a template to expand this walk-through into a shareable code-base for more advanced and complex web services clients. These files can be found on GitHub here.

 

What's Needed

  1. Microsoft Visual Studio 2012 or later (earlier versions may be compatible with this walk-through and the build process; I used Visual Studio 2015 Community Edition)
  2. Microsoft Web Services Enhancements (WSE) 2.0 SP3 or greater Download Web Services Enhancements (WSE) 3.0 for Microsoft .NET from Official Microsoft Download Center

 

Resources/Links

Tutorial: Build The C# SOAP Sample Library

GitHub - allenvanderlinde/wsclient: Base classes to accompany .NET Learn Web Services Client walk-through.

 

Building the Library

To begin we'll need to download the prepackaged web services library ZIP file which can be pulled from any Learn environment via the System Admin panel.

 

System Admin > Building Blocks module > Building Blocks > Proxy Tools > Download Sample Tools

 

You'll find within the ZIP several folders including "dotnetclient". From this folder we'll run through the process of building the Learn web services library we'll be using in our basic tool's solution. You will also notice a folder entitled "javaclient". We'll be focusing on .NET for the entirety of this walk-through, however there are several articles and discussions which also go over building the library for Java developers.

 

Following the original tutorial Tutorial: Build The C# SOAP Sample Library if you find that not all .cs files are generated, run sampleGenClient.cmd several times to make sure the necessary sources are pulled from the WSDLs as needed. You'll want to make certain you have the following in your automatically generated BbWsClient/gen folder:

 

AnnouncementWS.cs

CalendarWS.cs

ContentWS.cs

ContextWS.cs

CourseMembershipWS.cs

CourseWS.cs

GradebookWS.cs

NotificationDistributorOperationsWS.cs

UserWS.cs

UtilWS.cs

 

Note: The original tutorial also mentions how to modify the above web services source files to allow your client to run on load-balanced environments. For testing purposes on a single application server, excluding the load-balanced code will be fine.

 

When each source file is pulled that you'd like, go ahead and build the Visual Studio solution found in the BbWsClient folder. This will produce a .DLL that will be included in the actual source code of our tool.

 

 

Web Services Requests At-A-Glance

Essentially the goal of our tool for the purposes of this walk-through is to reach out remotely to a Learn environment, send a request for a single record or more of user information, store it, and display it in a meaningful way. We'll also be calling read-only methods this time around. A more technical, but still basic breakdown of this process could be seen as follows:

 

  1. Client establishes connection to remote environment.
  2. Client requests to log in.
  3. Tool registration or remote-user account existence is checked before allowing access.
  4. Entitlements are requested and stored via tool registration or pulled from level of permissions when logging in as a user.
    • If previously registered, tool entitlements are included within raw SOAP requests.
  5. The actual request is sent.
  6. The Learn server responds with the requested data through populating certain VOs (e.g., UserVO[] for users, CourseVO[] for courses) which are stored in memory on the client PC.
  7. The client sends request to log out.
  8. The established connection is safely broken with successful log-out.

 

We'll be using the same methods found in the web services base classes you can find here. This walk-through will go over just the bare-minimum needed to make a successful request and leave it up to you to integrate any additional features.

 

 

Saying Hello...

To begin, we want to 1) establish a connection to the remote Learn environment, 2) identify ourselves, and 3) register. This walk-through focuses on the web services client being a proxy tool as opposed to impersonating an admin-level user through which to make our user information request(s).

 

It is possible to produce a proxy tool record which will be visible in your System Admin panel should you simply try to connect to a web service. Doing so is not recommended unless it's a first step in testing whether your tool can get out a connection request in the first place.

 

Note: Simply hitting the server as a proxy tool without proper registration will deny any later connections entitlements since they must specifically be requested during the registration process.

 

Our first step is to write a method that reaches out to the server, establishes a connection, and generates a session. It is entirely up to you how to handle variables for your own tool, and it is never recommended to pass plaintext passwords into an application. Learn's web services handles the password nicely over-the-wire since without an SSL-encrypted connection, the server will never allow requests to get through from the client (no trust relationship will be established).

 

To make this step quick-and-easy for us for the walk-through, we'll be assuming that we've taken the host, a password for the tool, the word "register", and the global password required as configured in "Manage Global Properties" in your Proxy Tool System Admin area, from the command-line.

 

>wsclient https://mybbenv.blackboard.com toolPassword register globalPassword

 

Learn's web services calls handle a great deal of the grunt work in the background, so for us, registering our tool is fairly straightforward.

 

1. Connecting.

Notes:

a. WS is a WebserviceWrapper object.

b. vendor and tool are some arbitrary strings.

c. lifetime has a value of 1 hour.

 

public static bool connectAndInitialize(string _host)
{
    host = _host;
    try
    {
        // Instiantiate the primary web services wrapper
        WS = new WebserviceWrapper(host, vendor, tool, lifetime);
    }
    catch(Exception e)
    {
        Console.WriteLine("Error occurred: {0}", e.Message);

        return false;
    }

    // Initialize the primary web services object
    WS.initialize_v1();

    if(WS.debugInfo().Equals("NO session"))
    {
        Console.WriteLine("Error: Unable to generate session.");

        return false;
    }

    sessionId = WS.debugInfo();
    Console.WriteLine("Session generated: {0}", sessionId);

    return true;
}

 

What we're doing here is building our own WebserviceWrapper object with our host, some vendor and tool name for the web services client, and an expected lifetime of 1 hour. initialize_v1() does all the magic for us handling the actual connection. Our goal is to have the services give us back a session ID string. Checking that that string doesn't equal "NO session" means we've made a connection.

 

Referring to Program.cs of the wsclient project, we use a static class to run certain web services tasks with, so we could call the above with:

 

bool success = WebServices.connectAndInitialize(host);

 

Our host address is the only thing passed in for example's sake. Our tool and vendor strings and our lifetime are constant identifiers in the WebServices class in this project for convenience.

 

At this point, with no failure, we've established a connection to the environment's web services and made the system aware that we'll be trying to pull data soon. Our next step is to register, which will allow us to specify which entitlements we'll be using for the tool so that we can pull the data we're looking for.

 

2. Registering.

Notes:

a. We're assuming we've already checked that the third argument string to wsclient is "register".

b. "This is my web services tool." is an arbitrary description of the tool we're registering (required for registration).

c. sharedSecret is the password gathered from the command line we want to use for the tool.

b. WebServices.logout() is also used to safely break the connection and check for any issues regardless whether our tool is actually logged in.

 

try
{
    if(!WebServices.register("This is my web services tool.", args[3], sharedSecret))
    {
        Console.WriteLine("\nUnable to register tool in host {0}.\n\tMore info: {1}\n\t*Global password may be invalid.", host, WebServices.getWS().getLastError());

        return -1;
    }
    else
    {
        Console.WriteLine("\nRegistered tool successfully!");
        WebServices.logout();

        return 0;
    }
}

 

Here we're assuming that the string "register" was passed into wsclient as the third argument, and we pass the global registration password into the WebServices.register() method as args[3]. The only reason for this is to specify to our tool that we wish to register a new proxy tool connection. Future runs of wsclient will involve just the host, tool password, and user ID of the user we'd like basic information on. Again, it's up to you to determine how you'd like your tool to receive input for logging in and requesting data.

 

Note: Once a proxy tool is registered, it will need to be set to "Available" in the GUI after it appears in the Proxy Tools list in your System Admin > Building Blocks module > Building Blocks > Proxy Tools page. This is also a good opportunity to make sure the requested entitlements were requested successfully as they'll appear as a list on the same "Edit" page for the proxy tool.

 

We're going to take a closer look at what WebServices.register() actually does as that's where our direct Blackboard web services calls takes place for this walk-through.

 

Notes:

a. toolRegResult is a Learn RegisterToolResultVO object that stores information about our registration previously declared as a private member of the WebServices class.

b. initialSecret is the "global registration password" found in System Admin > Building Blocks module > Building Blocks > Proxy Tools > Manage Global Properties.

c. sharedSecret is the password that will be used for the tool.

d. entitlements is the string array of specific Learn identifiers which represent the actions we want to perform on the data we're trying to pull. (Details follow example.)

 

public static bool register(string desc, string initialSecret, string sharedSecret)
{
    toolRegResult = WS.registerTool(desc, initialSecret, sharedSecret, entitlements, null);

    if(toolRegResult.status)
    {
        return true;
    }
    else
        return false;
}

 

Here we simply call the WebserviceWrapper object's registerTool() method which takes a set of its own arguments. Since we're using a proxy tool as the medium through which to pull our user data, we're passing in null for the final parameter of registerTool().

 

Afterwards we check that our RegisterToolResultVO object has a status of true to confirm that our tool was registered without issue.

 

Note: Don't forget to make your registered tool "available" in the GUI on the Proxy Tools page.

 

3. Entitlements.

In order to fully register a tool certain entitlements, or privileges, must be requested alongside the registration. These are identifiers which Learn recognizes as permissible actions when pulling data from its web services. Certain web services calls, whether reading data or writing data, will fail should the necessary entitlement not be a part of the requesting tool's registration. They are fairly self-explanatory, so I'll simply list what this walk-through uses below. A fuller list can be found in the comments at Proxy Tools

 

{
    "Context.WS:emulateUser", "Context.WS:logout", "Context.WS:getMemberships",
    "Context.WS:getMyMemberships", "Util.WS:checkEntitlement", "User.WS:getServerVersion",
    "User.WS:initializeUserWS","User.WS:saveUser", "User.WS:getUser",
    "User.WS:deleteUser", "User.WS:saveObserverAssociation", "User.WS:getObservee",
    "User.WS:deleteAddressBookEntry", "User.WS:getAddressBookEntry", "User.WS:saveAddressBookEntry"
}

 

Note: Make sure that your Util.WS web service is available and that you initialize it in future requests so that data sources can be exposed.

 

 

Pulling Data

Now that our tool is registered and our entitlements have been granted, we can actually request a user's account data. The process is very straightforward and almost identical for all "read" methods with Learn's web services.

 

1. Build a new filter.

2. Define filter properties.

3. Specify the "thing" you're looking for based upon filter type.

4. Build the necessary VO to hold response data.

5. Do something with your data.

 

With a bit more pseudocode, we'll go through these steps and then discuss any details.

 

Notes:

a. RequestType is a custom enumeration which has identical values for constants as those found in your web services documentation.

b. GET_USER_BY_NAME_WITH_AVAILABILITY indicates to pull a user or users by their username (as opposed to PK1).

 

            uf = new UserFilter();
            uf.filterType = (int)RequestType.GET_USER_BY_NAME_WITH_AVAILABILITY;
            uf.filterTypeSpecified = true;

            /* Since we're only looking for 1 user's information,
             * we instantiate index 0 of the string[] for the filter. */
            try
            {
                uf.name = new string[] { userId };
            }
            catch(NullReferenceException)
            {
                return false;
            }

            /* Build the UserVO[] with information about
             * the requested user. */
            try
            {
                users = WebServices.getUserWS().getUser(uf);
                if(users == null)   // Most likely services aren't available for the tool
                    throw new NullReferenceException();
            }
            catch(NullReferenceException)
            {
                return false;
            }

            return true;

 

Ignoring basic exception handling, what were doing is using our UserFilter object to store criteria we want to pull data on. The member filterType is vital as that lets Learn known how we're requesting the data (e.g., in this case, by username).

 

The UserFilter class also has a member name and member id (among others). name is a string array we populate with usernames we want account information on. id is also a string array but refers to the user's PK1 and is in the format "_XXXXXX_1".

 

Recollecting that WebServices is a static utility class for this walk-through, we see that it's really just getting a handle to the UserWrapper object member. We pass in our filter which defines the "things" we want data on and have Learn's web services build our UserVO[] with the request's results (e.g., users).

 

 

Look at this!

I hope this walk-through has been a useful resource for you in your Learn web services journey and hopefully has provided you with some additional insight into how the system works.

 

At this point we've written a tool which connects to Learn's web services, registers itself, and pulls some user data. From here we can manipulate that data any way we'd like by referring to an index in the UserVO array and specifying the member we're interested in (i.e., users[0].userBatchUid).

 

When we know what we're looking for, we could format the strings as needed to even write a feed file....

 

But I'll leave that to you.

Hi,

 

I have recorded a video tutorial about how to use the Blackboard Learn Web Services Framework. I wanted to be language agnostic so I have used SoapUI.

 

You will learn how to setup your Blackboard Learn environment to let an external tool to register, login and consume the SOAP webservices.

 

In this video we will get all the users with default system role. To this we will get a session, register a proxy tool with the required entitlements, login this tool and create the desired filter and the request to obtain the users.

 

Sorry for my english, it is not very "dynamic", but I think you can understand almost everything with the screen sharing.

 

mkauffman

Simple SaaS Logging Tips

Posted by mkauffman Sep 20, 2016

A recent conversation with a Partner was the impetus for this blog post. We have documentation here on logging here Preparing Your Building Blocks For Learn SaaS. But the Partner asked "In the old days, I would use System.out.println and see the data in stderr-stdout.  Can you suggest how to do something as simple as what I want?" If you long for the good old days there's good news! You can still write log entries to the tomcat stdout-stderr file and then view them in Kibana.

 

Here is an example. I've added the following to a B2's home controller:   System.out.println("springmvcb2 - Is this visible in Kibana? I didn't intentionally log it in the correct format."); Here is a video showing how to find that log entry in Kibana: http://screencast.com/t/tu1m9RO6vNKZ

(For some reason the link to the screencast above is given an odd name by Jive. Ignore that, and click the link.)

 

The code for the B2 is here GitHub - mark-b-kauffman/springmvcb2: demo spring controller to handle requests and copy handler. You will also find in this code example how to set up log4j as discussed here Preparing Your Building Blocks For Learn SaaS. In the code from github you see this line: logger.error("There was NOT an error. This is a test message."); This creates an entry that you can find using Kibana as shown here: http://screencast.com/t/bkFrnVw1PBVG

 

Another good recording on using Kibana to look at logs:   https://www.screencast.com/t/tdw9JCXlzeYF

 

Happy Coding!

 

MBK

Hi,

 

I read article 41101 following a bulletin but realise that it did not address the main issue related to the weak default DH ciphers / Diffie-Hellman key.

Basically the problems with Blackboard 9.1 Oct 14 and 9.1 Q4 15 boils down to:

a) Java 7 is the certified JDK for these 2 version

b) Java 7 uses 768-bit for DH by default

c) Java 7 has limited strength policy so AES is maxed out at 128 bits

 

The following articles I found online (especially the first) helped me a lot in understanding why my security team was harping on this when the cipher list in article 41639 & 40766 has AES256, DH, DHE ciphers. The articles pointed out that I needed to have:

1) “-Djdk.tls.ephemeralDHKeySize=2048” under bbconfig.jvm.options.extra.tomcat”

2) Apply JCE unlimited strength policy so that I could use AES256 cipher and that security team will not complain about the DH cipher being weak.

 

Reference:

https://blog.eveoh.nl/2014/02/tls-ssl-ciphers-pfs-tomcat/

http://suhothayan.blogspot.sg/2012/05/how-to-install-java-cryptography.html

 

Hope this helps someone here.

 

Regards,

Lum

Here you are a video with the process to disable and re-enable users using SIS Flat Files. It is an easy way to keep low the number of users that count to license while you keep all the user data for historic or audits purpose : http://1428242.mediaspace.kaltura.com/media/t/1_wftw3en3

 

The video is in spanish but closed captions are provided in english.

 

Hope this help.