44 Replies Latest reply on Apr 21, 2017 12:57 PM by md0045412

    Enterprise Surveys

    renrodee

      Our school is starting to utilize the Enterprise Survey tool for End of Course surveys at a departmental and | or divisional level. I am interested in discussing successes and failures other schools have had using this tool. Best practices, Do'es and Don'ts, Tips for Success etc. are also of interest

        • Re: Enterprise Surveys
          md0045412

          Ren,

           

          I assisted our largest school at our college convert to Enterprise Surveys. Previously, some of the departments in the school were using it and they may have been using different set of questions or creating separate surveys for each course instead of using response periods. It's worked out fairly well for some things such as consistency and cutting down on the manual labor of using paper evals or scantrons. Use Response Periods, it will save a lot of time.

           

          Group training of Survey Authors (possibly individual follow-ups) is vital if you're having the depts create the surveys. Make sure you know what the bugs are (or what Bb determines as working as designed). For example, you can't use the large setting in open-ended questions (survey hyperlink becomes inoperable, the pie chart on the individual reports are inaccurate (Bb stated working as designed) - it reflects the whole response period but it appears to be for the individual person so it's inaccurate or completely wrong. I believe we had an issue where the report feature wouldn't work so we had to get a query to fix the database. After applying an annual Bb Q2 or Q4 there's a known issue where the hyperlinks won't work and you have to make some modifications.

           

          The Send to All is the default so we've had issues where surveys were sent to all users in Bb accidentally. Yes, user-error but easy enough to do since the interface isn't the most user-friendly. I asked Bb if the default could be changed but they said no (I figured they could change the data-base setting from a 1 to a 0). There's also a bug that even if you select the appropriate people/courses and click on one of the tabs at a later date to review your settings, it reverts back to the default Send to All.

           

          I recommend utilizing the email option, reminders every 24/48 hours AFTER survey deployment to increase response rates and to give class time to complete the evaluations. I also like the option to release survey results after grades close so that instructors can get their own results.

          We've had some success converting over to electronic evals and it's an improvement over the paper copies. The response rates were steadily climbing as the students got used to taking them. With that said, we've had to deal with a lot of issues and bugs. It's a free add-on so you get what you pay for. It can be useful but we're investigating other options at this point.

           

          Mike

          1 of 1 people found this helpful
            • Re: Enterprise Surveys
              renrodee

              Mike,

               

              Thanks so much for the information. We are still in the adaptive stage of enterprise surveys, but after 10 years of availability we are starting to get some traction. I stumbled across the response period solution as opposed to a new survey every semester so we will see how that roles. We also use the response periods for less than full term classes so we will see how that works. Any direction in that area would be appreciated.

               

              Have run across the pie chart issue and loved the phrase “working as designed” Some of the other bugs will be revealed as we move forward. My default statement is it’s not the greatest thing since sliced bread so if you have a better way to gather the date go for it. No one has gone back to paper and pencil yet.

               

              Thanks again

               

              Ren

              • Re: Enterprise Surveys
                gs0044169

                Mike,

                 

                We are using Enterprise Survey and have just updated to Q2 2016 and when we build a response period that has 7-10 users with valid emails it only sends to three and then say the others are invalid. Do you have any idea why this might be happening?

                 

                Thanks,

                Grant

                  • Re: Enterprise Surveys
                    md0045412

                    Grant,

                     

                    I don't think I ever ran into that issue. How were the recipients selected? Did you select the courses in the response period? If that's the case, it should be emailed to their email listed in their Blackboard account. Sounds like those emails would be incorrect but sounds like you already checked for validity.

                     

                    There's also an option to select by users or another option is to upload email addresses via text file. Are you using one of these other methods?

                    This wouldn't affect your issue but depending on who your hosting provider is, you might want to white-list that email address so that the survey invitations don't go into the students spam/junk folder.

                     

                    I just launched a pilot using EvaluationKIT. Hoping to transition from Enterprise Surveys soo.

                     

                    Mike

                • Re: Enterprise Surveys
                  julie.mulvey

                  Been using Enterprise Surveys since it was released.  Yes its better than paper.  Needs to be "carefully" used to say the least.  Yes its free.  Its basic.  But there are better products out there that cost but will give you a much better return on investment in terms of what you can do with the data.  I'm definitely looking.

                  1 of 1 people found this helpful
                  • Re: Enterprise Surveys
                    julie.mulvey

                    We have evaluated Explorance Blue rather than EvaluationKIT and am hoping the positive pilot leads to us buying this.  Enterprise Surveys will meet some needs but if an institution is really serious about data then using it at scale then it needs a more robust solution.

                      • Re: Enterprise Surveys
                        md0045412

                        We looked into Explorance also. They were a worthy vendor but their price-point was considerably higher. Comparing the vendors, couldn't really justify the increased cost for the benefits.

                        Hoping our pilot works out too. Good luck!

                          • Re: Enterprise Surveys
                            renrodee

                            So is your pilot based on the system Enterprise Survey tool?

                              • Re: Enterprise Surveys
                                md0045412

                                Pilot is based on EaluationKIT. They integrate the product with your LMS and allow you to use their product at no cost for three months and you can decide if you would like to sign a loner-term contract.

                                 

                                We piloted Enterprise Surveys about a year and a half ago. We have our largest school using it now with multiple other departments coming on board. With the increased use, more bugs became apparent. Hoping the use of Enterprise Surveys will help the transition to EvaluationKIT. 

                                  • Re: Enterprise Surveys
                                    renrodee

                                    More Thanks Michael. We are also rolling it out a piece at a time. Just found a bug (?) where survey authors cannot select individual students in a response period . Am waiting for Bb’s response.

                                     

                                    Any recommendation on how to use point values in multiple choice question when used to “select all that apply” Right know our results show Overall Average of 0.

                                     

                                    Ren

                                      • Re: Enterprise Surveys
                                        julie.mulvey

                                        I’ve never added students to the institutional hierarchy to be able to add them to Response periods.

                                        If anyone has wanted to deliver a survey to specific students – we have just used the upload a .txt file with the student’s email addresses listed.

                                         

                                        As for the points values – are you putting zeros in the point fields.  Blackboard treats these as a negative and I would recommend leaving the point fields blank.

                                         

                                        Thanks

                                        Julie

                                         

                                        Julie Mulvey  BA(Hons), MSc, FHEA, CMALT

                                        Learning Technologist (Social Science & Health inc Queen’s Campus)

                                        Durham University

                                        Computing and Information Services

                                        Rowan House, Upper Mountjoy, South Rd, Durham, DH1 3LE

                                         

                                        T: +44 (0)191 334 2781/0286

                                        E: julie.mulvey@durham.ac.uk<mailto:julie.mulvey@durham.ac.uk>

                                         

                                        www.durham.ac.uk/cis<http://www.durham.ac.uk/cis>

                                         

                                        Durham University is a world top 100 university.

                                         

                                        <http://www.durham.ac.uk/nwp>

                                         

                                         

                                        The information in this e-mail and any attachments is confidential. It is intended solely for the addressee or addressees. If you are not the intended recipient please delete the message and any attachments and notify the sender of misdelivery. Any use or disclosure of the contents of either is unauthorised and may be unlawful. This e-mail has been created in the knowledge that Internet e-mail is not a 100% secure communications medium. We advise that you understand and observe this lack of security when e-mailing us. Although steps have been taken to ensure that this e-mail and any attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free. All liability for viruses is excluded to the fullest extent permitted by law.

                                          • Re: Enterprise Surveys
                                            renrodee

                                            Julie,

                                            In this specific case all options were valued at 1 point, see below. The result was an "Overall Average 0%" display. Are we using the wrong question type? Would a matrix question be better suited?

                                             

                                            On the select by user it was a small subset of class and seemed the easiest approach. To generate the txt file you must of had to search for the specific students at some point. Six of one, 1/2 a dozen of the other.   [Edit Response Period -> Select Recipients -> Choose Users -> Send to Specific Users -> Select users -> email: contains: schoolemailaddress.edu]

                                          • Re: Enterprise Surveys
                                            md0045412

                                            There was a bug where I would leave the points field blank but Bb would automatically enter in the 0. It gave us false data. Told it was a known issue. Not sure if it's been fixed yet. Get used to being told that it's working as designed. I've heard that so many times.

                                            I usually give the highest point value to the best choice and work down to 1 point.

                                             

                                            If you're using a multiple select (matrix), you don't need to use points unless you're trying to rank them.

                                            No points:

                                            Question: Previous to enrolling....what did you experience (select all that apply)?

                                            Answers: Email (0 pts), letter (0 pts), Internet Ad (0 pts).

                                             

                                            Points:

                                            Rate the factors that contributed to.....

                                            Row options:

                                            Phone call

                                            Open House

                                            Cost

                                             

                                            Column Options:

                                            Not very important (1 pt)

                                            Important (2 pts)

                                            Not important (1 pt)

                                            N/A (0 pts)

                                             

                                            How you design the question will determine the data you receive.

                                             

                                            Also, I've used the txt file method to send the surveys to the emails but if you only have a few students you can search by user too.

                                            Keep in mind that both methods are a manual option and you would need to create individual response periods for each course.

                                             

                                            I prefer to create a Spring Response Period for each department and search for all of the Spring Courses and add them all at once.

                                            Since our SIS integrates the enrollment, it automatically sends the email survey invitation to each student in each course selected. This can all be completed in under 15 minutes.

                                      • Re: Enterprise Surveys
                                        julie.mulvey

                                        EvaluationKIT doesn't really have any traction here in the UK.  eXplorance is starting to sign institutions up now.  We had a lot of considerations about where the data is stored as well so a Canadian hosted solution was much more a preferred provider than a US based one.   When I was a BBWorld a couple of years ago - I ran a Enterprise Survey User Group with Jason Rhode.  A lot of the new Enterprise Survey users were EvaluationKIT users who jumped because of the cost per survey (their words not mine). 

                                    • Re: Enterprise Surveys
                                      renrodee

                                      How does the Enterprise Survey tool tie into the Analytics tools available from Blackboard?

                                      • Re: Enterprise Surveys
                                        julie.mulvey

                                        Its the N/A option that added the 0 points instead of a null value.

                                        However they do disregard the point values and calculate the percentage in their own way.

                                        Malcolm Murray who I work with wrote a guide about how Blackboard calculates the percentages within Enterprise Surveys.

                                        We here also need mean values so we needed a method to help us convert the figures.

                                        Here's a link to Malcolm's guide:  http://community.dur.ac.uk/lt.team/?kbe_knowledgebase=calculating-percentages

                                        Hope you find it of use.

                                        • Re: Enterprise Surveys
                                          renrodee

                                          Part of what we (I) like about the survey is we (I) don't have to create, distribute, collate etc. We work with the departments and put them in charge of their own destiny.

                                          • Re: Enterprise Surveys
                                            hijazi.stephanie

                                            Greetings!

                                            I wanted to add our experience with Enterprise Surveys which our Nursing School is now using for all of their courses and instructor reviews. Our experience has been mostly that you get what you pay for... a number of bugs and "working as designed".  We are MH currently on April 2014 version.

                                            Surveys are sent out at the end of the semester anonymously to Nursing students - there can be several surveys for the same class. So for each survey, the designer will create a separate response period and then choose the different classes it should be sent to. They also use the email system to send out emails every 48 hours until the end of the survey (or when the survey is taken).

                                            A couple of glitches:

                                            - I have yet to find a way to survey two or more instructors in a course using our method ..  many courses are team taught and students when doing the instructor evaluation can't tell who they are evaluating as it's survey based (ie. one generic survey across many courses).

                                            - we had an issue with the survey email not being sent with the proper information in the title  - I can't remember exactly what the issue was but Blackboard had to fix it.

                                            - a designer accidentally sent the survey to everyone in every course on Blackboard .... (send to all option).  That was not a fun day. Every survey release requires me to go in and check every single response period to make sure things are correct.

                                            - the results (as previously pointed out) are off when releasing to instructors. The department has had to manually download and do all of their calculations in Excel.

                                            - emails were being sent to students that had dropped the class (students who drop during the semester are dropped not disabled). This was a huge issue as data from previous semesters was compromised. BB has said they have fixed the issue... but still ... not good.

                                             

                                            We, as others, are trying to get a better, more robust platform for surveys .. but this works for now if used with caution.

                                            • Re: Enterprise Surveys
                                              julie.mulvey

                                              We wanted to achieve sampling.  So with 6 teachers on a course with a cohort of 120 students we then got a list of email addresses of all students and split them up creating a response period using a subset of email addresses in a txt file and so on.  This was to avoid survey fatigue - we wouldn't ask them to redo the survey for each teacher.

                                              1 of 1 people found this helpful
                                              • Re: Enterprise Surveys
                                                ia37789

                                                We have been using enterprise survey's for a few semesters for the course evaluations for the whole campus. I use institutional hierarchy's business units to organize the courses by department (science, business, engineering, etc.) I then load the business units into their own response period. Easier to analyze the results for a department.

                                                 

                                                I would recommend making one survey forever, and just add new response periods every semester. This allows you to analyze data across semester from an admin. perspective. If you create a new survey every semester or year, you cannot analyze results between surveys, using Blackboard's built in analytics. You would have to export the data and analyze it another way.

                                                 

                                                Limitations:

                                                • Faculty cannot compare results between semesters.
                                                • Department chairs cannot get a dashboard to view their department analytics, someone with admin rights has to pull this data for them.
                                                • You can easily see how many surveys went out and how many have been completed, while the survey is live, per response period. But you cannot see this data easily on a course by course basis. This takes work. 
                                                • Our response rates are down. We want faculty still taking class time to have students fill out the surveys but we do not think many are doing this.
                                                  • Re: Enterprise Surveys
                                                    md0045412

                                                    Ian,

                                                     

                                                    We're piloting the use of EvaluationKIT to deploy the course surveys.

                                                    It's been pretty impressive so far. The dashboard gives live updates on the response rates broken down by the day.  There's an option to see the live response rates broken down by course. We had a low response rate on some courses and realized it was our program for high school students.

                                                     

                                                    You can set up the rights so chairs/secretaries can only see their departments (using hierarchy mapping). You can set up a role for the Dean so they can see their whole school. Reports can automatically be emailed to the faculty, comments can be entered in the reports, and the report is an easy to understand layout. You can flag certain results if they're below a benchmark and generate reports based on selected questions, courses, instructors, etc.

                                                     

                                                    There's an option for team-taught courses where students can answer for each instructor. You can also create the survey so that there's campus-wide questions and then departments or instructors can add their own. There's options for branching questions too.

                                                     

                                                    Lots of features and I'm still learning but so far so good.

                                                     

                                                    Mike

                                                    • Re: Enterprise Surveys
                                                      peter.rognrud

                                                      Hi Ian,

                                                      We handle our Enterprise Surveys a lot like you do, but we have several surveys based on the the course delivery method and the date range of the course. We have 5 week, 8 week and 16 week courses, with Online, Blended, and web-enhanced delivery. We also tailor the surveys for courses with labs.

                                                       

                                                      I also recommend making the one set of surveys and then creating response periods within those to help with evaluating them term to term.

                                                       

                                                      Our response rate have actually gone up quite a bit since we started using Enterprise Surveys. It really helps that we have 100% Blackboard adoption across our campus. When I deliver the surveys, I make sure that they are sent via email and in the courses as well. That way they also show up on the Home Page of the courses as well as in their email. I also schedule a reminder at the mid point of the survey. We also encourage students to download and use the Bb Student mobile app. That makes it really easy for them to take the survey on the go.

                                                       

                                                      There are limitations with Bb Enterprise Surveys, but with the budget crunches that we are all experiencing, our IRR, Deans, Division Chairs, and faculty seem to be pretty happy with the results we are getting without having to pay for a third-party survey tool. I also use the Institutional Hierarchy to separate the courses into their respective divisions and that helps to deliver the relative results to the right people.

                                                       

                                                      -Pete

                                                      1 of 1 people found this helpful