knight.jonathan

Q2 2018 CU10 patch

Discussion created by knight.jonathan on Jun 7, 2019

I did the security update last night on our production system moving from CU6 to CU10 on our Q2 2018 installation on CentOS 6.

 

Normally these are dead easy and take 40 minutes or so (my best time is 25 minutes).  This time I got an error on the installation and it took a few attempts to get the installer to run.  In all it took 90 minutes to install the patch so I thought I'd post the resolution here before I pop a few calls into Blackboard.

 

I hit two bugs, one was a well known one and one was new (at least to me).

 

The first is the permissions on the log area so I have this as a standard thing to do:

chown -R bbuser:bbuser /usr/local/blackboard/logs/update-tools

The second is a new one and only affects people using "Grades Journey".

 

We've recently started using Grades Journey using the flat file integration.  What I hadn't spotted (any why should I) was that the file containing the list of assessment columns to create is kept here:

/usr/local/blackboard/content/vi/BBLEARN/plugins/bbgs-GradeJourney/config/archive

which is worrying as I don't think there's anything that tidies that up - it probably just grows in size.  I didn't find this on the test system because the upgrade didn't bomb.  The other key problem is that the files are owned by root.

 

By root?!?

How did that happen?  As far as I know Learn runs as the user bbuser so how did this file get written by root?  It wasn't me as I shift the file into Learn using a web endpoint and curl.  It does imply that there's a bit of the Learn infrastructure running with superuser privileges and although that's not always a bad thing it's usually something that's best avoided.

 

My test system does not show the same problem.   All the files are indeed owned by bbuser and didn't have the problem with the upgrade, so what caused the production system to have the files with the wrong owner?

 

Installer Crash

So the installer bombs and dumps many dozens of lines of output.  The key one is this:

Caused by: java.io.IOException: Bad return value from chown: 1 (chown: changing ownership of `/usr/local/blackboard/content/vi/BBLEARN/plugins/bbgs-GradeJourney/config/archive/a.
col.1550239053052.processed': Operation not permitted

Which raises two points:

  1. Why was the file not owned by bbuser
  2. Why is the installer (which runs as bbuser and can't change the owner) trying to change the owner

 

Resolution

So what I needed to do was the following:

chown -R bbuser:bbuser /usr/local/blackboard/logs/update-tools

chown -R bbuser:bbuser /usr/local/blackboard/content/vi/BBLEARN/plugins/bbgs-GradeJourney/config/archive

and then restart the installer.

 

I've no idea how the grades journey files got owned by root.  My test system did not have this behaviour and had no problem with the CU10 upgrade so it was a bit of a surprise when the production system bombed.

 

Instead of being down for the advertised 40 mins it turned into 90 mins.

Outcomes