Page 1

Learning  Management  Systems   Five-­‐Year  Evaluation   Academic  Technology  Council   May  2,  2010       Background     In  1999,  Choate  Rosemary  Hall  implemented  Blackboard,  its  first  learning   management  system  (LMS).  It  was  a  stand-­‐alone  platform  to  deliver  both  course   content  and  information  over  the  web.  In  2005,  Choate  completed  its  initial   evaluation  of  Blackboard  with  an  ad  hoc  task  force,  and  concluded  that  the  school   would  be  best  served  by  continuing  to  use  Blackboard.  In  addition,  the  school   decided  to  expand  its  use  of  Blackboard  to  include  the  Community  and  Content   systems,  creating  the  school's  internal  portal  called  Chip.  Five  years  later,  the   Academic  Technology  Council  (ATC)  embarked  on  the  second  evaluation  of   Blackboard.  There  were  several  considerations  that  made  the  timing  of  this  study   particularly  appropriate.     1. Blackboard  had  recently  released  their  next  generation  learning   management  platform  (Version  9).  The  new  interface  for  teachers  would   necessitate  retraining.  If  retraining  were  required,  then  the  switch  to  a  new   platform  would  be  no  more  complex  than  learning  the  new  Blackboard   interface.     2. The  Blackboard  system  configuration  is  complex,  making  the  installation  of   upgrades  and  new  releases  time  consuming.  It  has  become  another   enterprise  application,  and  the  ATC  wondered  if  there  were  alternative   solutions  that  were  easier  to  maintain.     3. The  Communications  and  ITS  Departments  were  contemplating  a  new   internal  portal  that  would  replace  the  Blackboard  Community  and  Content   systems.     Recommendation      After  evaluating  ten  potential  alternatives,  the  ATC  recommends  that  the  school  use   Blackboard  Learn  as  our  learning  management  platform  with  the  understanding   that  it  will  be  integrated  with  any  internal  portal  ultimately  developed  by  the  school.   The  primary  considerations  supporting  this  recommendation  were  as  follows:     1. Blackboard  controls  more  than  80%  of  the  learning  management  system   market  since  its  acquisition  of  WebCT.  The  combined  resources  of  the  higher   education  and  K-­‐12  markets  make  the  future  of  Blackboard  bright  and  stable.    

2. Many of  the  other  mature  learning  management  solutions  were  similar  in   structure  and  functionality  to  Blackboard,  making  a  shift  in  vendors   questionable.     3. The  newer,  more  innovative  systems  have  few  customers  and  are  not  as  well   capitalized,  making  a  long-­‐term  partnership  with  Choate  risky.     4. Blackboard  continues  to  add  business  partners  that  provide  value-­‐added   modules  to  enhance  the  capabilities  of  the  basic  Blackboard  system.  In   addition,  Blackboard  continues  to  add  relationships  with  book  publishers,   creating  course  cartridges,  including  test  materials,  which  may  be  loaded  into   a  Blackboard  course.     5. Blackboard  has  partnered  with  companies  that  will  deliver  all  course   materials  to  mobile  devices  such  as  iPhone  and  Blackberry.  The  ATC  believes   that  these  mobile  devices  will  become  ubiquitous  in  our  student  body  over   the  next  several  years,  and  thus,  Bb  will  permit  us  to  adapt  to  the   proliferation  of  these  devices  should  we  choose  to  do  so.  

Analysis     The  following  alternative  learning  management  solutions  were  examined,  and  a   brief  evaluation  of  each  is  provided.     • Moodle  –  Arguably  the  most  widely  used  learning  management  system  in   education  (statistics  are  difficult  to  obtain  because  Moodle  is  Open  Source   software,  and  thus  not  purchased  from  a  vendor).  Schools  and  colleges  that   are  on  very  limited  budgets  often  install  Moodle,  and  then  purchase  a   support  contract  from  a  third  party  company.  Moodle  is  a  "conversation-­‐ oriented"  platform  with  communication  between  students  the  central   feature,  driving  the  importance  of  course  content.  The  ATC  felt  Moodle's   design  was  not  consistent  with  our  approach  to  teaching  and  learning.  There   is  a  conversion  utility  to  move  course  materials  from  Blackboard  to  Moodle.     • Desire2Learn  –  One  of  the  few  commercial  direct  competitors  to   Blackboard.  Functionality  and  user  interface  are  similar  enough  that  a   change  of  systems  would  not  be  justified.  Desire2Learn  does  not  have  a   Community  Portal.     • Pearson  Learning  Studio  (formerly  eCollege)  –  Another  commercial   product  that  Choate  looked  at  ten  years  ago.  This  platform  is  a  hosted   solution  only  (cannot  be  installed  locally),  and  is  changing,  with  additional   working  capital  provided  by  Pearson  Publishing.  There  is  a  very  tight   interface  between  the  Learning  Studio  and  Pearson  textbooks,  but   Blackboard  provides  course  cartridges  from  a  wider  range  of  publishers.  


  Sakai  –  Perhaps  the  solution  with  the  most  promise,  Sakai  is  an  open  source   joint  development  effort  between  MIT,  Stanford,  and  University  of  Michigan.   Other  universities  have  adopted  the  product  because  it  provides  many  hooks   to  third  party  applications,  and  interfaces  to  major  college  administrative   systems.  There  are  several  third  party  companies  that  support  the  product,   but  much  of  the  innovation  in  the  product  is  related  to  ePortfolios,  online   student  assessment,  and  higher  education  course  registration,  functionality   that  would  not  be  as  valuable  to  secondary  schools.  Again,  the  course   management  interface  is  very  similar  to  Blackboard.     Hot  Chalk  –  Another  hosted  solution  tailored  to  public  schools  with  a   community  of  teachers  providing  lesson  plans  that  meet  national  curriculum   standards,  a  rudimentary  course  management  site,  and  a  grades  module   designed  to  provide  parents  with  access  to  student  grades  and  the   appropriate  teacher.  While  the  concept  of  the  system  is  not  a  good  fit  for   Choate,  some  of  the  curricular  lessons  might  be  appropriate.  Blackboard  is   now  providing  a  building  block  that  allows  integration  of  HotChalk  lessons   into  Blackboard  Learn.  When  our  system  is  upgraded  to  Learn,  we  will  be   able  to  utilize  this  capability.     Wikispaces  or  Google  Sites  –  The  underlying  assumption  in  using  a  teacher   web  site  was  to  save  money  and  dispense  with  the  large  number  of   Blackboard  features  that  are  not  widely  used  by  our  faculty.  These  types  of   sites  are  very  easy  to  populate  with  course  content,  but  lack  the  additional   tools  for  assessment,  grade  books,  time  sensitive  content,  etc.  that  are   integrated  with  Blackboard.  Consequently,  additional  tools  would  have  to  be   incorporated  to  result  in  an  equivalent  Blackboard  experience.  Furthermore,   the  ATC  tested  the  conversion  of  content  from  Blackboard  to  Wikispaces   with  unsatisfactory  results.  There  was  an  extensive  amount  of  cleanup   required.     Epsilen  –  An  early,  but  good  example  of  the  future  of  personal  learning   environments  (PLE  –  defined  as  the  tools,  communities,  and  services  that   constitute  the  individual  educational  platforms  learners  use  to  direct  their   own  learning).  While  there  were  some  wonderful  tools,  the  system  was  not   as  visually  attractive  as  many  others  (designed  for  adults).  Because  there   were  so  many  features  available,  the  ATC  felt  it  was  both  overkill  with   respect  to  our  needs  and  unusually  complex  to  navigate.  There  was  no  ability   to  convert  Blackboard  materials  to  Epsilen.     Haiku  –  A  new  and  fresh  approach  to  delivering  course  content,  Haiku  takes   advantage  of  the  latest  advances  in  web  technologies.  Teachers  are  provided   with  multiple  design  options  (unlike  Blackboard),  and  design  templates  that   bring  materials  to  life.  The  interface  is  much  easier  and  intuitive  to  navigate   3  

than Blackboard  (without  any  training),  and  adding  content  is  a  "drag  and   drop"  process.  Unfortunately,  Haiku  is  a  relatively  new,  hosted  product   delivered  by  a  small  Canadian-­‐based  company.  They  have  a  limited  customer   base  and  are  funded  by  private  capital.  There  was  a  good  deal  of  concern   about  the  sustainability  of  the  company,  particularly  as  other  developers  of   learning  management  solutions  take  advantage  of  new  web  technologies.  The   ATC  agreed  that  we  should  keep  our  eye  on  this  solution,  but  it  was   premature  to  risk  a  conversion  at  this  time.     Udutu  –  A  sophisticated,  but  easy-­‐to-­‐use  course  content  creation  system  that   is  best  suited  for  more  complex  presentations  such  as  simulations  and   interactive  materials.  Udutu  is  designed  for  those  who  lack  the  technical   expertise  to  program  in  Flash  (similar  to  Mathematica,  but  more  broadly-­‐ based)  or  Java.  Udutu  is  not  a  comprehensive  learning  management  platform,   so  it  lacks  many  of  the  tools  provided  by  Blackboard  (although  these  tools   could  likely  be  developed  in  Udutu  with  sufficient  expertise).  More   importantly,  Udutu  content  can  be  exported  to  Blackboard  using  the   industry-­‐standard  SCORM  format,  so  those  teachers  who  might  wish  to   develop  more  sophisticated  content  can  utilize  the  free  version  of  Udutu  as  a   supporting  tool  for  Blackboard.     Google  Wave  –  While  it  is  still  a  beta  product,  lacking  a  great  deal  of   documentation,  Wave  appears  to  be  Google's  contribution  to  social   networking,  advanced  threaded  discussion,  and  delivery  of  dynamic  content   as  a  form  of  communication  between  teacher  and  students.  It  has  promise  in   the  sense  that  it's  design  is  consistent  with  the  collaborative  models  that   have  been  proposed  for  the  future  of  teaching  and  learning.  The  conclusion  of   the  ATC  was  that  Choate  was  not  quite  ready  to  enter  that  next  generation   educational  paradigm  yet,  and  that  Wave  was  not  ready  for  prime  time.    

Summary  and  Conclusion     This  report  provides  a  specific  recommendation  that  the  school  continue  to  follow   the  Blackboard  Learn  upgrade  path  for  the  next  three  to  five  years,  while  watching   web-­‐based  developments  that  might  impact  the  ease  with  which  teachers  could   create  online  courses  and  course  materials.  There  are  several  developments  in  the   area  of  interactive  learning  and  simulation  on  the  horizon  that  could  potentially   have  a  profound  positive  impact  on  the  needs  of  our  faculty.  In  the  interim,  the  ATC   feels  it  would  be  most  prudent  to  stay  the  course  with  the  incumbent  Blackboard   system,  upgrading  to  Learn  during  the  2010-­‐2011  academic  year.  There  is  minimal   risk  associated  with  this  strategy  since  Blackboard's  financial  strength  is  formidable,   and  the  next  generation  product  will  meet  the  bulk  of  our  current  and  short-­‐term   future  needs.  


Choate LMS Evaluation - 2010  

Comparison of Learning Management Systems

Choate LMS Evaluation - 2010  

Comparison of Learning Management Systems