Air system failure results in thousands of lost BOL e-mails

If you were screaming at the computer because you couldn’t
log onto Bruin OnLine Web mail two weekends ago, you weren’t
alone.

A Web mail failure recently resulted in the loss of massive
amounts of e-mail. Communication Technology Services was forced to
take immediate preventative measures.

CTS deals with UCLA’s telephone and data components,
including the Web mail service.

In order to repair the problem and make sure it does not happen
again, CTS had to figure out the source of the outage.

“Just prior to the outage, the air handling systems in
that server room failed … and it resulted in a hardware
failure,” Michael Schilling, acting director of CTS,
said.

The air handling systems act as the air conditioner in the
server room, the location of Web mail support.

As a result, BOL Web mail was inaccessible from 9:12 a.m. on
Jan. 31, until Feb. 2 when the hardware upgrade was completed,
according to the BOL Web site.

A similar outage had occurred two weeks earlier, with no loss of
mail, but it was only after the last failure that CTS realized
these problems were due to the air handling systems, said
Schilling.

“I personally took immediate steps, and purchased portable
backup air units,” he said.

The units were installed last Tuesday to prevent another system
failure, especially considering the impact of the second
outage.

“(There were) 3,500 to 4,500 e-mails that were so badly
corrupted that they couldn’t be redirected to the appropriate
users,” Schilling said.

Due to both the loss of e-mail, and the inaccessibility of the
service, the BOL Help Desk received more than 100 phone calls by
that Monday evening, from students who needed support, he said.

The main question on the minds of students who lost mail was
whether they could retrieve it again.

“I would characterize it as very regrettable that
we’ve lost some e-mail, but those messages that were lost are
not recoverable,” Schilling said.

Students who lost e-mail may have trouble sorting things out as
a result of the outage, but many students did not even know about
the failure, and said they would continue to use the e-mail
service.

“The system’s not perfect,” said Nicole
Henderson, a first-year undeclared student. “Mistakes will
happen.”

Henderson said she finds the e-mail service reliable, and will
not stop using it.

Others, who have accounts with free services like Hotmail and
Yahoo!, say that they will not rely as heavily on BOL Web mail, but
the UCLA service remains convenient for them.

“It’s easier to access BOL Web mail because if
you’re already there for your courses, (you think),
“˜let me check my mail’,” said Farah Awan, a
third-year political science student.

The issue remains a concern with CTS, and they will continue to
ensure improved services for users.

“E-mail is an absolutely vital resource and a robust
service should be a given. We recognize that any outage can
significantly and negatively impact the productivity of the UCLA
community,” said Jim Davis, associate vice chancellor for
information technology.

CTS will review the need for upgrading the air handling system
as well as the BOL servers, and installing disaster recovery
capabilities as possible preventative measures, Schilling said.

A notice was sent out after the outage, informing users about
downloading BOL e-mail using a Post Office Protocol e-mail client
to guard against e-mail loss. This allows e-mail to be downloaded
onto a local computer so if the original message is lost, the copy
is saved.

The e-mail urged students to contact the BOL Help Desk if they
need help using the POP client, or have questions regarding the
outage.

The BOL Help Desk can be reached at (310) 825-7452, or by e-mail
at consult@ucla.edu.

Leave a comment

Your email address will not be published. Required fields are marked *