Canadian Blood Services – September 2019

A life saving data center migration

02

Inside Canadian Blood Services’ risk management driven data center migration WRITTEN BY

HARRY MENEAR PRODUCED BY

JAMES BERRY

03

w w w.b lood . ca

CANADIAN BLOOD SERVICES

David Grant,Associate Director, Enterprise Services at Canadian Blood Services, discusses his role in the organization’s recently completed data center migration and ongoing digital transformation

T

he ability to gather, interpret and protect its data is increasingly becoming the metric by which a company survives or

perishes. Data has become, in short, the lifeblood of the modern organization. However, in the past 04

five years, the staggering speed at which IT advancements have swept across every industry has placed pressure on enterprises looking to house IT systems and data in onsite legacy infrastructure. “The sheer volume of IT services and the explosion of data means that it all has to be stored somewhere, and if it’s not in your own data center, it needs to be in somebody else’s. It’s led to enormous growth in capacity across the commercial space, and those new data centers have been built with the latest technologies which can quite often put your own in-house data center to shame,” explains David Grant, Associate Director, Enterprise Services at Canadian Blood Services. “When you look at the needs of the modern digital workplace, then attempt to retrofit your own data center real estate to give it the same capabilities as some

05

w w w.b lood . ca

CANADIAN BLOOD SERVICES

of those commercial ones, it becomes

“ Canadian Blood Services is the first place where I’ve had the opportunity to make a difference to somebody’s personal life, as opposed to just improving the bottom line, or increasing shareholder value.

It’s a nice feeling” 06

— David Grant, Associate Director, Enterprise Services, Canadian Blood Services

enormously expensive and, in some cases, just isn’t feasible.”

With a career spanning more than 20 years in the data center space, Grant came to Canadian Blood Services in 2016. We sat down with him to discuss his role in the organization’s recently completed migration from two legacy data centers in Ottawa to commercially operated colocation facilities, a move calculated to ensure the safety and resilience of Canadian Blood Services’ vital IT operations.

Founded in 1998, Canadian Blood Services is the country’s lifeline, providing a link between the generosity of over 410,000 annual blood, plasma and organ donors from Vancouver to Newfoundland, and the patients that need it. “We drive the donor experience so that it’s as pleasant as possible in order to encourage people to continue in their generosity. At the same time, we continue to innovate and improve the products we offer, so that we can improve patient outcomes and ultimately save lives. It’s quite a mission to be involved in,” enthuses Grant.

“I’ve worked in IT for a long time, but Canadian Blood Services is the first

CLICK TO WATCH : ‘THERE ARE LOTS OF REASONS TO JOIN #CANADASLIFELINE’ 07 place where I’ve had the opportunity

transformation at the front end, we

to make a difference to somebody’s

have improved that donor experience.”

personal life, as opposed to just

However, as in most aspects of

improving the bottom line or increasing

industry, Grant admits that the

shareholder value. It’s a nice feeling.”

attention paid to upgrading systems at

Canadian Blood Services has

the front end had outstripped that paid

already taken steps over the last three

to the internal workings of the organiza-

years to improve its front end customer

tion. “That front end user experience

experience, digitalising booking

stuff tends to move pretty fast, but

processes and streamlining workflow.

not so much the back end plumbing.

“Donors generously give not only their

But, without that plumbing, the front

blood, but also their time,” notes Grant.

end isn’t really much use,” he explains.

“The last thing that we want to do is

In 2016, Canadian Blood Services

waste that time chasing bits of paper

engaged Gartner to review its data

around our clinics. By doing this digital

center strategy and the results w w w.b lood . ca

The Trusted Healthcare

Advisor

Contact us today to learn more about our IT services.

Learn More

An always-on datacenter is fundamental to the operation and urgency of blood delivery and organ donation.

We enable Canadian Blood Services to consistently remain responsive, relevant and patient-focused. www.teramach.com | 877-226-6549

“ By doing digital transformation at the front end, we’ve improved that donor experience” — David Grant, Associate Director, Enterprise Services, Canadian Blood Services

one in our HQ and one in a building that houses most of the IT team. The distance between the two buildings was less than four miles,” Grant says. “In terms of disaster recovery (DR), that’s basically next door to one another.”

In September 2018, while Canadian Blood Services were midway through their migration, an unprecedented total of six tornadoes swept across Ottawa and Quebec, causing damage in excess of $300mn and leaving

revealed a particularly worrying

hundreds of thousands of residents

vulnerability. “We realized that our data

without power. “It was a timely reminder

centers were both located in Ottawa –

that the ongoing change in weather

E XE CU T I VE PRO FI LE

David Grant David Grant is an enthusiastic IT professional with over 30 years’ experience in the provision and support of IT services in both private sector and government organisations. Focusing on Data Centers and IT Infrastructure he has led several successful initiatives around data centre migrations and consolidations, and IT infrastructure transformation. David has a successful track record in developing IT strategies to take advantage of new and emerging technologies and delivering success through strong leadership and effective team building.

w w w.b lood . ca

09

CANADIAN BLOOD SERVICES

patterns we’ve seen over the past five to 10 years is going to continue, and having two data centers that were literally next door to one another was probably not a good idea,” says Grant.

The review of Canadian Blood Services’ systems not only exposed areas in the organization’s data centers that had single points of failure – areas which would have been both costly and disruptive to upgrade to Uptime Institute Tier Three certification – but also the threat posed by the state of its DR plan. “Our contingency at the time 10

was a cold site about four hours to the south in an IBM data center. In the event of the loss of data centers in Ottawa, the DR strategy was to drive down there, collect the necessary hardware and then restore our systems from backup tapes. The estimated recovery time for an event like that was between one and six days,” Grant explains.

“When you’re a digital business, being down or out for 24 hours is a major issue. Being out for five or six days – well, you’re lucky if you survive.”

The need for revision of Canadian Blood Services’ data center strategy was, to Grant, clear. Change, however, needed to be effected as safely as

possible. “We are a risk-averse organization, as suits people who are in the business of saving lives,” says Grant. “So, we adopted the colocation approach and went looking for a partner who could provide facilities in locations that were geographically diverse.”

Canadian Blood Services eventually settled on Rogers Communications, which now provides data center services in Calgary and near to Toronto (both locations are near one of the organization’s major manufacturing sites).

A veteran of more than a few migration strategies, Grant is well aware that each has its own unique challenges. “We’re regulated by Health Canada, so there are a lot of protocols around testing and validation, which all had to be embedded into our strategy,” he explains. “We broke it down into a series of overlapping waves based around business capabilities: our blood management system, our donor management system, etc. Then, we further subdivided those into our non-production and production-based systems.” The process saw a huge number of careful, methodical tests, warranty periods and planned outages, that allowed the migration to be carried w w w.b lood . ca

11

CANADIAN BLOOD SERVICES

“ When you’re a digital business, being down or out for 24 hours is a major issue. Being out for five or six days - well, you’re lucky if you survive” — David Grant, Associate Director, Enterprise Services, Canadian Blood Services

OnX Canada is proud to support Canadian Blood Services OnX leverages technology to inspire innovation, promote growth, drive efficiency, and accelerate our clients’ success. Our focus is on understanding our clients’ unique business challenges and desired outcomes.

At OnX, we think about the entire lifecycle of the technologies employed by our clients and work with them using a holistic approach to help them achieve their goals.

We do this by assessing, designing, building, and managing technology environments with specific expertise in cloud and managed services, digital solutions, and information technology solutions.

For more information, visit onx.ca

Consult

Build

Transform

Support

13 out effectively and with minimal risk.

“We did stretch our team quite a bit just to stay on top of the overlapping waves, but they were real troopers and they came through for us,” Grant recalls.

Another key metric in the success of the migration is the increased resilience of Canadian Blood Services’ DR plan. “It was calculated that the business could only tolerate four hours of downtime,” says Grant. To shrink the process of getting the organization’s IT infrastructure back on line from six days to four hours, Canadian Blood Services bought enough capacity in w w w.b lood . ca

CANADIAN BLOOD SERVICES

its new colocation sites that the entire business can be run from either one.

“We replicate our VMware workloads from one data center to the other continuously. Should we lose a data center site, we use Zerto to reactivate those VMs in the other data center.”

Zerto is a Tel-Aviv-based virtual replication software company that specializes in providing operational continuity in the direst of circumstances.

The company’s duplication software was also, according to Grant, a key tool in the migration itself. “We used 14

Zerto replication to move the data from our Ottawa data centers to our new Rogers partners data center site,” he explains.

10 years ago, if a company embarked on a digital project, or bought a new piece of software, the expectation was that there was a finite price tag and completion date. In 2019, if executed correctly, a digital transformation project will cost an unlimited amount of money and take an unlimited amount of time. With the completion of the migration, Grant at the team are preparing to immediately begin reevaluating Canadian Blood Services’

“ We did stretch our team quite a bit just to stay on top of the overlapping waves, but they were real troopers and they came through for us” — David Grant, Associate Director, Enterprise Services, Canadian Blood Services

data center and DR strategies. “When

years, we’ve seen a lot of development

we were first doing this, a lot of the

in that space. I think our next data center

options and ideas getting thrown

strategy will look at what we’re still

around were things like, ‘Why don’t we

running in-house and in our colocation

move DR to the cloud? Why don’t we

facility, and determine what’s the next

move this to the cloud? Why don’t we

step for these systems.”

move that to the cloud?’ Even three years ago, a lot of those services were in their infancy and weren’t really to be trusted, and certainly not by blood operator - somebody who is committed to patients and donors and saving lives wouldn’t want to risk that,” Grant explains. “In the intervening three w w w.b lood . ca

15

Canadian Blood Services 1800 Alta Vista Drive, Ottawa Ontario K1G 4J5, CA T 1-877-709-7773 www.blood.ca