Ie security easylly july august 2016

Contents | Zoom in | Zoom out

For navigation instructions please click here

Search Issue | Next Page

Benefits of Bug Bounties Securing the IoT Preventing Data Leakage ■

■

July/August 2015 Vol. 13, No. 4

Contents | Zoom in | Zoom out

For navigation instructions please click here

Search Issue | Next Page

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

2016

Richard E. Merwin Distinguished Service Award CALL FOR AWARD NOMINATIONS Deadline 15 October 2015 ABOUT THE MERWIN AWARD The highest level volunteer service award of the IEEE Computer Society for outstanding service to the profession at large, including significant service to the IEEE Computer Society or its predecessor organizations.

ABOUT DICK MERWIN Dick Merwin was a pioneer in digital computer engineering who participated in the development of the ENIAC, MANIAC, and STRETCH computers.

Despite a busy and productive technical career, Merwin found time to be active in professional societies, including the IEEE Computer Society, ACM and AFIPS. His generosity of spirit and genuine helpfulness was an important element in the progress of the computer profession.

AWARD A bronze medal and $5,000 honorarium are awarded.

PRESENTATION The Richard E. Merwin Award is presented at the IEEE Computer Society’s Annual Awards Ceremony.

RANGACHAR KASTURI 2015 Richard E. Merwin Distinguished Service Award For exemplifying a true volunteer spirit and a commitment to excellence through significant and continuing contributions supporting the vision and mission of the IEEE and the IEEE Computer Society.

REQUIREMENTS This award requires 3 endorsements.

NOMINATION SUBMISSION Nominations are being accepted electronically www.computer.org/web/awards/merwin ______________________________

AWARDS HOMEPAGE www.computer.org/awards _____________________

CONTACT US awards@computer.org

_________________

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

July/August 2015 Vol. 13, No. 4

Cover art by Peter Bollinger, www.shannonassociates.com

Contents

30 A Socio-technical Framework for Threat Modeling a Software Supply Chain Bilal Al Sabbagh and Stewart Kowalski

Also in This Issue 40 Toward Anti-fragility: A Malware-Halting Technique Kjell Jørgen Hole

47 Distance-Bounding Protocols: Are You Close Enough?

Christos Dimitrakakis and Aikaterini Mitrokotsa

52 Can Content-Based Data Loss Prevention Solutions Prevent Data Leakage in Web Traffic?

David Gugelmann, Pascal Studerus, Vincent Lenders, and Bernhard Ager

60 Packet Inspection for Unauthorized OS Detection in Enterprises Rohit Tyagi, Tuhin Paul, B.S. Manoj, and B. Thanudas

66 Bypassing EMET 4.1 Jared DeMott

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

76

Columns 3

M q M q

M q

MqM q

84

Departments

From the Editors

A Strategic Plan for Cybersecurity Research and Development Terry Benzel

88 Last Word The Right to Be Unobserved Daniel E. Geer Jr.

7

Interview

Silver Bullet Talks with Katie Moussouris Gary McGraw

80 Building Security In Garage Door Openers: An Internet of Things Case Study Jonathan Margulies

73 It All Depends The Importance of Security Cases: Proof Is Good, But Not Enough John Knight

84 In Our Orbit Mass-Marketing Fraud: A Growing Concern Monica T. Whitty

76 Security & Privacy Economics Ideas Ahead of Their Time: Digital Time Stamping Michael Lesk

Also in This Issue 89 | IEEE Computer Society Information 89 | IEEE Reliability Society Information 91 | Advertiser Index

On the Web 2012 Annual Index www.computer.org/ security/12index

___

Postmaster: Send undelivered copies and address changes to IEEE Security & Privacy, Membership Processing Dept., IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854-4141. Periodicals postage rate paid at New York, NY, and at additional mailing offices. Canadian GST #125634188. Canada Post Publications Mail Agreement Number 40013885. Return undeliverable Canadian addresses to PO Box 122, Niagara Falls, ON L2E 6S8. Printed in the USA. Circulation: IEEE Security & Privacy (ISSN 1540-7993) is published bimonthly by the IEEE Computer Society. IEEE Headquarters, Three Park Ave., 17th Floor, New York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720, phone +1 714 821 8380; IEEE Computer Society Headquarters, 2001 L St., Ste. 700, Washington, D.C. 20036. Subscribe to IEEE Security & Privacy by visiting www.computer.org/security. IEEE Security & Privacy is copublished by the IEEE Computer and Reliability Societies. For more information on computing topics, visit the IEEE Computer Society Digital Library at www.computer.org/csdl.

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

FROM THE EDITORS

A Strategic Plan for Cybersecurity Research and Development

W

e are under a constant onslaught of cyberattacks. It seems that every day we learn of yet another large-scale data breach. Increasingly, these attacks are being attributed to well-organized, well-trained, and determined adversaries, and many have targeted government information systems across the world.

Part of the cyberexploits’ success arises from the fact that technology regulations, policies, and profit structures aren’t well aligned with coordinated cybersecurity R&D. Neither the commercial marketplace nor government programs have been successful in providing ways to protect computer systems.

Transforming the cyber landscape calls for a coordinated R&D strategy that provides a basis for steering security R&D toward global strategic goals.

There’s now an effort to do so. The US government and its international partners are creating the Cybersecurity Research and Development Strategic Plan, which will be used to guide government funding for programs, policies, investments, and collaborations for the next five years.

The US National Science Foundation, the National Coordinating Office for the Networking and Information Technology Research and Development (NITRD) Program, and the US Department of Homeland Security (DHS) have been engaged in community meetings and discussions to shape this new strategy.

In this article, I provide a short report on these events as well as information on how you can join the conversation and contribute to these efforts. Although much of the work I discuss centers on US government– led efforts, it’s important to note that many of these organizations have strong international counterparts and collaborators.

1540-7993/15/$31.00 © 2015 IEEE

SECURITY& PRIVACY

IEEE

A New Strategy In 2011, the NITRD published “Trustworthy Cyberspace: Strategic Plan for the Federal Cybersecurity Research and Development Program,” a coordinated effort involving leading researchers, industry, and government.1 The Cybersecurity Enhancement Act of 2014 spurred changes to this strategy, examining the 2011 federal plan and identifying areas in which its priorities should change—for example, removing some topics, continuing areas previously identified, and adding new areas and challenges.2 Although government organizations are leading the coordination effort, the new strategy recognizes the important role of industry and small business. A strategic road map for global cybersecurity R&D can help prioritize commercial investments and balance the tradeoffs. Furthermore, a key aspect of any strategy is an emphasis on technology transition, or “transition to practice.” In this way, early fundamental investments in research will have a well-planned path toward productization and application in both the private and government sectors.

In May 2015, I had the opportunity to participate in several forums focusing on the new strategy, including the 2015 IEEE Symposium on Security and Privacy’s NITRD panel and Birds of a Feather session as well as a DHS national conversation.

Terry Benzel Associate Editor in Chief

NITRD Panel During the IEEE Symposium on Security and Privacy, the NITRD Council hosted a panel session to discuss this effort. The panel described the process of creating the 2011 strategic plan, the coordination across different agencies, and a high-level view of the 2011 strategic plan’s funding (more than US$700,000 across multiple agencies). The panel reached

Copublished by the IEEE Computer and Reliability Societies

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

3 M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

FROM THE EDITORS

an important segment of the community and gave rise to questions of interest to the researchers.

Not surprisingly, one area of discussion was research in cryptography and issues with export control regulations. Interestingly, another area of discussion focused on capacity building and education for a broader community than just academic researchers. This also included a conversation on how students can develop careers in government and agency recruiting.

Birds of a Feather Session

■ What objectives that weren’t included in the 2011 strategic plan should be prioritized for federally funded cybersecurity R&D? ■ What innovative technologies could enhance digital infrastructure’s security, reliability, and resiliency?

These questions were discussed in two sessions over the course of several hours. Each working group of 10 to 12 people included a wide cross-section of industry, academia, and critical infrastructure providers. A report on this event and the working groups’ recommendations are available on the DHS Ideascale portal.3

■ ■ ■ ■

mobile security, privacy, usability, cybersecurity education training, and ■ economics and policy.

and

The second working group focused on the 2011 strategic plan’s topic areas of ■ ■ ■ ■

designed-in security, tailored trustworthy spaces, moving-target defense, and economic incentives for cybersecurity.

Later that day, Doug Maughan of This group recommended that DHS hosted an informal Birds of a designed-in security continue to be Feather session on the 2011 stratean area of emphasis in gic plan. Approximately the 2015 plan and high30 people from academia, lighted the importance industry, and government What innovative technologies could of accessible, affordable, participated in a wideenhance digital infrastructure’s and understandable ranging discussion on approaches to designing the security and privacy security, reliability, and resiliency? in security—rather than of critical infrastructure. adding it after the fact.

Some of the more extenThis led to the issues of sive discussions focused on the science of security versus One working group focused secure architectures and systemtransition to practice, the need for on the 2011 strategic plan’s theme of-systems security. Interestingly, real systems and testbeds to support of developing scientific founda- this conversation naturally segued research, and the Internet of Things tions of cybersecurity. This group into discussions on both tailored as critical infrastructure. A complete recommended that the new plan trustworthy spaces as an approach report on this session is available on continue this theme and extend to composition of subsystems with the DHS Ideascale portal.3 the definition of science to include varying degrees of security, and the human behavior and social, psy- need for context-aware and specialA DHS National chological, and economic models. ized security requirements.

Related to this discussion is Conversation The group discussed several analOn 4 June 2015, DHS held one of ogies with other scientific disci- the need for economic incentives a series of national conversations plines, including the importance of for cybersecurity. A prerequion a “trusted cyber future.” The experimental science as a compo- site to economic incentives is the objectives of this meeting were to nent of any science, and extended development of metrics or meagain the cybersecurity communi- discussions to include the impor- surements that can be used for ty’s input and responses to ques- tance of privacy and addressing evidence-based information on return on investment. Such metrics tions that would help shape DHS’s operational needs.

Science and Technology Division This working group recom- are needed to make business deciand federal cybersecurity R&D mended the following extensions sions on designing in security and for the next five years. Participants and additions for the updated stra- developing techniques for contextspecific cybersecurity. were asked to consider the follow- tegic plan: The working group also noted ing questions: ■ cybersecurity research infra- that moving-target defense seemed to describe a set of mechanisms structure, ■ What areas of research of the rather than a rich R&D area. If this 2011 strategic plan don’t need to ■ network and cloud security, topic is to remain a focus area in the ■ cyber-physical systems, be prioritized anymore?

4

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

2015 plan, then the research challenge is to determine the context in which it’s most applicable, along with an understanding of the characteristics of systems in which it can be deployed, and which properties it can augment. Furthermore, some cybersecurity community members questioned whether moving targets might actually introduce unexpected vulnerabilities, undermining the goal of increasing effort for attackers.

Recommendations for additional areas to include in the new plan are ■ holistic approaches to securing systems, ■ system-of-systems security requirements, ■ resiliency as a requirement, ■ software assurance through languages and standard practices, ■ virtual organizations as security containers, ■ privacy-preserving data sharing, ■ consensus algorithms as an approach to data, ■ human behavior modeling, and ■ experimental science as a research topic.

The results from the previous events and ongoing DHS national conversations will provide input to the coordinated new cybersecurity strategy.

International Strategy Development The US isn’t alone in prioritizing strategy development and focus ing on cybersecurity as part of federal funding and program development. In December 2014, the UK unveiled a strategy that’s notable for its breadth. This strategy is intended to create business drivers for the UK, with the goal of “making the UK one of the most secure places in the world to do business.”5 This strategy and similar efforts in Europe, the EU Joint

Research Council,6 and Singapore7 stress the importance of the commercial marketplace.

Cybersecurity is a global problem requiring global solutions. We can’t afford to have a serious discussion of cybersecurity that’s influenced by partisanship or nationalism.

W

e’re at a critical crossroad.

The level of awareness around the importance of transformative cybersecurity R&D is at an all-time high. There are legislative, government, and industry actions for funding R&D. Now, for the second time in five years, we have the opportunity to help shape a coordinated federal strategy. The Cybersecurity Enhancement Act of 2014 calls for ■ a review of cybersecurity threats to our critical infrastructure; ■ development of R&D programs to create new cyberdefenses and stronger critical infrastructure; ■ coordination, collaboration, and education for enhanced technology transfer from R&D to practical cyberdefenses; and ■ education and workforce development to build the next generation of cyberdefenders who will use this new technology.

There is much work to be done.

Developing a strategy is only one aspect of the effort required to transform cyberspace. Nonetheless, without input from the community, we will miss the opportunity to develop a strategy to guide many future efforts.

Let your voice be heard:

■ review the reference documents and participate in discussions on the DHS Ideascale portal,3 ■ work with your colleagues and organizations to help build the community addressing these pressing problems, or

■ provide feedback on the new strategic plan.4 References 1. “Trustworthy Cyberspace: Strategic Plan for the Federal Cybersecurity Research and Development Program,” US Executive Office of the President, Dec. 2011; www.white house.gov/sites/default/files/ microsites/ostp/fed_cybersecurity ____________________ _rd_strategic_plan_2011.pdf. _________________ 2. S.1353—Cybersecurity Enhancement Act of 2014, 113th Congress, congress.gov, 2014; www.congress .gov/bill/113th-congress/senate -bill/1353. ______ 3. “A Trusted Cyber Future: Protecting Privacy, Commerce, and Community,” Ideascale, Dept. Homeland Security, 2015; http://scitech.idea scale.com/a/pages/cyberreference.

4. “Request for Information (RFI)— Federal Cybersecurity R&D Strategic Plan,” Federal Register, vol. 80, no.

80, 2015, p. 22305; http://federal register.gov/a/2015-09697.

5. “UK Cyber Security Strategy: Statement on Progress 3 Years On,” gov. uk, 11 Dec. 2014; www.gov.uk /government/speeches/uk-cyber ____________________ -security-strategy-statement-on ____________________ -progress-3-years-on. ___________ 6. “Table on the Implementation of the ‘Cybersecurity Strategy of the European Union: An Open, Safe and Secure Cyberspace’ ( JOIN(2013)1),” working document, European Commission, 28 Feb. 2014; http://ec.europa.eu /information_society/newsroom ____________________ /cf/dae/document.cfm?doc_id ____________________ =4623. ____ 7. “Annex A: Factsheet on National Cyber Security Masterplan 2018,” iDA Singapore, July 2013; www ___ .ida.gov.sg/~/media/Files/About %20Us/Newsroom/Media%20 ____________________ R eleases/2013/0724_ncsm ____________________ /AnnexA.pdf. ________ Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

5

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

www.computer.org/security

PUBLISHING COSPONSORS

EDITOR IN CHIEF Shari Lawrence Pfleeger | Dartmouth College shari.l.pfl eeger@dartmouth.edu ________________ ASSOCIATE EDITORS IN CHIEF Terry Benzel | USC Information Sciences Institute Robin Bloomfield | City University London Jeremy Epstein | SRI International Bill Horne | HP Labs Susan Landau | Worcester Polytechnic Institute Ahmad-Reza Sadeghi | TU Darmstadt EDITORIAL BOARD Iván Arce | Fundación Dr. Manuel Sadosky George Cybenko* | Dartmouth College Dorothy E. Denning | Naval Postgraduate School Marc Donner | MSCI Dieter Gollmann | Technical University Hamburg-Harburg Feng Hao | Newcastle University Carl E. Landwehr* | George Washington University and LeMoyne College Roy Maxion | Carnegie Mellon University Nasir Memon | Polytechnic University Rolf Oppliger | eSECURITY Technologies Sean Peisert | Lawrence Berkeley National Laboratory and University of California, Davis Yan Sun | University of Rhode Island Neeraj Suri | Darmstadt Technical University John Viega | SilverSky Taieb Znati | University of Pittsburgh *EIC Emeritus DEPARTMENT EDITORS Attack Trends | Thorsten Holz, Ruhr-University Bochum Basic Training | Richard Ford, Florida Institute of Technology; and Deborah A. Frincke, National Security Agency Building Security In | Jonathan Margulies, Qmulos Crypto Corner | Peter Gutmann, University of Auckland; David Naccache, École normale supérieure; and Charles C. Palmer, IBM Education | Melissa Dark, Purdue University Interview/Silver Bullet | Gary McGraw, Cigital In Our Orbit | Alessandro Acquisti, Carnegie Mellon University; and Angela Sasse, University College London It All Depends | Mohamed Kaâniche, French National Center for Scientific Research; and Aad van Moorsel, Newcastle University

SECURITY& PRIVACY

IEEE

Privacy Interests | Katrine Evans, Hayman Lawyers; and Deven McGraw, Manatt, Phelps & Phillips Security & Privacy Economics | Michael Lesk, Rutgers University; and Jeff rey MacKieMason, University of Michigan Systems Security | Patrick McDaniel, Pennsylvania State University; and Sean W. Smith, Dartmouth College

TECHNICAL COSPONSORS

COLUMNISTS Last Word | Bruce Schneier, Resilient Systems; Steven M. Bellovin, Columbia University; and Daniel E. Geer Jr., In-Q-Tel STAFF Content Editor | Christine Anthony Lead Editor | Brian Kirk Staff Editors | Lee Garber, Meghan O’Dell, and Rebecca Torres Publications Coordinator | security@computer.org Production Editor/Webmaster | Monette Velasco Production Staff | Mark Bartosik, Erica Hardison, and Jennie Zhu-Mai Graphic Design | Alex Torres Original Illustrations | Robert Stack Director, Products & Services | Evan Butterfield Sr. Manager, Editorial Services | Robin Baldwin Content Development Manager | Richard Park Sr. Business Development Manager | Sandy Brown Membership Development Manager | Erik Berkowitz Sr. Advertising Coordinator | Marian Anderson, manderson@computer.org ______________ CS MAGAZINE OPERATIONS COMMITTEE Forrest Shull (chair), Nathan Ensmenger, Mazin Yousif, Miguel Encarnacao, George K. Thiruvathukal, Sumi Helal, Brian Blake, Daniel Zeng, San Murugesan, Lieven Eeckhout, Yong Rui , Maria Ebling, Shari Lawrence Pfleeger, Diomidis Spinellis CS PUBLICATIONS BOARD Jean-Luc Gaudiot (VP for Publications), Forrest Shull, Ming C. Lin, Alfredo Benso, David S. Ebert, Alain April, Laxmi Bhuyan, Greg Byrd, Robert Dupuis, Linda I. Shafer, H.J. Siegel EDITORIAL OFFICE IEEE Security & Privacy c/o IEEE Computer Society Publications Office 10662 Los Vaqueros Circle, Los Alamitos, CA 90720 USA Phone | +1 714 821-8380; Fax | +1 714 821-4010

IEEE Engineering in Medicine & Biology Society

Editorial | Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or fi rm’s opinion. Inclusion in IEEE Security & Privacy does not necessarily constitute endorsement by the IEEE or the IEEE Computer Society. All submissions are subject to editing for style, clarity, and length.

Submissions | We welcome submissions about security and privacy topics. For detailed instructions, see the author guidelines (www.computer.org/ security/author.htm) or log onto IEEE Security ________ & Privacy’s author center at ScholarOne (htt ps:// ___ mc.manuscriptcentral.com/cs-ieee).

Reuse Rights and Reprint Permissions | Educational or personal use of this material is permitt ed without fee, provided such use: 1) is not made for profit; 2) includes this notice and a full citation to the original work on the fi rst page of the copy; and 3) does not imply IEEE endorsement of any third-party products or services.

Authors and their companies are permitt ed to post the accepted version of IEEE-copyrighted material on their own Web servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the fi rst screen of the posted copy. An accepted manuscript is a version which has been revised by the author to incorporate review suggestions, but not the published version with copyediting, proofreading, and formatt ing added by IEEE. For more information, please go to www.ieee. org/publications_standards/publications/rights/ paperversionpolicy.html. Permission to reprint/ __________ republish this material for commercial, advertising, or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ, USA 08854-4141 or pubs-permissions@ieee. _________ org. Copyright © 2015 IEEE. All rights reserved. _ Abstracting and Library Use | Abstracting is permitt ed with credit to the source. Libraries are permitt ed to photocopy for private use of patrons, provided the per-copy fee indicated in the code at the bott om of the fi rst page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.

IEEE prohibits discrimination, harassment, and bullying: For more information, visit www.ieee. org/web/aboutus/whatis/policies/p9-26.html.

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

INTERVIEW Editor: Gary McGraw, gem@cigital.com _________

Silver Bullet Talks with Katie Moussouris Gary McGraw | Cigital

Hear the full podcast at www.computer.org/silverbullet. Show links, notes, and an online discussion can be found at www.cigital.com/silverbullet.

penetration-testing skills doing that when I worked at MIT. I became a Linux developer when I first moved to California. I was mostly working with the different systems administration tools, fixing a lot of bugs. I realized we didn’t have a formal computer security response program there, so I started one. That’s where I began my career in security response and vulnerability handling and coordination.

Tell us in very basic terms how a bug bounty program works.

You started on a Commodore 64 programming in BASIC. What’s the coolest program you wrote as a kid?

It was probably my first one. I was really into Choose Your Own Adventure books, and I wrote a text-based adventure game. That was really fun for me.

Did you use and adapt some code from other people?

K

atie Moussouris is the chief policy officer for HackerOne, a San Francisco–based startup that developed a vulnerability management and bug bounty platform.

Moussouris oversees the company’s philosophy and approach to vulnerability disclosure. For seven years before that, she worked at Microsoft, ending up as a senior security strategist. She also headed the company’s bug bounty program, served as content chair for its BlueHat internal hacker conference, and helped with its secure development life cycle process.

1540-7993/15/$31.00 © 2015 IEEE

SECURITY& PRIVACY

IEEE

I think I took some examples from a book on case statements. I definitely borrowed some code. I’m sure it had bugs in it.

A bug bounty program is a cash reward offered by a vendor in exchange for vulnerability information. It’s typically offered on a perbug basis: one bug, one bounty.

But a lot of the incentive programs I worked on, for example, at Microsoft, were looking at broader things like learning defensive techniques and new attack or mitigation bypass techniques. That’s essentially a new way to exploit and bypass all the safety measures on a platform’s latest version. These were more structured incentive programs that looked for a higher-order outcome than you look for with basic bug bounties.

Then you decided to study biology and made a quick switch back to the computer field.

Is this what you do at HackerOne, or is it more bug bounty related?

I wanted to cure cancer and AIDS. I got to work on the Human Genome Project, which was great.

But I found myself going back to the computing world. I made a couple different career shifts, working as a systems administrator for a while. I learned the pain of having to defend your own network from attackers and polished my

HackerOne is a platform and toolset that enables response teams to receive vulnerability reports. You can do that on our platform with or without a bug bounty. We’re really about vulnerability coordination.

The platform is free if you want to use it for vulnerability coordination. We charge a fee only if there’s a bounty involved. We also advise our

Copublished by the IEEE Computer and Reliability Societies

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

7 M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

INTERVIEW

About Katie Moussouris Katie Moussouris is the chief policy officer for HackerOne, a platform provider for coordinated vulnerability response and structured bounty programs. She oversees the company’s philosophy on vulnerability disclosure, advises customers and researchers, and works to legitimize and promote security research to help make the Internet safer for everyone. After working at MIT, Moussouris joined Symantec and later Microsoft, where her work encompassed industry-leading initiatives such as Microsoft’s bounty programs and Microsoft Vulnerability Research. She is also a subject matter expert for the US National Body of the International Organization for Standardization (ISO) on vulnerability disclosure and handling processes, secure development, and penetration testing.

customers on the best way to create an incentive program if that’s what they’re ready for. You really have to do a lot more security homework for a bounty.

Bug bounties are something we saw emerge in the BSIMM [Building Security In Maturity Model] community (http://bsimm.com). It makes sense to have a bug bounty or vulnerability coordination system as a software vendor, but it’s iffier when it comes to financial services and healthcare firms. What do you recommend for those sorts of firms?

Financial services and healthcare firms—really any organizations or verticals at high risk of cyberattack—already hire penetrationtesting companies. So they’re very comfortable with the BSIMM, including the idea of bug bounties.

Instead of looking at it as, “I’m only comfortable hearing about vulnerabilities from a specific small set of individuals that I hire,” we want to open up that mindset to, “If anybody could tell me about a vulnerability in my product, I’m interested in hearing about it. And I’d rather hear about it first than wait for an attack to happen.”

I was going to ask whether it’s better to use a HackerOne system or hire an internal penetration test team. But I know you’re going to say both.

8

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

It’s absolutely both. There’s a time and a place to do different kinds of testing as you’re developing the software. You and I are both big advocates of secure software development, and there are appropriate times and methodologies and tools to use when you’re testing along the entire life cycle. Using external folks of any kind, whether they’re penetration testers under contract or the unwashed masses of the Internet, you really just want to know if there’s any way to break into your system.

Do some of the researchers or hackers that you coordinate with work on all 68 bug bounty programs? Do you have a stable of people who do that?

Our platform has a built-in reputation system for hackers. And we’re very transparent about how reputation is affected by different outcomes of a hacker reporting a vulnerability. You can imagine that their reputation improves when they find an issue that’s serious enough to award a bounty. But to answer to your question, there are folks who are very prolific. They came in looking at one piece of software—maybe they came from Russia because they were looking at a specific piece of Russian software that was hosted on our platform for vulnerability reporting.

And then they ended up spreading

their knowledge and finding a lot of vulnerabilities in many other people’s programs.

Let’s talk about breaking and building in security. Many conferences focus on breaking things and getting bounties. Is it possible to turn the people who do the breaking into builders who can develop secure systems? Do we even want to do that?

I think it’s possible. As much as we talk about getting people to think more like attackers, getting attackers to think like defenders is as hard or harder in a lot of ways. Generally speaking, they don’t have to worry about the piece of software functioning in exactly the way the user or creator envisioned. But that functionality is going to be a huge part of securing software—you have to secure it so that it still works.

An example of incentivizing this switch is the bounties that I set up at Microsoft. There was a mitigation bypass bounty and a defense bounty. If you came up with a way to defeat all the mitigations but also came up with a way to defend against your new attack technique, you got more money.

This actually was paid out for the first time to researchers who work at ZDI [Zero Day Initiative].

Three bounties were launched in 2013, including one defensive bounty. Just a few months ago, ZDI’s security researchers not only got [US]$100,000 from Microsoft for a new mitigation bypass but also claimed the very first defensive award of an additional $25,000. I thought it was cool that somebody actually created that shift in thinking from being a breaker to thinking, “Here are some viable ways that the platform could introduce measures to defend against this.”

So, Microsoft gets something for $25,000 that might cost a million bucks in the open market, if it were a real technology transfer.

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Who else except for Microsoft a brief period and my friend for a colleagues and say, “I’m interested would buy a defensive technique? lot longer when I worked at MIT. in moving into this area. Can you Looking for those gaps in markets He would tell me if some crazy idea make some introductions?” and market motivations was defi- I had was actually viable and give This past year at HackerOne, I’ve nitely one of the things that went me different strategies for making been fortunate in being able to take into my thinking when creating it happen. When I was working at my expertise from vulnerability disMicrosoft’s bounties. closure and coordination Create incentives for and parlay that into speakOne of the biggest problems I what you want and pay ing to more and more polihad as a penetration tester was attention to those gaps in cymakers and lawmakers the market where you can about things that might convincing a developer that somebody develop an incentive for affect security research, would think like a bad guy. something that’s useful like the recent proposals to only to you. expand the US Computer Fraud and Abuse Act. This I’ve always wondered whether you MIT on network planning, I basi- year has been about me leveraging should teach developers to think like cally outlined how we should do it, my peer mentor network and getting a bad guy. I used to think that you but the folks were kind of politely introduced to people who are willing should, but then Microsoft’s Steven and not so politely dismissing my to hear about what this might mean Lipner changed my mind and con- advice. So I went into Bob’s office to security. vinced me that you can’t teach all and said, “I need to borrow your developers how to think like a bad gray beard for a minute. Will you guy. What do you think? just tell them this, that, and the he Silver Bullet Podcast with You’re right—you can’t teach them other thing?” And he said, “Look, I Gary McGraw is cosponsored all. I think one of the biggest problems can tell them, and you’re right that by Cigital and this magazine and is I had as a penetration tester was con- they’ll probably listen to me. But syndicated by SearchSecurity. vincing a developer that somebody eventually, you’re going to have to grow your own gray beard.” would think like a bad guy.

Gary McGraw is Cigital’s chief techHe basically said that at some nology officer. He’s the author of They look at you like you killed their point, you’re going to have to be Software Security: Building Secupuppy. able to communicate and command rity In (Addison-Wesley 2006) Exactly. Who would kill my puppy? authority in areas where you have and eight other books. McGraw It was made of software. It was beau- expertise. It’s not always going to be received a BA in philosophy from tiful. So, convincing them of the easy, and you’re right that I’m going the University of Virginia and a likelihood that somebody would to have an easier time walking into dual PhD in computer science [think like a bad guy] is often the a room and having my expertise and cognitive science from Indiassumed, whereas you’re going to biggest challenge. ana University. Contact him at have to work a little harder to estabgem@cigital.com. ___________ What’s the most effective way you lish it. But you can do it. I believe in found to do that? you. And that was that.

Selected CS articles and columns are also available for free at Ever since then, I’ve been growUnfortunately, you actually do have to show them something exploit- ing this gray beard. I’m kidding. But http://ComputingNow.computer.org. able most of the time. I can’t tell you I’ve tried to make sure that I lead how many times I’ve actually had with my knowledge and try to be to deliver an exploit to somebody helpful in a situation first. And usuwhen we were trying to tell them ally that ends up working. If you about a vulnerability. Then you have look young, you’re going to have to Letters for the editor? the danger of them thinking that lead with your authoritative foot.

Please email your I’ve had a lot of experiences exploit is the only vector. comments or feedback to editor Brian Kirk (bkirk@ ____ with peer mentors. It ends up that computer.org). ________ All letters Tell me about some of your mentors. your whole network is a mentor, will be edited for brevity, What did they help you with? in a way. You decide where you clarity, and language.

My mentors have really run the want to steer your ship, and then gamut. Bob Bruen was my boss for you go to your network of peers or

T

9

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY GUEST EDITOR’S INTRODUCTION

Learning from Other Disciplines

Shari Lawrence Pfleeger | Editor in Chief

I

n mid-April, I was discussing security engineering with Kristin Esterberg, president of SUNY-Potsdam, one of New York State’s comprehensive universities. A sociologist by training, she’s a passionate advocate for STEAM, not just STEM: programs that incorporate the arts and humanities into science, technology, engineering, and mathematics education.

Esterberg told me how she asked some of her engineering students if art could be relevant to engineering. Their answer: no. So she asked a more targeted question: Is it important for a bridge to be beautiful? Their answer: no; as long as it’s structurally sound, that’s enough. Are you nodding in agreement with the engineers? Think again. As engineers, we often focus only on so-called functional requirements, and only in a very narrow way.

Consider the Brooklyn Bridge, connecting the New York City boroughs of Manhattan and Brooklyn. Completed in 1883, it was designed to carry horse-drawn carriages, railcars, pedestrians, and bicycles. Of course, today it carries automobiles instead of horse-drawn vehicles, and it still moves people using other transportation modes. This bridge functions not just to transport people and vehicles from one side of the East River to the other but also to connect two neighborhoods and the diverse cultures they represent, exposing citizens to different ways of thinking and living. For more than a century, people have been drawn to the bridge by its beauty as well as its amazing engineering—the first steel-wire suspension bridge ever

10

July/August 2015

SECURITY& PRIVACY

IEEE

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

constructed. Its connective functions and artistic appeal are just as important as its structural integrity.

Interdisciplinary Insights In the same way, it’s important for us to think about security and privacy broadly so that we understand the context in which our technology will be perceived and used. Nicholas Kristof recently wrote a cogent analysis of the many ways that social sciences can inform our understanding of technology. He noted that “[l]iterature seems to offer lessons in human nature that help us decode the world around us,” and “science depends upon the humanities to shape judgments about ethics, limits, and values.”1 In the past, IEEE Security & Privacy’s authors and editors have addressed the value of other disciplines in many articles and special issues. From analyses of the economics of information security to explorations of usability, S&P has emphasized the benefits of multidisciplinary research and thinking. Indeed, it was for these reasons that we introduced the In Our Orbit department when I became editor in chief of this magazine. Department editors Angela Sasse and Alessandro Acquisti have sought input from researchers who work in other disciplines and apply it to security and privacy.

For example, in November/December 2013, sociologist Harvey Molotch described how studying normal routine behavior informs us of the mechanisms through which safety and social order can be maintained. From ethnographic studies of public sites such as the New York subway system, he reports on how to determine whether workers’ routines will fit into official security policies coming from above. From this research, he suggests concrete mechanisms likely to enhance cybersecurity while providing users with the collateral benefits of enhanced efficiency and pleasure.

Similarly, in May/June 2014, Martin Ortlieb provided an anthropologist’s view of privacy, and in November/ December 2014, Benjamin Edelman explored the security and privacy implications of online advertising optimization. Fascinating articles like these highlight important advances and insights that come about when we think differently about our technology’s design and use.

when this instrument was administered: 1985, 1996, and 2006. They explore the effects of race, age, education, and income across these time periods and suggest explanations for what they find. In particular, they note the effects of technology diffusion, cultural lag in technology acceptance, and increased government and commercial surveillance capabilities.

In “Improving Cybersecurity Incident Response Team Effectiveness Using Teams-Based Research,” Julie Steinke and her organizational psychology colleagues at George Mason University examine three types of emergency responders—emergency medical technicians, military responders, and nuclear power plant operators—and analyze what makes their teams effective. Then, the authors apply their findings to cybersecurity incident response teams, suggesting how the lessons learned in other response communities can help make our own incident response more effective.

Finally, in “A Sociotechnical Framework for Threat Modeling a Software Supply Chain,” information scientists Bilal Al Sabbagh and Stewart Kowalski apply sociotechnical networking techniques to the problem of modeling threats to a global supply chain. Describing their framework by applying it to a case study, they demonstrate how their techniques can assist in performing a thorough risk analysis.

M

ultidisciplinary perspectives are becoming more important as our technology becomes more pervasive. In fact, you will see more articles like these in special issues of S&P in 2016, including a special issue on the Internet of Things. Our community is terrific at generating data, and researchers continue to develop and improve techniques for examination, analysis, and prediction. But biologist and naturalist Edmund Osborne Wilson advised caution: “We are drowning in information, while starving for wisdom.

The world henceforth will be run by synthesizers, people able to put together the right information at the right time, think critically about it, and make important choices wisely.”2 At IEEE Security & Privacy, we plan to continue to help provide the right information at the right time.

In This Issue Because a multidisciplinary perspective can stimulate new and sometimes groundbreaking understanding, I’ve invited three sets of authors to describe how application of their disciplines’ techniques can lead to important security and privacy insights. In “Big Brother in the Information Age,” sociologists Denise Anthony, Timothy Stablein, and Emily Carian investigate trends in attitudes toward privacy by comparing the results of the US General Social Survey at three points in time

References 1. N. Kristof, “Starving for Wisdom,” New York Times, 16 Apr. 2015, p. A27; www.nytimes.com/2015/04/16/opinion /nicholas-kristof-starving-for-wisdom.html?_r=0. ____________________________ 2. E.O. Wilson, Consilience: The Unity of Knowledge, Alfred A. Knopf, 1998, p. 294.

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

11

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Big Brother in the Information Age: Concerns about Government Information Gathering over Time

Denise Anthony | Dartmouth College Timothy Stablein | Union College Emily K. Carian | Stanford University

Revelations about the extent of the US National Security Agency’s surveillance activities have led to a great deal of debate regarding government data gathering. The 1985, 1996, and 2006 iterations of the General Social Survey reveal public perceptions of whether such government power is an increasing threat to privacy.

I

n June 2013, The Washington Post and The Guardian published stories revealing vast and previously unknown surveillance activities being conducted by the US National Security Agency (NSA).1,2 The revelations of former NSA contractor Edward Snowden prompted significant public discussion about government surveillance activities, citizen privacy, and the role of technology in such activities.3,4 Indeed, a 2014 survey by the Pew Research Center found that most participants knew something about the NSA activities from the Snowden revelations and were worried about the government monitoring their data online.5 However, concerns about privacy and surveillance related to information and communication technologies (ICTs) and the dramatic increase of information gathering, storage, and exchange they’ve enabled existed long before the Snowden revelations.6–9 It would be a mistake to assume that these citizen concerns and government activities are new, even if aspects of the recent case

12

July/August 2015

SECURITY& PRIVACY

IEEE

appear unprecedented. Current debates about citizen privacy and government surveillance must be placed in a larger historical and cultural context to understand how the current state of affairs differs and what the implications might be today given a richer understanding of how it fits into past patterns.

Privacy in Context Discussions of privacy in the US typically begin with Supreme Court Justice Louis Brandeis’ opinion regarding American citizens’ fundamental “right to be let alone” (Olmstead v. United States, 1928).

The basic elements of a “right to privacy” were developed over time, including that American citizens have a “reasonable expectation of privacy” (Katz v.

United States, 1967). The 1974 Privacy Act further defined this expectation with respect to government. It includes a Code of Fair Information Practice that governs the collection, maintenance, use, and

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

dissemination of individuals’ personally identifiable Data and Methods information maintained by federal agencies. However, For this study, we used cross-sectional data from three many scholars suggest that US privacy law hasn’t kept years of the General Social Survey (GSS): 1985, 1996, pace with new technologies, causing citizens to experi- and 2006.18 The GSS has been conducted annually or ence fewer privacy protections and subsequently expect biennially since 1972 to a nationally representative samless privacy.10–12 ple of English-speaking persons 18 years of age or over Views of government information gathering in living in noninstitutional housing in the US. the US are important as we consider ongoing polIn the three years we analyzed, the GSS asked about icy debates about new technologies and informa- perceptions of government data gathering as part of the tion security and privacy in such varied areas as law International Social Survey Program supplement quesenforcement, national security, and healthcare. Soci- tions. For the statement, “The federal government has ologists have explored how state surveillance activi- a lot of different pieces of information about people ties facilitate the control not only of “suspects” but of which computers can bring together very quickly,” we all citizens.7,13,14 Anthony compared those responGiddens echoes phidents who stated they losopher Michel Fouthought this was a Sociologists have explored how cault in arguing that “very serious threat to state surveillance activities facilitate surveillance is a key individual privacy” to the social control not only of feature of modernity all other respondents. ‘suspects’ but of all citizens.

Unfortunately, this in which privacy is question hasn’t been not only “what may asked since 2006. not be revealed” but Table 1 shows descripalso how actors are related tive statistics for variables in each of the years of interto authority systems.15 Polling data suggests that attitudes toward privacy in est and across all three years. Across the three years, the the US have changed over time. For example, more peo- percentage of respondents who thought government ple were concerned about Big Brother—government information gathering was a serious threat to privacy information gathering on citizens—in the late 1980s varied—approximately 32 percent in 1985, nearly 40 than in earlier periods.16 Similarly, Alan F. Westin percent in 1996, and only about 27 percent in 2006.

To evaluate whether perceptions of government reported survey results through the 1990s that the public identified the government as the largest poten- data gathering as a threat to privacy vary across groups, tial threat to privacy.17 He attributed increased privacy we analyzed several sociodemographic characteristics. concerns during this period and into the early 2000s to Females comprised just over half of all respondents both the spread of new ICTs and increased government across the three years, but the number of female respondents between each survey year varied (see Table 1). surveillance activities.17 To examine the relationship of age to perception, we However, few studies have analyzed how individual sociodemographic characteristics and other factors, categorized age into three groups: 18 to 34 years, 35 to such as changing levels of trust in government, influ- 64 years, and 65 years and older. Across all three survey ence views of the government’s data gathering across years, approximately 25 percent of respondents were in time. It’s important to understand whether some groups the 18 to 34 age group, 55 percent were in the 35 to 64 are more or less likely to perceive government data gath- age group, and 15 percent were 65 years or older.

We recoded years of education into four categories: ering as a threat to privacy and whether these patterns less than high school, high school graduate, some colhave changed over time.

In this article, we explore public views of government lege, and college graduate or higher. As Table 1 shows, information gathering as a threat to individual privacy less than 20 percent had less than a high school educaover time. Using data from three decades, we examine tion in the combined years as well as in 1996 and 2006, whether and how sociodemographic characteristics but one-quarter were in this category in 1985. For are associated with viewing government data gathering each year, annual family income was converted into as a threat to privacy and whether views have changed 2006 dollars, and then divided into three categories: over time. Although this data doesn’t cover the period less than or equal to US$25,000, $25,001 to $50,000, of the most recent government surveillance activities, it and $50,001 and more. For combined years, approxienables us to analyze the public’s opinion about govern- mately one-third of the sample fell in each income catment data gathering specifically (not merely online pri- egory, but there was some variation between years (see Table 1). vacy more generally) over time.

13

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Table 1. Perceptions of government data gathering as threat to individual privacy.*

Very serious threat Male Gender

Female

Age

Annual income

Race

1996

2006

All three years

39.5% (48.9)

26.7% (44.2)

32.3% (46.8)

N = 620

N = 1,211

N = 1,492

N = 3,323

59.5%

45.5%

46.4%

48.5%

40.5%

54.5%

53.6%

51.5%

N = 620

N = 1,211

N = 1,492

N = 3,323

18 to 34 years

31.6%

33.2%

25.3%

25.4%

35 to 64 years

50.2%

52.6%

59.4%

55.2%

65 years and older

Education

1985 31.8% (46.6)

18.3%

14.1%

15.2%

15.4%

N = 618

N = 1,210

N = 1,484

N = 3,312

Less than high school

25.5%

16.0%

17.9%

18.7%

High school graduate

33.5%

29.1%

29.8%

30.3%

Some college

19.7%

27.2%

25.9%

25.3%

College graduate or higher

21.3%

27.6%

26.2%

25.8%

N = 620

N = 1,211

N = 1,486

N = 3,314

Less than or equal to US$25,000

34.6%

30.8%

35.3%

33.5%

$25,001 to $50,000

38.7%

27.8%

30.2%

31.0%

$50,001 or more

26.7%

41.4%

34.5%

35.5%

N = 581

N = 1,078

N = 1,280

N = 2,939

Nonwhite

10.5%

17.8%

26.7%

20.4%

White

89.5%

82.3%

73.3%

79.6%

N = 620

N = 1,211

N = 1,492

N = 3,323

People can’t be trusted

n/a

Government can’t be trusted: scale of 0 to 3

n/a

60.1%

65.9%

N = 789

N = 1,402

1.6 (1.1)

1.6 (1.1)

N = 1,142

N = 1,455

n/a n/a

* Standard deviations are in parentheses. N is the number of cases.

We measured race with a dichotomous variable for nonwhite race (nonwhite race = 1 and white race = 0).

Approximately 20 percent of respondents were nonwhite across the three years, with approximately 11 percent in 1985 and nearly 27 percent in 2006.

Based on questions asked only in the years 1996 and 2006, we created a variable for trust in government by recoding and combining participants’ five-point Likertscale responses to the following three statements: “people we elect to Congress try to keep the promises they have made during the election,” “most government administrators can be trusted to do what is best for the country,” and “the average person can influence politicians.” These statements weren’t asked in the 1985 survey. We recoded all three statements into dichotomous variables, coding the 14

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

disagree and strongly disagree responses as 1 and all others as 0. We then added the three together to create a scale variable with a range of 0 to 3 to capture the extent of lack of trust in government. As Table 1 shows, the mean level of trust in government was 1.6 in both 1996 and 2006.

Similarly, in 1996 and 2006, GSS respondents were asked about generalized trust: “Generally speaking, would you say that most people can be trusted or that you can’t be too careful in dealing with people?” We recoded responses so that “people cannot be trusted” was coded as 1 and other responses were coded as 0.

Nearly two-thirds of respondents in both years said that most people couldn’t be trusted.

Finally, because we wanted to examine whether concerns differed across time, we used two indicator July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Table 2. Logistic regression coefficients of government data gathering as a very serious threat to individual privacy (separate years).* 1985**

Education

2006 Model 2

Model 1

Model 2

0.27 (0.21)

0.11 (0.13)

0.15 (0.17)

–0.06 (0.13)

0.01 (0.14)

35 to 64 years

0.97† (0.27)

0.58† (0.15)

0.64† (0.19)

0.48† (0.16)

0.51† (0.18)

65 and older

0.89† (0.34)

0.85† (0.22)

0.89† (0.29)

0.59† (0.22)

0.60† (0.23)

High school graduate

–0.16 (0.28)

–0.003 (0.21)

–0.09 (0.28)

–0.10 (0.20)

–0.21 (0.21)

Some college

–0.31 (0.32)

0.15 (0.21)

0.03 (0.29)

0.01 (0.20)

–0.21 (0.22) –0.48‡ (0.24)

Female Age

1996 Model 1

College graduate or higher

–0.25 (0.35)

–0.13 (0.22)

–0.39 (0.30)

–0.43‡ (0.22)

$25,001 to $50,000

–0.21 (0.27)

0.19 (0.17)

0.38§ (0.23)

–0.08 (0.16)

–0.02 (0.17)

$50,001 or more

–0.04 (0.33)

–0.17 (0.17)

–0.13 (0.22)

–0.42‡ (0.18)

–0.33§ (0.19)

0.38 (0.34)

0.41‡ (0.17)

0.48‡ (0.23)

0.64† (0.15)

0.69† (0.16)

Government can’t be trusted

n/a

n/a

0.26† (0.08)

n/a

0.30† (0.07)

People can’t be trusted

n/a

n/a

0.33§ (0.18)

n/a

0.28§ (0.16)

–1.40† (0.35)

–1.02† (0.23)

–1.64† (0.38)

–1.31† (0.22)

–2.02† (0.30)

579

1,077

663

1,273

1,193

Annual income Nonwhite race

Constant N

* Standard deviations are in parentheses. ** Year 1985 was weighted for nonresponse bias in the split sample using weighting variable FORMWT. † p < 0.01. ‡ p < 0.05. § p < 0.10. The p-value is a function of the observed sample results that, if less than 0.05, indicate that the null hypothesis (that is, there is no relationship between the independent and dependent variable) should be rejected. “Constant” is the expected value of the dependent variable when the predictors equal zero. N is the number of cases used in the analytic model. (Source: General Social Survey.)

variables for the survey year (1996 = 1, else = 0; 2006 = 1, else = 0) to evaluate whether concerns in those years were higher, lower, or no different from the base year of 1985 and, for analyses that included trust variables, from the base year of 1996 (compared to 2006 only).

The analyses for each year (see Table 2) used multivariate logistic regression models that predict the odds that a respondent said government data gathering was a very serious threat to individual privacy or not based on the factors defined earlier (the log odds of the outcome are modeled as a linear combination of the predictor variables). We used the same analytic technique for data from all three survey years to examine whether views changed over time (see Table 3). We used sampling weights for year 1985 to account for the split-sample response rates.19

Results Table 2 shows the logistic regression coefficients for views of government data gathering as a very serious threat to privacy for each of the three years separately. In 1985, older age groups were significantly

more likely to perceive government data gathering as a threat to privacy compared to the youngest category of 18 to 34 year olds (the suppressed category in the model). No other sociodemographic characteristics were statistically significant in 1985 (the p-value was less than 0.05).

For 1996, model 1 shows that older age groups were significantly more likely to perceive government as a serious privacy threat than 18 to 34 year olds. Race was also positive and statistically significant; nonwhites were more likely to perceive government data gathering as a threat. Gender, education, and income variables weren’t statistically significant.

For 1996, model 2 adds the variables for trust and shows that, in addition to age and race, respondents with lower levels of trust in government were significantly more likely to see government data gathering as a privacy threat. The relationship between low levels of trust in other people and perception of threat was only marginally significant.

For 2006, model 1 shows that age was consistent with the patterns in 1985 and 1996, and race remained 15

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Table 3. Logistic regression coefficients of government data gathering as a very serious threat to individual privacy (multiple years).* Years 1985**, 1996, and 2006

Years 1996 and 2006 Model 1

Model 2

0.06 (0.09)

0.04 (0.10)

0.06 (0.11)

35 to 64 years

0.62† (0.10)

0.49† (0.11)

0.54† (0.13)

65 years and older

0.75† (0.14)

0.66† (0.16)

0.68† (0.18)

High school graduate

–0.09 (0.13)

–0.10 (0.15)

–0.18 (0.17)

Some college

–0.01 (0.13)

0.03 (0.15)

–0.13 (0.17)

College graduate or higher

–0.31‡ (0.14)

–0.30§ (0.16)

–0.47† (0.19)

$25,001 to $50,000

–0.02 (0.11)

0.04 (0.12)

0.12 (0.14)

$50,001 or more

–0.27‡ (0.12)

–0.27‡ (0.13)

–0.27§ (0.14)

0.54† (0.11)

0.61† (0.12)

0.63† (0.13)

Government can’t be trusted

n/a

0.33† (0.05)

0.29† (0.05)

People can’t be trusted

n/a

n/a

0.26‡ (0.12)

1996

0.37† (0.12)

n/a

n/a

2006

–0.37† (0.13)

–0.72† (0.10)

–0.81† (0.11)

Constant

–1.18† (0.16)

–1.29† (0.19)

–1.3† (0.24)

2,929

2,274

1,855

Female Age

Education

Annual income Nonwhite race

N

* Standard deviations are in parentheses. ** Year 1985 was weighted for nonresponse bias in the split sample using weighting variable FORMWT. † p < 0.01. ‡ p < 0.05. § p < 0.10. The p-value is a function of the observed sample results that, if less than 0.05, indicate that the null hypothesis (that is, there is no relationship between the independent and dependent variable) should be rejected. “Constant” is the expected value of the dependent variable when the predictors equal zero. N is the number of cases used in the analytic model. (Source: General Social Survey.)

consistent with 1996: older age groups and nonwhite respondents were significantly more likely to view government as a privacy threat than younger and white respondents. In addition, college graduates and those with an annual family income of $50,001 or more were significantly less likely to view government data gathering as a privacy threat in 2006.

Finally, as in 1996, respondents in 2006 with lower levels of trust in government were significantly more likely to view government as a threat to privacy, whereas lack of trust in other people was marginally significant.

Perceptions of Privacy Threat across All Years Table 3 shows the logistic regression results for data pooled across all three years, and for 1996 and 2006.

For the combined years, we see the same findings for sociodemographic characteristics as in Table 2: older age groups and nonwhite respondents were significantly more likely to view government as a privacy threat, whereas college graduates and higher-income 16

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

respondents were less likely to view government as a threat. In addition, controlling for individual characteristics, 1996 respondents were significantly more likely than those in 1985 to see government as a threat to privacy, whereas 2006 respondents were significantly less likely than 1985 respondents to see government as a serious privacy threat.

These findings indicate that the number of people concerned about government surveillance was higher in 1996 than 1985. However, the year 2006 is significant and negative in all models, indicating that the number of people who viewed government as a threat to privacy was lower in 2006 than in both 1985 and 1996.

Models 1 and 2 in Table 3 analyze the years 1996 and 2006 together and introduce the variables for trust.

Findings for sociodemographic characteristics are consistent with previous models. In addition, respondents who said government can’t be trusted were significantly more likely to say government data gathering is a serious threat to privacy (models 1 and 2), and those who July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

said that people can’t be trusted were significantly more likely to view government as a serious privacy threat.

60

Differences by Age

40

The most noteworthy finding in all models in every year is that age is strongly positively associated with views of government as a serious threat to privacy. We demonstrate this association further by showing differences in perceived threat by age groups across years. Figure 1’s simple bar graph of unadjusted means shows that, as in the regression models, more older respondents (those aged 35 to 64 and those 65 and older) perceived a threat to privacy than younger respondents (18 to 34 years old) within each year. By 2006, there was no statistically significant difference between age groups in the simple unadjusted means comparison, shown in Figure 1. However, recall that age categories were statistically significant in the multivariate regression for 2006. Comparing within each age group across time periods (see F-statistic values reported in Figure 1’s caption), more 1996 respondents perceived a threat than 1985 or 2006 respondents. For 35- to 64-year-olds and those 65 and older, the percentage who perceived a serious threat in 2006 was lower than in both 1985 and 1996.

30

Discussion In recent decades, ICTs have played an increasing role in nearly every aspect of personal and public life. Governments have come to rely on ICTs and, in doing so, now have access to extensive information about the activities of citizens and noncitizens alike. In concert with the increasing use of ICTs in society and by government, questions and concerns about privacy and surveillance have increased. However, our findings indicate that although more people in 1996 than 1985 perceived the government’s use of ICTs as a serious threat to privacy, the number of people with this perception declined significantly by 2006.

Why would there be an increase in the perception of threat in the latter part of the 20th century but then a decline through the early 21st century? We offer two possible explanations but acknowledge that our discussion is just the beginning rather than a definitive explanation of such patterns. First, sociologists of technology have long suggested that as new technologies become more widespread and integrated in society, concerns and fears about them decrease.20 That is, there’s often a cultural lag in acceptance of new technologies, in which concerns and fears arise with the spread of new technologies but eventually subside after the technology becomes widely accepted. For example, privacy concerns were present at the introduction of the telephone, but such concerns decreased over time once myths about the dangers of telephone use were dispelled and

%

50 F = 10.5**

F = 10.9**

18–34 years 35–64 years 65+ years F = 2.2

20 10 0

1985

1996 Year

2006

Figure 1. Perceptions of government data gathering as a threat to privacy by age categories across years. F-statistic for differences within age groups across years are 18 to 34 years: F = 6.0 (p < 0.01), 35 to 64 years: F = 16.8 (p < 0.01), and 65 and older: F = 9.7 (p < 0.01). The F-statistic is computed for analysis of variance (ANOVA), where F is the ratio of the estimate of between-group variance to the estimate of within-group variance. The p-value is a function of the observed sample results that, if less than 0.05, indicates that the null hypothesis (that is, there is no relationship between the independent and dependent variable) should be rejected.

people experienced personal and social benefits from its use.21 As people became more familiar and comfortable with computer technologies and databases over time, they might also have become more accepting of the government’s use of such technologies.

In 1985, less than 10 percent of US households had a computer. By 1997, that number had increased to approximately one-third of households, but fewer than one in five had Internet access. Such levels indicate that the diffusion of the Internet between 1985 and 1996 was in the lower part of an S-shaped diffusion curve of the adoption and spread of new technologies in society,22 a point at which more people began learning about computer technologies but relatively few had direct experience with them. When awareness of a technology is increasing but familiarity is still low, sociologists of technology suggest that cultural expressions of fear and concern increase. Our findings show just such an increase in concerns about government information gathering between 1985 and 1996.

In contrast, by 2006, nearly two-thirds of households had Internet access, indicating that the diffusion of ICTs was beginning to crest the S-shaped diffusion curve and that direct experience with such technologies had become widespread. At this point, the lag in cultural acceptance of new technologies caught up with their actual use, which might explain our findings that views of government data gathering as a threat to privacy declined by 2006.

A different argument suggests that privacy concerns might have declined in the early 21st century only because, at this point, people expected lower levels of 17

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

privacy in reaction to policies and actions by both government and private firms.13 If critics are correct that government surveillance has increased and thereby eroded privacy, then our findings support the notion that the public has grown less concerned about government information gathering in the face of an ever-expanding surveillance state. In this way, cultural lag might have interacted with complacency about the government’s use of those technologies such that, by 2006, people had become so accustomed to ICTs and high levels of data gathering and monitoring that privacy expectations had significantly declined.

Although the number of people who perceive government information gathering as a serious threat to privacy changes over time, we consistently find that older compared to younger people are more likely to see government as a privacy threat. These findings seem to be consistent with claims that younger people “think differently” about privacy than older people.23 For example, well-respected studies show how young people think about privacy online,24 but such studies don’t compare privacy perceptions between young and old or how perceptions of government threats to privacy change over time.

In our study, although the relationship between age and perceived threat remained significant even in the most recent data, all age groups showed a decline in perception of government as a privacy threat, with the largest declines occurring in older age groups. With this data, it’s impossible to determine whether this change in older age groups is due to changes in experience with technology in all age groups, cohort replacement over time of people with less concern about privacy, or both. Future work should consider the dynamic changes occurring in both experience with technology and changing views over time to understand whether differences in perceptions of privacy are related to age, cohort, or both. This is particularly important given ongoing claims about how young people and “digital natives” are somehow different from older adults in how they think about privacy.

We also find that nonwhites are consistently more likely than whites to perceive government data gathering as a threat to privacy. Overall, members of minority racial groups have less access to technology,24,25 which might mean they have less knowledge of and experience with ICTs. Because the GSS didn’t include questions on technology usage in the years in which questions about government data gathering were asked, we can’t determine whether individual access to or experience with ICTs mediates concerns. We considered that because, according to the study, minority racial groups are generally more distrustful of government, they perceive government information gathering as a threat. When we 18

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

include measures of distrust as both distrust of government specifically and lack of generalized trust, nonwhite race remains positive and significant, indicating that nonwhites perceive government as a threat to privacy in addition to having more distrust in government.

Furthermore, consistent with previous findings that trust in institutions can moderate concerns about privacy, we find that those who have stronger feelings of distrust toward government are significantly more likely to perceive government information gathering as a threat to privacy. In addition, we find that low generalized trust is also significantly associated with perceived threat, though not as strongly as distrust of government. Such findings suggest that the consequences of declining trust in major social institutions such as government might have implications for expectations and behavior related to privacy that can reverberate across society. If fewer people express concerns about the extent of government surveillance activities or are willing to seek legal limitations on datagathering practices (whether by government or other corporate entities), people in the US might find they no longer have a choice about their “right to be let alone.”

O

ur findings don’t refute critics’ claims of increases in government monitoring and information tracking or that such actions erode privacy. Instead, we show that the public’s perception of government information gathering as a serious threat to privacy declined between 1996 and 2006, possibly because of increased surveillance activities and the erosion of privacy protections.

Although we don’t have the data that would let us examine how the most recent revelations about government surveillance activities, such as NSA surveillance programs, have affected views of government data gathering since 2006, our findings suggest that concerns might not have increased. Whereas some suggest that privacy concerns are quite high given public awareness of surveillance activities,5 it might be the case that even amid the Snowden revelations, cultural lag theorists are correct: as people become more familiar and experienced with new technologies and as those technologies become more integrated into their lives, the fears and concerns related to such technologies decline—even when they’re used by the government. This view suggests that fears about privacy have diminished because of changes in social norms and practices related to ICTs. The two processes might even interact such that people became more familiar and comfortable with ICTs at the same time they experienced increased surveillance and declining privacy protections, leading to decreasing concerns about privacy.

More important, our study clearly demonstrates that perceptions of government data gathering as a threat July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

to privacy vary across groups, change over time, and are related to perceptions of trust and confidence in government. Given the extensive changes occurring in both public policy (for instance, police wearing cameras and electronic transmission of health information) and new technologies (for example, ubiquitous cameras and embedded GPS tracking), it might be particularly problematic that the public in general—and marginal groups in particular—have low expectations of privacy from the government. The government might be less likely or willing to ensure that privacy assurances and our American “right to be let alone” are protected.

Acknowledgments The National Science Foundation award CNS-0910842 supported this research. The statements, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References 1. B. Gellman and L. Poitras, “US, British Intelligence Mining Data from Nine US Internet Companies in Broad ___ Secret Program,” Washington Post, 6 June 2013; www .washingtonpost.com/investigations/us-intelligence -mining-data-from-nine-us-internet-companies-in-broad ________________________________ -secret-program/2013/06/06/3a0c0da8-cebf-11e2 ________________________________ -8845-d970ccb04497_story.html. ___________________ 2. G. Greenwald, “NSA Collecting Phone Records of Millions of Verizon Customers Daily,” Guardian, 5 June 2013; www.theguardian.com/world/2013/jun/06/nsa-phone ________________ -records-verizon-court-order.

3. S. Landau, “Hightlights from Making Sense from Snowden: What’s Significant in the NSA Surveillance Revelations,” IEEE Security & Privacy, vol. 11, no. 4, 2013, pp. 54–63.

4. S. Landau, “Making Sense of Snowden Part II: What’s Significant in the NSA Surveillance Revelations,” IEEE Security & Privacy, vol. 12, no. 1, 2014, pp. 62–64.

5. M. Madden et al., “Public Perceptions of Privacy and Security in the Post-Snowden Era,” Pew Research Center, Nov. 2014; www.pewinternet.org/2014/11/12/public -privacy-perceptions. ___________ 6. D. Lyon, Electronic Eye: The Rise of Surveillance Society, Univ. of Minnesota Press, 1994.

7. D. Lyon, Surveillance Society: Monitoring Everyday Life, Open Univ. Press, 2001.

8. H. Nissenbaum, Privacy in Context: Technology, Policy, and the Integrity of Social Life, Stanford Univ. Press, 2010.

9. S. Pfleeger, “The Eyes Have It: Surveillance and How It Evolved,” IEEE Security & Privacy, vol. 12, no. 4, 2014, pp. 74–79.

10. J. Rosen, The Unwanted Gaze: The Destruction of Privacy in America, Random House, 2000.

11. F.H. Cate, Privacy in the Information Age, Brookings Inst.

Press, 1997.

12. D.J. Solove, Understanding Privacy, Harvard Univ. Press, 2010.

13. G.T. Marx, “A Tack in the Shoe: Neutralizing and Resisting the New Surveillance,” J. Social Issues, vol. 59, no. 2, 2003, pp. 369–390.

14. O.H. Gandy Jr., The Panoptic Sort: A Political Economy of Personal Information, Westview Press, 1993.

15. A. Giddens, The Consequences of Modernity, Stanford Univ. Press, 1990.

16. J.E. Katz and A.R. Tassone, “Public Opinion Trends: Privacy and Information Technology,” Public Opinion Q., vol.

54, no. 1, 1990, pp. 125–143.

17. A.F. Westin, “Social and Political Dimensions of Privacy,” J. Social Issues, vol. 59, no. 2, 2003, pp. 431–453.

18. T.W. Smith, P.V. Marsden, and M. Hout, “General Social Survey, 1972–2010,” Storrs, CT: Roper Center for Public Opinion Research, Univ. of Connecticut/Ann Arbor, MI: Inter-university Consortium for Political and Social Research, 2011.

19. T.W. Smith and B.L. Peterson, Problems in Form Randomization on the General Social Surveys, tech. report 58, General Social Surveys, NORC, 1986.

20. S. Shapiro, “Places and Spaces: The Historical Interaction of Technology, Home, and Privacy,” Information Society, vol. 14, no. 4, 1998, pp. 275–284.

21. C. Marvin, When Old Technologies Were New: Thinking about Electric Communication in the Late Nineteenth Century, Oxford Univ. Press, 1990.

22. E.M. Rogers, Diffusion of Innovations, Free Press, 1962.

23. J. Palfrey and U. Gasser, Born Digital: Understanding the First Generation of Digital Natives, Basic Books, 2008.

24. E. Hargittai and Y.P. Hsieh, “Digital Inequality,” The Oxford Handbook of Internet Studies, W.H. Dutton, ed., Oxford Univ. Press, 2013.

25. d. boyd, It’s Complicated, Yale Univ. Press, 2014.

Denise Anthony is a professor of sociology at Dartmouth

College. Her research focuses on cooperation, trust, and privacy in both online and offline environments.

She received a PhD in sociology from the University of Connecticut. Contact her at ___________ denise.l.anthony@ dartmouth.edu. _________ Timothy Stablein is an assistant professor of sociology

at Union College. His research focuses on social deviance, health, and technology. He received a PhD in sociology from the University of Connecticut. Contact him at ____________ stableit@union.edu.

Emily K. Carian is a PhD student in sociology at Stan-

ford University. Her research focuses on gender, family, and social movements. She received an AB from Dartmouth College. Contact her at ecarian@ ______ ________ stanford.edu.

19

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Improving Cybersecurity Incident Response Team Effectiveness Using Teams-Based Research

Julie Steinke, Balca Bolunmez, Laura Fletcher, Vicki Wang, Alan J. Tomassetti, Kristin M. Repchick, Stephen J. Zaccaro, Reeshad S. Dalal, and Lois E. Tetrick | George Mason University

Examining other emergency response teams’ methods of adaptation, communication, problem solving, trust building, and developing shared knowledge among team members can improve cybersecurity incident response teams’ performance.

M

ore often than not, cybersecurity incident response involves collaboration among multiple analysts. Cybersecurity incident response teams (CSIRTs) exist in a dynamic and constantly changing environment in which they must effectively engage in information management and problem solving while adapting to complex circumstances.1 For example, multiple analysts might collaborate to determine the extent to which an attack has penetrated a system, while others might coordinate announcements and alerts to additional teams in the organization or even outside agencies.

A critical factor in CSIRT members’ success is their ability to function well within a team. Although many people believe their effectiveness depends largely on technological aspects (for example, system infrastructure and cyberanalysts’ technical skills), organizations and researchers are also incorporating behavioral research on individual and team characteristics that drive successful team performance.1,2 Several characteristics define CSIRTs’ core operating and performance environments.1 CSIRTs must

20

July/August 2015

SECURITY& PRIVACY

IEEE

effectively manage information to operate in complex environments and must adapt to changing performance requirements while still monitoring their networks and responding to threats. Other types of response teams share several of these characteristics despite operating in distinctly different domains. In this article, we examine three types of response teams and consider how their strategies can benefit CSIRTs.

Team Definitions and Characteristics CSIRTs are composed of two or more individuals who interact with one another as well as with IT infrastructure, other IT personnel, end users, management, and other CSIRTs to prepare for and respond to computer security incidents.3 Most notably, CSIRTs are responsible for protecting networks from threats (for example, exfiltration of data and denial-of-service attacks) and maintaining a secure operating environment for a defined constituency (for example, a firm or a nation).

The typical CSIRT operating environment is complex—high in information load, information

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

diversity, and degree of uncertainty (for example, the rate of information change).4 Further, the structure of the environment in which CSIRTs function can change quickly and continually, requiring analysts to collaborate adaptively as they monitor networks and respond to threats.

Teams that function in complex environments face unique circumstances that potentially influence their effectiveness. Behavioral scientists, such as organizational psychologists, study these teams’ characteristics and processes to develop strategies for improving team functioning in such environments. However, CSIRTs remain relatively unexamined and could benefit from developmental strategies known to enhance the effectiveness of other response teams. Based on overlapping team characteristics (for example, effectively managing information, problem solving, and functioning adaptively in complex environments) and the convergence of physical and cybersecurity components, we identified three teams that are conceptually similar to CSIRTs: emergency medical systems (EMS), military response (MR), and nuclear power plant operating (NPPO) teams. All these teams work in complex, time-constrained environments in which failure to adapt to the changing environment can result in catastrophic performance failures.

Emergency Medical Systems Teams EMS teams engage in services ranging from advising patients on emergency phone lines to providing acute care to patients in emergency departments.5 These team members are involved in prehospital emergency response and include emergency dispatchers, paramedics, and ambulance drivers as well as hospital-based emergency responders, such as nurses and physicians.

EMS team members must adapt to patients’ changing health needs, often under high time constraints, in incidents ranging from small-scale situations, such as car accidents, to highly complex mass casualties, for instance, resulting from natural disasters. For effective teamwork to exist in EMS teams, team members must collaborate effectively to handle coordination, risk communication, initial patient care, patient handoff, and treatment documentation.5

Military Response Teams MR teams include rapid-response teams often specializing in particular events such as medical outbreaks, natural disasters, and other events requiring a timely response to a developing situation as well as incident response teams; fleet antiterrorism security teams; marine expeditionary units; and navy sea, air, and land teams. MR teams need to effectively communicate with internal and external agencies, adapt to changing missions, manage large amounts of information, and meet time-sensitive

Adaptation

Shared knowledge of expertise

Communication Team effectiveness

Trust

Collective problem solving

Figure 1. Model of team effectiveness. Five factors that can improve emergency response teams’ success.

demands while responding to complex emergency situations.6 Furthermore, MR team members must develop a shared understanding of the situation, instantly and effectively work together in intricate time-critical events, and engage in continuous learning by performing learned procedures under new conditions.7

Nuclear Power Plant Operating Teams NPPO teams are the “‘brains’ of the complex nuclear systems they manage.”8 They make complex decisions that involve interpreting vast amounts of information while managing numerous systems. In addition, NPPO teams make these decisions under difficult circumstances that often include sensitive time constraints and heavy workloads.8 After abnormalities are detected, NPPO teams determine the cause, take corrective action, and record their actions in operating data logs.9 NPPO team members benefit from skills, such as communication and adaptability, that help them perform these functions effectively.8

Recommendations to Improve CSIRT Effectiveness Figure 1 presents aspects of team functioning that research we describe later has demonstrated to enhance EMS, MR, and NPPO teams’ effectiveness. We focus on five factors: adaptability, problem solving, communication, trust building, and creating shared team knowledge among team members.

21

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Improve CSIRT Adaptation CSIRTs operate in environments in which their tools and networks are continually changing and increasing in complexity. Moreover, their adversaries are continually learning, growing, and improving. These factors require CSIRTs to change their core performance strategy to protect their networks and constituencies. In other words, CSIRTs must be able to adapt to new and unexpected events. Team-level adaptation entails shifts in team performance that reflect necessary reactions to changing circumstances.10 Teams use information from their environment to adjust their performance strategies, thereby adapting to maintain their ability to perform effectively. Consultants who have worked with NPPOs have stated, “standardized procedures might not be safe in unexpected and unknown situations that have not been anticipated while designing the procedures.”11 This statement applies equally to the CSIRT domain.

A strategy that has been shown to enhance team adaptation in military contexts is perturbation training, which aims to improve a team’s ability to adapt by counteracting habituation and procedural rigidity associated with routine team interactions.12 During perturbation training, teams go through realistic, simulated situations that closely represent what they would actually do on the job. In early trials, teams tackle the incident with no interruptions. In later trials, the standard coordination procedures are disrupted—for example, by disabling technology that is critical to mission success—thus forcing teams to coordinate in new ways.

In one study, perturbation training enabled teams trained in this method to significantly outperform other teams not similarly trained (by 13 percent) in two of three critical training missions.12 The same tactic of disabling a relied-upon resource could be applied to any CSIRT training involving multiple trial periods to encourage adaptation of procedures that will transfer to novel, real-life situations.

When teams are forced to adapt, they will likely experience stress. Stress can often lead to narrowing of team perspectives such that team members under stress place more focus on their individual needs than on team needs.13 Another training protocol that might benefit CSIRTs is one that will prepare individuals to maintain effective performance levels while resolving high-stress, time-critical incidents.

One such protocol that has been used with military teams is stress exposure training (SET).14 SET’s primary objectives are to increase teams’ familiarity with environments, teach skills that help individuals successfully maintain task performance, and prepare teams to maintain effective functioning in stressful situations.14 According to research in this area, there are three stages of SET training:14 22

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

■ providing individuals with information regarding why stress training is important and identifying specific stressors in the environment (for example, recapping past events in which stress significantly affected the outcome), ■ providing individuals with specific cognitive and behavioral skills to help them cope with stressors while maintaining effective performance, and ■ having individuals apply and practice skills under increasingly representative conditions of the actual environment (for example, increasing time pressure over multiple training phases).

In one study conducted with military personnel, SET training resulted in decreased perceptions of stress and an 18 percent performance improvement.15 These benefits were retained among study participants who performed novel tasks in the presence of new stressors.

CSIRTs could realize similar gains if they utilized SET. Other research on military personnel found that preparatory activities such as contingency planning, “war gaming” (simulations between opposing forces that represent potentially real conflict scenarios), and frame switching during mission planning were instrumental in fostering adaptive mindsets. Thinking through a more complete set of attack strategies can lead to adaptation by fostering knowledge of additional potential attack mitigation solutions. Such thinking can also reflect a level of cognitive flexibility that is necessary for adaptive performance.16 In CSIRTs, this technique is sometimes called penetration testing or red teaming.17 In such exercises, team members evaluate their vulnerabilities by attempting to infiltrate their own system. Despite its potential to increase CSIRT effectiveness, a survey indicated only 17 percent of CSIRTs used penetration testing.18 CSIRTs can utilize information from their own attacks to guide understanding of how they should act to prevent future attacks. Beyond attacking teams’ own systems, red teaming can involve playing devil’s advocate during discussions of approaches to problems, searching for flaws in technology, bringing in others who act out hacking attempts in a safe environment, and anticipating innovative technological advances by adversaries.19 Many CSIRTs use these strategies; however, typical uses address only technical skills and lack a focus on the development of teamwork skills.19

Enhance CSIRT Problem Solving A major part of CSIRTs’ work is problem solving in ambiguous domains where a clear solution isn’t always readily apparent. To help team members interact effectively when solving complex problems, researchers studying other types of teams have investigated several team-training strategies. One such strategy is the “Think July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Like a Commander” tactical adaptive thinking training actually communicated using fewer, less complex intermethod utilized in MR teams.20 This method’s over- actions. There was a 93 percent decrease in team memarching purpose is to increase the number of domain- bers involved in communication patterns and 88 percent specific thought habits team members use automatically fewer actor switches (more actor switches indicate in a crisis.20 Thought habits are cognitive behaviors that two-way exchanges of information whereas fewer actor can be trained to the point of automaticity so that cog- switches indicate more one-way communication).24 nitive resources can be used for more complex probWhen events arise, team members need to be able lems.20 Each of these “habits” guides decision makers’ to effectively communicate all information associated behaviors and should be reviewed with team members with those events throughout the team. In EMS teams, in a group discussion. Doing so increases the amount poor communication during handoffs (for instance, of critical information that team members are capable the transfer of patient care from one EMS team memof pinpointing in tactical ber to another) accounts scenarios.21 for nearly 70 perCSIRTs can bencent of mistakes efit from such trainthat occur.25 On Simulation tasks can be altered to represent ing by making clear the other hand, aspects of uncertainty CSIRT members the informational when handled cormight face during incident response. cues team memrectly, handoffs can bers must recognize improve patient across incidents. safety by facilitatIdentifying and discussing ing information sharing these cues might reduce the likelihood that team mem- and decision making across team personnel shifts.26 In bers miss vital information when making a decision or CSIRTs, many cyber–incident handling errors occur handing off incidents. during handoffs.27 The “Team Coordination Model” (TCM) is another EMS teams have developed several methods to strategy designed to help military teams who work in improve the quality of handoffs. Foremost among these dynamic environments.22 TCM involves identifying a are the use of mnemonics designed to prompt the two defined mission comprising a network of tasks. Individ- parties to exchange all appropriate information clearly ual team members are assigned separate tasks that must and concisely.28 For example, the mnemonics SBAR be coordinated for mission success, thereby developing (situation, background, assessment, and recommenshared mental models across team members—that is, dation) and SHARED (situation, history, assessment, individuals formulate shared understandings about top- risks, events, and documentation) are used in emergency ics such as their own roles and responsibilities within situations due to their effectiveness in conveying relthe team as well as those of other team members.22 evant information.28,29 Research has shown that mediTCM simulations are structured so that team members cal teamwork training that emphasized the use of such learn how, when, and to whom to communicate infor- mnemonics reduced the number of negative patient outmation during decision-making scenarios. These types comes (for instance, medication and transfusion errors) of simulations can provide a way to analyze team col- due to communication problems by 65 percent.30 Other laboration and subsequently team performance.22 This researchers have also found that medical teamwork method could prove useful for CSIRTs because simula- training shortened patient handoff times by 12 percent tion tasks can be altered to represent aspects of uncer- on average while slightly improving patient outcomes.29 tainty CSIRT members might face during incident Several types of handoffs occur in CSIRTs durresponse, thus reflecting their work’s dynamic structure. ing the response process. Specifically, the sending and receiving of information can occur between person and Enhance Communication person, person and technology, person and team, and team and team, creating multiple places for errors and among CSIRT Members To handle events, CSIRT members must communicate mistakes.31 Although there are several descriptions for effectively. Successful communication involves send- how handoffs occur, there are few prescriptions for how ing messages that are accurate, relevant, and timely and to make them effective. As such, checklists created by that target the right persons.23 A study on NPPO teams cybersecurity analysts (ideally from different levels) suggests that it’s not the amount or complexity of infor- and mnemonics adapted to the CSIRT context offer mation but rather the communicated information’s rich- realistic, actionable, and promising options for CSIRTs ness that’s vital in effective crisis management.24 In this to improve the quality of communication during handstudy, high-performing NPPO teams in crisis situations offs. In addition, when preparing for incidents, CSIRTs 23

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

can engage in structured team strategy discussions to as they occur, leading to fewer opportunities for team better understand team goals and how to achieve them. members to develop initial trust. Thus, virtual or ad hoc Communication briefings held before teams begin teams might take longer to share information and coopa mission or problem can be instrumental in subse- erate effectively and face the potential for higher conquent team performance. Such briefings can reduce flict; they could benefit from strategies that promote the communication breakdowns by creating shared mental initial development of trust.

In medical teams, nurses will seek information or models (shared expectations about what information should be given and received), improving team mem- advice from help providers they perceive as trustworbers’ understanding of upcoming tasks, and enhancing thy. Such trust-based behaviors reduce medical errors, coordination and cohesion.32 As we noted, these brief- and the effects are even stronger under conditions of ings can also foster team adaptation by creating oppor- greater uncertainty.37 Various actions such as buildtunities for teams to discuss ing team members’ reputacontingency plans tions through word of in the case of unexmouth, finding simiShared mental models are important pected events.33 larities with others, to team functioning because they help In medical conacknowledging team texts, one- to fourmembers’ expertise, team members interpret information and minute structured ensuring good perforsituational contingencies in similar ways. briefings involving a mance, and being satchecklist of operaisfied with the group tions and communication as a whole have all been points identified by nurses, surgeons, trainees, and suggested as important factors for developing long-term anesthesiologists reduced communication failures (for trust in military teams.38 Leaders can encourage trust-building behaviors that example, failing to provide, or delaying the provision of, useful information) by 64 percent.34 Such results sug- could eventually develop into team norms. Team trustgest that CSIRTs could benefit from structured brief- based norms help develop a climate of psychological ings that emphasize team communication breakdown safety—the mutually held “belief that the team is safe for interpersonal risk taking.”39 Coaching team leaders points prior to incident response and during handoffs.

Military teams that conducted a 10-minute team and minimizing subordinate concerns about power difstrategy meeting prior to a simulated military opera- ferences among medical operating-room teams were tion task demonstrated 33 percent more developed found to significantly increase trust and “speakingshared mental models than teams that simply talked to up” behaviors, which led to improved learning.40 This each other prior to the simulation.7 Further, processes research has shown that in medical settings, leaders among teams involved in strategy meetings (for exam- can downplay power differences by inviting and actple, coordination activities and leadership) improved ing on others’ suggestions, communicating that other 7 percent compared to teams not involved in strategy roles are just as critical to success as the surgeon’s role, meetings.7 Again, methods that have worked well to and encouraging others to be aware of their own limenhance communication among EMS and MR teams its. Indeed, research with medical teams in which memcould benefit CSIRTs. Researchers should examine var- bers were encouraged to speak up about their mistakes ious types of mnemonics, checklists, and briefing styles detected approximately twice as many errors compared to determine those most effective in CSIRT communi- to teams that didn’t exhibit these behaviors.41 Such cation development. activities enabled these team members to learn from each other’s failures and improve their performance, Develop Trust among CSIRT Members which in turn increased psychological safety among Trust among team members—that is, team members’ members and created greater trust in team leaders. belief that they can rely on each other in risky situaMethods to create initial levels of trust among team tions—is one of the most important factors associated members before they begin to work closely with each with team success.35 Trust increases information shar- other can be crucial for information sharing during ing and cooperation and reduces conflict, all of which the initial stages of operations. In pre-mission stages of are important for team effectiveness.36 Both initial lev- military operations, this swift trust was shown to be 7 els of trust and long-term trust in teams are important percent higher when team members had a shared idenfor team effectiveness. However, members of the same tity (for example, belonged to the same regiment).42 team don’t always work in close proximity to each other, Even if teams come together on an ad hoc basis to solve and ad hoc teams are often formed to handle events an incident, trust expectations are important. For ad 24

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

hoc or newly formed teams, developing a shared identity among team members by acts as simple as creating a team name or motto could promote higher levels of initial trust. Identifying shared associations and experiences (for example, involvement in similar training programs or participation in similar military branches) could help CSIRT members initially rely on one another.

Develop Shared Team Knowledge among CSIRTs CSIRTs—along with EMS, MR, and NPPO teams— must integrate vast amounts of information in their work. Therefore, team members’ ability to effectively manage and share information can greatly influence team effectiveness. Organizational psychologists focus on two forms of knowledge in teams: shared mental models43 and transactive memory—shared knowledge of who knows what.44 Shared mental models are important to team functioning and team member interaction because they help team members interpret and understand information and situational contingencies in similar ways.45 Shared mental models also allow team members to understand how their roles and expertise fit together. With this shared understanding, team members can successfully coordinate actions during complex tasks because each team member knows what behaviors must occur in coordination and synchrony for effective task completion.44 Of course, it’s conceivable that team members share inaccurate mental models.46 Thus, developmental strategies must improve the extent to which accurate mental models are shared.

Several such methods to foster effective shared mental models, such as cross-training, have been developed and empirically validated. These methods can be used to increase mental model accuracy—that is, the team members’ understanding of who should do what and when.47 Team members who received cross-training scored 33 percent higher than those who didn’t in their impression of how well they understood other team members’ roles and what was expected of them during tasks.48 Teams already possessing shared mental models (that is, seasoned teams) communicated and performed better over three cross-training sessions than newly formed teams lacking in shared mental models.48 Various cross-training methods can be used. Team members can hear about other members’ specific task responsibilities (for example, through information presentations), watch them do what they do, or actually perform these specific tasks normally completed by other members.47 Some initial short-term costs are associated with cross-training because experienced individuals move into roles in which they’re less

experienced. Nonetheless, over time, cross-training is quite beneficial to teams.

Similarly, guided team self-correction training fosters shared mental models.49 This training method entails providing guidance to team members on topics that can help them identify team performance problems.49 In a recent study, prescriptive expert models of teamwork were provided to military teams who were instructed to base their self-critique, feedback, and planning activities on the examples in the provided models. In this study, teams that debriefed using the expert model demonstrated more accurate teamwork mental models, resulting in a 38 percent increase in teamwork processes and a 110 percent increase in performance.49 This approach has been used successfully during combat system teams’ simulation exercises, in nuclear power industry accident investigations, and elsewhere as a tool to enhance onthe-job performance.49 Role identification behaviors include sharing information related to experiences, skills, and abilities with team members. These simple interpersonal interactions help team members gain greater understanding of their teammates’ capabilities and expertise. CSIRTs can easily integrate role identification practices into their training exercises to develop shared knowledge. The Think Like a Commander method has also been shown to aid in the development of shared team knowledge, especially in training situations with ambiguous, incomplete, or inconsistent information.20,50 After-action reviews (AARs) and debriefings are learning opportunities in which discussions are held to review recent performance events.51 Pointing out where communication broke down, where interactions didn’t occur when they should have, and where actions weren’t properly coordinated helps team members build and refine a common understanding of how they need to better interact and coordinate on future problems and missions. To gain such an effect, AARs need to be structured carefully. Several evidence-based best practices for AARs have been identified from research on military teams:52 ■ focus on a few critical performance issues to maximize limited time and emphasize the most important outcomes, ■ shorten the delay between the incident and feedback so participants’ recall of the incident is better, ■ provide both individual and team-oriented feedback so team members and the team as a whole know what to do, ■ emphasize process feedback rather than outcome feedback so team members can focus on improving coordination rather than on merely attaining good outcomes, 25

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

■ ensure that people feel comfortable during the AAR so they don’t hesitate to speak frankly, and ■ take notes on lessons learned to preserve institutional memory.

In one study of six military training exercise sessions, a 10-minute AAR that utilized a set of these practices improved performance by 50 percent, efficacy by 22 percent, openness of communication by 11 percent, and cohesion by 15 percent in subsequent exercises.51 A variation of AARs can be used as a training exercise wherein managers have subject matter experts provide their ideal solution to a training scenario before reviewing what the trainees actually did. This method provides an easy way to compare experts’ and novices’ processes.

This technique could work in CSIRTs. Together, subject matter experts and junior analysts could review recordings and notes; discuss findings; and identify where inappropriate or unnecessary information was used, how communication can be made more effective, and unnecessary steps in the solution process.

In addition to having a shared knowledge of expected interactions and tasks throughout a team, team members must possess knowledge of other team members’ unique areas of expertise. This knowledge makes communication and information sharing faster because it allows team members to know who they can contact for help or who might need additional information during an event.53 For example, when a CSIRT analyst faces an unfamiliar incident, knowing that another team member has previously solved similar incidents is useful information because it facilitates advice seeking and reduces incident resolution time. However, not all areas of expertise might be known because analysts continually develop additional skills to keep up with the everchanging nature of their jobs.

CSIRTs can develop shared knowledge of unique expertise among team members by using several simple strategies that have been found to significantly improve team performance in other team types. These strategies work by providing role clarification among team members and educating members on the unique expertise that exists in their teams. For example, crosstraining enhances team members’ knowledge of other team roles, thus providing greater shared role clarification. To use this method in CSIRTs, managers can have team members give presentations to the team, providing information and insights into their unique areas of expertise.54 In addition, analysts who are less experienced in one role can job-shadow experienced team members to learn about other roles and how they solve problems.

Knowledge maps that display individual team members’ roles and expertise (for example, team directories) 26

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

can also enhance shared knowledge and provide a resource for members to use when incident-handling advice is needed. In addition, regular discussions in which team members share topics they’re currently working on can facilitate the development of each person’s knowledge of expertise throughout the team, ultimately allowing for more efficient team activities. These tactics have been used in various team types in addition to EMS, MR, and NPPO teams and have consistently enhanced the development of shared knowledge among team members.

S

ubstantial research identifies drivers of team effectiveness in EMS, MR, and NPPO teams; CSIRTs can use these strategies to improve incident response.

However, CSIRT managers should also consider various costs associated with these strategies. For instance, as we mentioned, cross-training involves short-term time (and perhaps performance) costs because in intensive cross-training, experienced individuals move into roles in which they possess less experience and knowledge. Other potential developmental strategies might involve similar costs associated with time or other resources (for example, staff time spent in training and hiring knowledgeable trainers). However, the amount of upfront time invested in these strategies could lower future incident response times by limiting teamworkrelated problems. Thus, we recommend that managers identify the most critical needs for their own CSIRTs and then select the developmental strategies that provide the highest benefit-to-cost ratio based on their unique circumstances.

Our recommendations come with some limitations.

The ever-changing nature of cybersecurity work means that no two CSIRTs are alike. Thus, strategies that work for one CSIRT might not work as well for others. These differences could be due to many factors including CSIRT size, type (for example, those housed in private, government, or coordinating organizations), team composition, the impact of cultural differences given the increasingly global boundaries of cybersecurity, or even the type of attacks CSIRTs face. In this respect, future research on how these kinds of factors might change the development and implementation of strategies that impact CSIRT effectiveness is necessary.

These caveats aside, evidenced-based practices used to enhance teams’ abilities to be adaptive, solve problems, communicate, build trust, and develop shared knowledge among team members have been shown to improve the effectiveness of EMS, MR, and NPPO teams. Such research and the best practices validated by this research offer a starting point for improving CSIRT effectiveness.

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

Acknowledgments This material is based on research sponsored by the US Department of Homeland Security (DHS) Science and Technology Directorate, Homeland Security Advanced Research Projects Agency, Cyber Security Division (DHS S&T/HSARPA/ CSD), BAA 11-02, and the Air Force Research Laboratory Information Directorate under agreement FA8750-12-20258. The US government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. We acknowledge the contribution of team members on this research effort who are not authors of this article.

References 1. S.J. Zaccaro et al., “A Taxonomic Classification of Cyber Security Incident Response Performance,” Psychosocial Dynamics of Cybersecurity, S.J. Zaccaro et al., eds., Routledge, to be published in 2015.

2. S.L. Pfleeger and D.D. Caputo, “Leveraging Behavioral Science to Mitigate Cyber Security Risk,” Computers and Security, vol. 31, no. 4, 2012, pp. 597–611.

3. T.R. Chen et al., “An Organizational Psychology Perspective to Examining Computer Security Incident Response Teams,” IEEE Security & Privacy, vol. 12, no. 5, 2014, pp. 61–67.

4. H.M. Schroder, M.J. Driver, and S. Streufert, Human Information Processing, Holt, Reinhart & Winston, 1967.

5. D. Schottke, First Responder: Your First Response in Emergency Care, Jones Bartlett, 2007.

6. L.A. Zimmerman et al., “Methods and Tools for Training Crisis Response,” research note 2012-07, US Army Research Inst. Behavioral and Social Sciences, 2012; www.dtic.mil/dtic/tr/fulltext/u2/a564316.pdf.

7. S. Dalenberg, A.L. Vogelaar, and B. Beersma, “The Effect of a Team Strategy Discussion on Military Team Performance,” Military Psychology, vol. 21, no. 2, 2009, pp. S31–S46.

8. M.J. Waller, N. Gupta, and R.C. Giambatista, “Effects of Adaptive Behaviors and Shared Mental Models on Control Crew Performance,” Management Science, vol. 50, no.

11, 2004, p. 1534.

9. “Occupational Outlook Handbook—Power Plant Operators, Distributors, and Dispatchers,” Bureau of Labor Statistics, US Dept. Labor, 2014; www.bls.gov/ooh /production/power-plant-operators-distributors-and ________________________________ -dispatchers.htm. _________ 10. C.S. Burke et al., “Understanding Team Adaptation: A Conceptual Analysis and Model,” J. Applied Psychology, vol. 91, no. 6, 2006, pp. 1189–1207.

11. J. Brüngger et al., “PUMA-Development and Application Tool for Supporting Nuclear Power Plant Operating Teams in Unexpected and Unknown Situations,” 5th Int’l Conf. Applied Human Factors and Ergonomics, 2014, p. 178.

12. J.C. Gorman, N.J. Cooke, and P.G. Amazeen, “Training Adaptive Teams,” J. Human Factors, vol. 52, no. 2, 2010, pp. 1–13.

13. J.E. Driskell, E. Salas, and J. Johnston, “Does Stress Lead to a Loss of Team Perspective?,” Group Dynamics: Theory, Research, and Practice, vol. 3, no. 4, 1999, pp. 291–302.

14. J.E. Driskell, E. Salas, and J. Johnston, “Stress Exposure Training,” Making Decisions under Stress: Implications for Individual and Team Training, J.A. Cannon-Bowers and E. Salas, eds., Am. Psychological Assoc., 1998, pp. 191–217.

15. J.E. Driskell, J.H. Johnston, and E. Salas, “Does Stress Training Generalize to Novel Settings?,” J. Human Factors, vol. 43, no. 1, 2001, pp. 99–110.

16. J.K. Nelson, S.J. Zaccaro, and J.L. Herman, “Strategic Information Provision and Experiential Variety as Tools for Developing Adaptive Leadership Skills,” Consulting Psychology J.: Practice and Research, vol. 62, no. 2, 2010, pp. 131–142.

17. C. Peake, “Red Teaming: The Art of Ethical Hacking,” SANS Inst., 16 July 2003; www.sans.org/reading -room/whitepapers/auditing/red-teaming-art-ethical ________________________________ -hacking-1272. ________ 18. G. Killcrece et al., State of the Practice of Computer Security Incident Response Teams (CSIRTs), tech. report CMU/ SEI-2003-TR-001, Software Eng. Inst., Carnegie Mellon Univ., 2003.

19. D.F. Longbine, “Red Teaming: Past and Present,” School of Advanced Military Studies, Army Command and General Staff College, 2008; www.dtic.mil/cgi-bin/Get TRDoc?AD=ADA485514. ______________ 20. S.B. Shadrick and I.W. Lussier, “Training Complex Cognitive Skills: A Theme-Based Approach to the Development of Battlefield Skills,” Development of Professional Expertise, K.A. Ericsson, ed., Cambridge Univ. Press, 2009, p. 287.

21. J.W. Lussier and S.B. Shadrick, “Adaptive Thinking Training for Tactical Leaders,” Symp. Advanced Technology for Military Training, 2003; www.dtic.mil/get-tr-doc /pdf?AD=ADA428347. _____________ 22. R. Giachetti and J.A. Rojas-Villafane, Simulation Modeling and Statistical Network Tools for Improving Collaboration in Military Logistics, tech. report AFRLRH-WP-TR-2009-0110, Air Force Research Laboratory, 2008; www.dtic.mil/dtic/tr/fulltext/u2/a510072.pdf.

23. T. Rench et al., “Accelerating Unit Adaptability: A Principle-Based Approach to Unit Communication,” Interservice/Industry Training, Simulation, and Education Conf., 2014.

24. A.A. Stachowski, S.A. Kaplan, and M.J. Waller, “The Benefits of Flexible Team Interaction during Crises,” J. Applied Psychology, vol. 94, no. 6, 2009, pp. 1536–1543.

25. K.M. Sutcliffe, E. Lewton, and M.M. Rosenthal, “Communication Failures: An Insidious Contributor to Medical Mishaps,” Academic Medicine, vol. 79, no. 2, 2004, pp.

186–194.

26. K. Wood et al., “Clinical Handovers between Prehospital and Hospital Staff: Literature Review,” Emergency Medical J., 2014; doi:10.1136/emermed-2013-203165.

27

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

27. Growing the Security Analyst: Hiring, Training, and Retention, tech. report 4AA5-3982ENN, Hewlett-Packard Development, Aug. 2014; www8.hp.com/h20195/v2 /getpdf.aspx/4AA5-3982ENN.pdf?ver=1.0. _________________________ 28. D. Siassakos et al., “Clinical Efficiency in a Simulated Emergency and Relationship to Team Behaviours: A Multisite Cross-Sectional Study,” BJOG: Int’l J. Obstetrics & Gynaecology, vol. 118, no. 5, 2011, pp. 596–607.

29. L.A. Riesenberg, J. Leitzsch, and B.W. Little, “Systematic Review of Handoff Mnemonics Literature,” Am. J. Medical Quality, vol. 24, no. 9, 2009, pp. 196–204.

30. S. Deering et al., “On the Front Lines of Patient Safety: Implementation and Evaluation of Team Training in Iraq,” Joint Commission J. Quality and Patient Safety, vol. 37, no.

8, 2011, pp. 350–356.

31. C. Alberts et al., Defining Incident Management Processes for CSIRTs: A Work in Progress, tech. report CMU/SEI2004-TR-015, Software Eng. Inst., Carnegie Mellon Univ., 2004.

32. Y. Xiao, S. Henrickson Parker, and T. Manser, “Teamwork and Collaboration,” Rev. Human Factors and Ergonomics, vol. 8, no. 1, 2013, pp. 55–102.

33. J. Tucker and K.M. Gunther, “The Application of a Model of Adaptive Performance to Army Leader Behaviors,” Military Psychology, vol. 21, no. 3, 2009, pp. 315–333.

34. L. Lingard et al., “Evaluation of a Preoperative Checklist and Team Briefing among Surgeons, Nurses, and Anesthesiologists to Reduce Failures in Communication,” Archives Surgery, vol. 143, no. 1, 2008, pp. 12–17.

35. R.C. Mayer, J.H. Davis, and F.D. Schoorman, “An Integrative Model of Organizational Trust,” Academy Management Rev., vol. 20, no. 3, 1995, pp. 709–734.

36. J.L. Wildman et al., “Trust Development in Swift Starting Action Teams: A Multilevel Framework,” Group Organization Management, vol. 37, no. 2, 2012, pp. 138–170.

37. D.A. Hofmann, Z. Lei, and A.M. Grant, “Seeking Help in the Shadow of Doubt: The Sensemaking Processes Underlying How Nurses Decide Whom to Ask for Advice,” J. Applied Psychology, vol. 94, no. 5, 2009, pp. 1261–1274.

38. A.I. Blair and J.B. Hanna, Trust and Partnering with the Joint Team, tech. report AU/AFF/003/2009-04, Air Command and Staff College, Air Univ. Maxwell Air Force Base, Apr. 2009; www.dtic.mil/dtic/tr/fulltext/u2/a53 9800.pdf. _____ 39. A.C. Edmondson, “Psychological Safety and Learning Behavior in Work Teams,” Administrative Science Q., vol.

44, no. 2, 1999, p. 354.

40. A.C. Edmondson, “Speaking Up in the Operating Room: How Team Leaders Promote Learning in Interdisciplinary Action Teams,” J. Management Studies, vol. 40, no. 6, 2003, pp. 1419–1452.

41. A.C. Edmondson, “Learning from Failure in Health Care: Frequent Opportunities, Pervasive Barriers,” Quality Safety Health Care, vol. 13, no. 2, 2004, pp. ii3–ii9.

28

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

42. B.D. Adams et al., “Swift Trust in Distributed Ad Hoc Teams,” tech. report CR 2007-139, Defence Research and Development, Toronto, Canada, Oct. 2007; www.dtic .mil/cgi-bin/GetTRDoc?AD=ADA477148. _________________________ 43. J.A. Cannon-Bowers, E. Salas, and S.A. Converse, “Cognitive Psychology and Team Training: Shared Mental Models in Complex Systems,” Human Factors Society Bull., vol.

33, 1990, pp. 1–4.

44. D.M. Wegner, T. Giuliano, and P. Hertel, “Cognitive Interdependence in Close Relationships,” Compatible and Incompatible Relationships, W.J. Ickes, ed., Springer, 1985, pp. 253–276.

45. J. A. Cannon-Bowers, E. Salas, and S. Converse, “Shared Mental Models in Expert Team Decision Making,” Individual and Group Decision Making, N.J. Castellan Jr., ed., Lawrence Erlbaum, 1993.

46. P.J. Ellis and M.J. Pearsall, “Reducing the Negative Effects of Stress in Teams through Cross-Training: A Job Demands-Resources Model,” Group Dynamics: Theory, Research, and Practice, vol. 15, no. 1, 2011, pp. 16–31.

47. J.A. Cannon-Bowers et al., “The Impact of Cross-Training and Workload on Team Functioning: A Replication and Extension of Initial Findings,” J. Human Factors, vol. 40, no. 1, 1998, pp. 92–101.

48. R. Espevik, B.H. Johnsen, and J. Eid, “Outcomes of Shared Mental Models of Team Members in Cross Training and High-Intensity Simulations,” J. Cognitive Eng. Decision Making, vol. 5, no. 4, 2011, pp. 352–377.

49. K.A. Smith-Jentsch et al., “Guided Team Self-Correction: Impacts on Team Mental Models, Processes, and Effectiveness,” Small Group Research, vol. 39, no. 3, 2008, pp.

303–327.

50. R. Woltjer et al., “Role-Playing Exercises to Strengthen the Resilience of Command and Control Systems,” Proc. 13th European Conf. Cognitive Ergonomics, 2006, pp. 71–78.

51. A.J. Villado and W. Arthur Jr., “The Comparative Effect of Subjective and Objective After-Action Reviews on Team Performance on a Complex Task,” J. Applied Psychology, vol. 98, no. 3, 2013, pp. 514–528.

52. E. Salas et al., “Debriefing Medical Teams: 12 EvidenceBased Best Practices and Tips,” Joint Commission J. Quality Patient Safety, vol. 34, no. 9, 2008, pp. 518–527.

53. A.R. Kim et al., “Analysis of Team Communication Characteristics Using SNA Technique,” Trans. Korean Nuclear Society Autumn Meeting, Oct. 2011.

54. E. Blickensderfer, J.A. Cannon-Bowers, and E. Salas, “Cross Training and Team Performance,” Decision Making under Stress: Implications for Training and Simulation, J.A. Cannon-Bowers and E. Salas, eds., Am. Psychological Assoc., 1998, pp. 299-312.

Julie Steinke is a postdoctoral research fellow in George

Mason University’s Industrial/Organizational Psychology program. Her research interests include July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

teams, competition and conflict, performance under stress and adversity, and resilience. Steinke received a PhD in industrial and organizational psychology from Wright State University. Contact her at jsteinke@gmu.edu. ____________

chology program. Her research interests include team processes, CSIRT effectiveness, and multiteam systems. Repchick received an MA in industrial and organizational psychology from George Mason University. Contact her at krepchic@gmu.edu. ____________

Balca Bolunmez is a doctoral student in George Mason

Stephen J. Zaccaro is a professor of industrial and orga-

University’s Industrial/Organizational Psychology program. Her research interests include judgment and decision making, job performance, organizational diversity, and research methods. Bolunmez received an MS in textile engineering from Dokuz Eylul University and an MBA in management of technology from New Jersey Institute of Technology. Contact her at bbolunme@gmu.edu. _____________

nizational psychology at George Mason University.

His research interests include group dynamics, team performance, multiteam systems, leadership, leader development, and work attitudes. Zaccaro received a PhD in social psychology from the University of Connecticut. Contact him at szaccaro@gmu.edu. ____________

Laura Fletcher is a graduate student in George Mason

University’s Industrial/Organizational Psychology program. Her research interests include teams, multiteam systems, networks, and creativity. Contact her at lfletch5@gmu.edu. ___________ Vicki Wang is a graduate student in George Mason Uni-

versity’s Industrial/Organizational Psychology program. Contact her at vwang2@gmu.edu. ___________

Reeshad S. Dalal is chair of the Psychology Department

and an associate professor of industrial and organizational psychology at George Mason University. His research interests include employee performance, work motivation, decision making, personality and its interactions with the work situation, job attitudes, and research methods. Dalal received a PhD in industrial and organizational psychology from the University of Illinois at Urbana–Champaign. Contact him at rdalal @gmu.edu. __________ Lois E. Tetrick is a professor in George Mason Univer-

Alan J. Tomassetti is a doctoral candidate in George

Mason University’s Industrial/Organizational Psychology program. His research interests include judgment and decision making, policy capturing, leadership, team effectiveness, and situational strength. Tomassetti received an MA in industrial and organizational psychology from George Mason University. Contact him at ____________ atomasse@gmu.edu.

Kristin M. Repchick is a doctoral candidate in George

Mason University’s Industrial/Organizational Psy-

sity’s Industrial/Organizational Psychology program.

Her research interests include occupational health and safety, occupational stress, the work–family interface, psychological contracts, social exchange theory and reciprocity, organizational commitment, and organizational change and development. Tetrick received a PhD in industrial and organizational psychology from Georgia Tech. Contact her at ___________ ltetrick@gmu.edu.

Selected CS articles and columns are also available for free at ___________________ http://ComputingNow.computer.org.

Subscribe today!

IEEE Computer Society’s newest magazine tackles the emerging technology of cloud computing.

computer.org/ cloudcomputing _________________

29

www.computer.org/security ____________

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

A Socio-technical Framework for Threat Modeling a Software Supply Chain

Bilal Al Sabbagh | Stockholm University Stewart Kowalski | Gjøvik University College

A new socio-technical framework studies software supply chain security problems from a systemic viewpoint. It addresses three main issues: modeling the target system, identifying threats, and analyzing countermeasures.

T

he term “supply chain” refers to the process of transforming natural resources and raw materials into a finished product delivered to end users.1 Software supply chains that build and maintain our new cyberworld share similarities with traditional product supply chains.

Acquirers of software products can deploy and use them directly or integrate them into a larger system, creating a system of systems. They can use the integrated system directly or reproduce it as a new product, transforming the acquirer into a supplier. The repetition of this process creates a software supply chain.

A software supply chain not only inherits risks from traditional product supply chains, such as late product delivery, counterfeit problems, human errors, and other internal and external risks, but has its own security risks, such as faulty code, which might be exploited by unauthorized parties. Faulty code can be inserted intentionally or unintentionally into a software product during development. For instance, the Heartbleed vulnerability has caused approximately two-thirds of websites and other open and closed source applications to be critically vulnerable to sensitive data exposure.2 A German open source developer unintentionally introduced the flaw while fixing bugs and adding more features to the affected libraries.3

30

July/August 2015

SECURITY& PRIVACY

IEEE

Many companies use risk management to control risks related to different software supply chain activities. (For more information on current practices in supply chain security, see the sidebar.) Risk management mainly focuses on reducing the risk of an unauthorized party having the ability to change software behavior in a way that adversely affects its security properties.4 How to eliminate, reduce, transfer, or accept risks are decisions that need to be made during risk management.

The first and most important step in risk management is to conduct an adequate threat model of the systems at hand. Because of the software supply chain’s complexity, it requires a systematic, in-depth threat model to deal with the total risk picture. From a practical perspective, these models must be comprehensible and useful but also complex enough to capture the technological nature of the issue of security.

Ontological Analysis In information science, ontology normally defines a set of concepts, properties, and axioms that provides rules to govern the use of these concepts and properties.5 Various ontologies have been applied to solve problems in the information security area.6,7 Threat modeling is an

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Current Practices in Supply Chain Security

T

he most essential practice for software supply chain security is derived from the classical CIA triad: confidentiality, integrity, and availability. These are the three key security principles in information security. Confidentiality entails the protection of sensitive information against disclosure by an unauthorized person or organization. Integrity comprises both data integrity and source integrity. Data integrity involves protecting data from unauthorized modification, and source integrity refers to the authenticity of the source. The concept of availability refers to the availability of information—data, information systems, data storage facilities, and access channels.

In addition to the CIA triad, other proposed security principles, including possession, authenticity, and utility, contribute to various systems’ security requirements. For a secure software supply chain system, the major security principles that need to be considered are confidentiality, data integrity, source authenticity, availability, and nonrepudiation. Nonrepudiation refers to the principle that a particular order for a product or service can’t be denied by either party—neither the one that placed the order nor the one that received it.

One of the latest contributions to information and communication technology (ICT) supply chain security comes from a National Institute of Standards and Technology (NIST)

ontological analysis of threats that aims to study the existence and nature of threats. Two processes are required for threat modeling: modeling the system in question and modeling threats to the system in question. For both, an appropriate ontology can provide a formal and comprehensive knowledge base and serve as a modeling language.

Using a socio-technical system as a concept was conceived in the late 1950s.8 With the increasing involvement of technology in the industry, the theory of socio-technical systems proposed that a system can no longer be viewed simply as a social system or a technical system. In most cases of working conditions in industries with theoretical and practical problems, social and technical aspects are tightly related to each other and form a socio-technical system. A software supply chain system is a typical example of a socio-technical system.

The most important aspect of a threat-modeling system is that it expresses and captures as many existing threats as necessary to manage existing risks and take appropriate countermeasures. The ISO 27005 standard defines a threat as a potential cause of an incident that might result in harm to systems and organizations. According to Eric Little and Galina Rogova’s threat ontology, each threat can be viewed as an integrated whole possessing three interrelated parts: intentions, capabilities, and opportunities.7

publication on supply chain risk management practices for federal information systems and organizations.1 NIST special publication 800-161, released in April 2015, provides guidelines on how to identify, assess, and mitigate ICT supply chain risks at the different levels of federal agencies’ organizations. Appendix C of the publication provides generic examples of ICT supply chain threat events based on NIST publication 800-30, revision 1.2 Although the document is extensive, it doesn’t provide a framework to organize, analyze, and communicate the threats.

The socio-technical framework for modeling software supply chain systems that we propose in this article is intended to be used as a thinking aid for analyzing and identifying such social and technical threats to software supply chains and for identifying appropriate countermeasures.

References 1. J. Boyens et al., “Supply Chain Risk Management Practices for Federal Information Systems and Organizations,” NIST special publication 800-161, Nat’l Inst. Standards and Technology, Apr. 2015.

2. R.S. Ross et al., “Guide for Conducting Risk Assessments,” NIST SP-800-30, revision 1, Nat’l Inst. Standards and Technology, 17 Sept. 2012.

The relation of foundational dependence exists among the three elements. For viable (actual) threats, all three elements must be present, whereas for potential threats, at least one element is missing.

For software supply chain security, threat modeling must be performed to identify both viable and potential threats. From an efficiency perspective, potential threats are of the most interest because the earlier action is taken, the more loss can be prevented. Depending on the desired risk level of an organization’s software supply chain, countermeasures can be taken to mitigate the three elements and transform viable threats to potential threats.

Vulnerability is a consequence caused by the combination of interrelated threat items, including various objects such as adversaries and the tools, equipment, and technology they use. These threat items can be viewed as dispersed wholes that stand independently of one another. Therefore, we can analyze software supply chain system vulnerabilities by finding inter- and intrarelations of the integrated elements (intention, capability, and opportunity) that compose each threat in the system.

Research Methodology Using a design science research methodology, we evaluate a socio-technical framework developed by 31

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Knowledge flow

Process steps Awareness of problem

Knowledge contribution

Suggestion

Artifact

refine the initial threat identification and as a knowledge base for identifying countermeasures.

Our interviews had three phases—system-modeling refinement, threats-listing refinement, and countermeasures identification—and we validated the model using the Delphi method. Validation interviews were carried out by a panel of anonymous security industry experts in Sweden who rely on global software supply chains for product development and service delivery.

Performance measures

The Socio-technical Framework

Outputs Proposal

Tentative design

Circumscription Development Operational principles and design theories

Evaluation

Conclusion

Results

Figure 1. The general methodology of design science research. (Source: “Design Science Research in Information Systems”; used with permission.10) The awareness of problem phase shows the security problem of software supply chains. The suggestion phase involves the tentative design of the artifact proposed to address the problem. In the development phase, the artifact is implemented. In the evaluation phase, the artifact’s performance is assessed and analyzed using a Delphi method, and in the conclusion phase, the artifact is proposed as satisfying the problem.

Kowalski9 to model security threats in the software supply chain of the Swedish Armed Forces (SwAF).8 Performing such modeling on a real case study lets us better understand the importance of considering the socio-technical approach when dealing with information systems security. The case study will also help us better understand the suggested framework and maybe enhance and improve it.

Figure 1 illustrates the general methodology and activities involved in design science research.10 The research begins with the awareness of problem phase, in which we observe the security problem of software supply chains and propose that a socio-technical framework is required to make a comprehensible threat model of the software supply chain. The suggestion phase involves the tentative design of the artifact proposed to address the problem. Next, during the development phase, the artifact is implemented. During the evaluation phase, the artifact’s performance is assessed and analyzed using a Delphi method,11 and in the conclusion phase, the artifact is proposed as satisfying the problem.

To propose a meaningful and valuable threat model for the SwAF’s software supply chain, we selected a qualitative research approach. We conducted several face-to-face and email interviews with a SwAF representative to gather professional opinions on threats to software supply chains and common countermeasures used in the industry. We used the interview results to 32

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

The socio-technical framework contains two basic models: a dynamic model of socio-technical changes, called the socio-technical system (see Figure 2), and a static one, called the security-by-consensus (SBC) model or stack (see Figure 3). The many arrows in Figure 2 heuristically outline the complex dynamic socio-technical systems. The social subsystem in Figure 2 is composed of culture—a collection of values—and structure—the distribution of power in a given system. The technical subsystems are composed of methods—techniques—and machines—technical artifacts.

Together, these four interconnecting subsystems (culture, structure, methods, and machines) determine the overall security posture or state of the system. For a system to be secure, it must maintain equilibrium among the four subsystems. That is, if a change occurs in one subsystem—for example, culture, such as a new generation of young managers— then structure, methods, and machines need to be changed accordingly or the systems will be at a disequilibrium. If appropriate changes don’t occur, the system will either reject the new culture (for example, dismiss the new managers) or continue in a state of disequilibrium (insecurity).

Reflecting the static nature of the socio-technical systems, the SBC stack is a multilevel structure that divides security measures into seven hierarchal levels of control, namely ethical, political and legal, administrative and managerial, operational, application, operating system, and hardware. It was originally developed to systemically model the US security control infrastructure after the national Computer Security Act of 1987.9 In the socio-technical framework, each system interacts with other systems rather than being an isolated system. Internal and external changes—both social and technical—will affect system security. Therefore, systematic deployment of security measures is required.

Socio-technical frameworks provide a way to perform system analysis through a systemic–holistic perspective.

A detailed introduction to socio-technical frameworks and their application to secure software supply chains can be found in the workshop paper that is the basis for this article.8 July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Application of Socio-technical Framework The benefit of applying a socio-technical framework while performing threat modeling for software supply chains is that it lets security reviewers focus on individual layers to identify what potential threats can exist and what countermeasures should be applied to various layers. Threats might exist at one layer but be mitigated at another. By perceiving a software supply chain as a system comprising several interconnected socio-technical systems (see Figure 4), security reviewers can not only review the layers of every corresponding supplier but also verify that the measures align between the same layers of different suppliers’ systems.

Our model’s software supply chain is one sociotechnical system comprising several interconnected socio-technical systems—that is, the suppliers and the acquirers. Each supplier in the chain is considered a socio-technical subsystem affected by its own existing culture, structure, methods, and machines. In addition, each subsystem has a layered control structure for its security measures.

Changes can happen in any part of any subsystem.

For the secure state of the software supply chain system, the four components in each subsystem should be in a state of equilibrium; to realize this, we can perform security analysis using the SBC stack along with the system dynamic model of culture/structure and method/ machine. Moreover, because these subsystems are interconnected, security measures taken in one subsystem should be aligned with security measures taken in other subsystems. For example, we can use the SBC model to check whether supplier 1’s operational security measures align with supplier 2’s measures. If not, there might be a security problem, and appropriate security measures—which won’t necessarily be the same for both—should be taken.

Figure 5 illustrates how inconsistent security measures taken by suppliers and acquirers can create a chain of security vulnerabilities. In this example, the parties involved in the supply chain are an open source compiler provider (supplier 1), a telecommunication equipment provider (supplier 2), a mobile communication service provider (supplier 3), and an end customer (SwAF).

Supplier 1 uses digital signatures for tamper-proof product delivery of the compiler; however, supplier 2 doesn’t have a signature verification procedure for integrity checks on received products. This leaves the compiler vulnerable to malicious code insertion. Consequently, the security threat is passed along the supply chain to the end customer.

Another potential inconsistency outlined in this figure is that supplier 3 employs data encryption to maintain confidentiality, but the end customer has a weak mechanism for authenticating internal users. This, in turn, leaves

Social

Technical

Culture

Methods

Structure

Machines

Figure 2. A socio-technical system. (Source: adapted from “IT Insecurity: A Multi-discipline Inquiry.”9) The many arrows heuristically outline the complex, dynamic socio-technical system. The four interconnecting subsystems together determine the overall security posture or state of the system.

Ethical Political and legal Administrative and managerial Operational Application Operating system Hardware

Figure 3. The security-by-consensus (SBC) model. (Source: adapted from “IT Insecurity: A Multi-discipline Inquiry.”9) The SBC model is a multilevel structure that divides security measures into a hierarchy of seven controls, namely ethical, political and legal, administrative and managerial, operational, application, operating system, and hardware.

the system vulnerable to unauthorized individuals who might gain access to confidential information, and again passes this problem on to the end customer.

These varying levels of security implementation among the involved parties can occur for several reasons. One significant reason is the existence of different security cultures among the organizations in the software supply chain. This difference can affect the level of implemented security measures regardless of the existing security legislation at a national level.

33

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Culture

Supplier 1

Supplier 2

Ethical

Ethical

Political and legal

Culture

Methods

End customer

Ethical

Political and legal

Methods

Culture

Ethical

Political and legal

Methods

Political and legal

Culture

Administrative and managerial

Administrative and managerial

Administrative and managerial

Administrative and managerial

Operational

Operational

Operational

Operational

Application Structure

Supplier N

...

Operating system

Application Machines Structure

Hardware

Operating system

Application Machines Structure

Hardware

Operating system

Methods

Application Machines Structure

Hardware

Operating system

Machines

Hardware

∑ [n > 1] Supplier n

Figure 4. Software supply chain from a socio-technical system perspective. (Source: adapted from “A Socio-technical Framework for Threat Modeling a Software Supply Chain.”8) By perceiving a software supply chain as a system comprising several interconnected socio-technical systems, security reviewers can not only review the layers of every corresponding supplier but also verify that measures align between the same layers of different suppliers’ systems.

Supplier 1

Culture

Supplier 2

Ethical

Ethical

Ethical

Ethical

Political and legal

Political and legal

Political and legal

Culture

Methods Compiler

Operational

Telecommunication equipment

Culture

Structure

Hardware

Operating system

Structure

Operating system

Application Machines Structure

Hardware

Use encryption for data confidentiality

Methods

Administrative and managerial Operational

Application Machines

Hardware

Use digital signature for tamper-proof delivery

Culture Mobile communication service

Operational

Application Machines

Methods

Administrative and managerial

Operational

Application Operating system

Methods

Administrative and managerial

Security problem

End customer (Swedish Armed Forces)

Supplier 3

Political and legal Administrative and managerial

Structure

No integrity checks on received products

Operating system

Machines

Hardware

Poor authentication mechanism

Figure 5. A software supply chain system security problem caused by inconsistent security measures. (Source: adapted from “A Socio-technical Framework for Threat Modeling a Software Supply Chain.”8) Supplier 1 uses digital signatures for tamper-proof product delivery of the compiler; however, supplier 2 doesn’t have a signature verification procedure for integrity checks on received products. This leaves the compiler vulnerable to malicious code. Moreover, supplier 3 employs data encryption to maintain confidentiality, but the end customer has a weak mechanism for authenticating internal users. This, in turn, leaves the system vulnerable to unauthorized individuals.

Identifying Threats In Figure 6, we sketch a software supply chain from a software company’s perspective. We identify six 34

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

processes, starting with the supplier’s sourcing process and ending with final product delivery. In the supplier sourcing process, a software company can act as both July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Outsourcing

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Open source product

COTS software product

Software package (installation package/source Finished software code/patches Sourced fragments and and manual) Sourced product documentation product Software Supplier Software development sourcing packaging and testing

Data media (for example, CD, DVD) with software and manual Software product manufacturing

Product delivery Customer

Shipping

Product delivery (through network)

Data media (for example, CD, DVD) with software and manual

Distributor

Warehousing

Installation package/source code/patches + manual

Customer Network transmission

Internet distributor

Data storage facility

Figure 6. A general scenario for a software supply chain. Identified processes include supplier sourcing, software development and testing, software packaging, software product manufacturing, and product delivery. Supplier sourcing and product delivery are two key processes that link different companies and lead to the formation of the software supply chain.

software provider and acquirer. Supplier sourcing and product delivery are two key processes that link different companies and can lead to the formation of a multilayered software supply chain consisting of generic processes such as software development and testing, software packaging, and software product manufacturing. Together, these comprise a company’s main functional chain.

Although supplier sourcing and product delivery are directly related to software supply chain activity, other processes that either consume the acquired product or produce the delivered product are also considered part of the supply chain. Therefore, for a comprehensive understanding of software supply chain security, all these processes must be considered and documented.

As shown in Figure 6, there are typically two delivery channels: physical product delivery and product delivery over the network. Some companies use both channels, whereas others rely on only one. For supplier sourcing, sourced products include COTS products, open source software products, and outsourced software components. These sourced products and their various uses bring about different threats. In this article, we focus only on the common threats related to these three kinds of sourced products involved in the supplier sourcing process.

Signaling diagrams provide a functional view of the software supply chain. As Figure 7 shows, software component products such as source code are transmitted between different processes, which are recognized as signals. Open source software components (and tools), COTS software products, and outsourced software components are transmitted from the supplier sourcing process to the software development and testing process. This includes secret information and security vulnerability information about the transmitted sourced products. After software development and testing, secret information and security vulnerability information about the software product, along with user guides, are transmitted to the software packaging process, where they’re packed together with source code or installation packages.

In a physical delivery channel, four elements—the software product’s secret information, security vulnerability information, source code or installation package, and user guide—pass through software product manufacturing to product delivery to the customer.

In network delivery, the four elements are transmitted directly from software packaging to product delivery.

Moreover, for physical delivery, data media is produced during software product manufacturing as a new element transmitted to product delivery to the customer.

35

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Supplier sourcing

Software development and testing

Software packaging

Software product manufacturing

Product delivery

Customer Threats

Open source tool COTS software product used as a tool Open source component Outsourced software component COTS software product used as software component

TT1

TT4, TT5

Secret information, for example, hard-coded key of sourced product Security weakness information of sourced product

Threats

ST1, ST2

TT2 Secret information, for example, hard-coded key

Secret information, for example, hard-coded key

Secret information, for example, hard-coded key

Security weakness information

Security weakness information

Security weakness information

TT8

ST9, ST10, ST11

TT7 Source code (including open source component, outsourced software component)/ installation package

Source code/installation package

Source code/ installation package

User guide

User guide

User guide

TT9, TT10, TT11

Data media, for example, CD

Patches

Patches

Data media, for example, CD Patches

ST17, ST18, ST19

ST28, ST29 TT24

Figure 7. A signaling model for software supply chains that shows common social (STs) and technical threats (TTs). Social and technical threats to the software supply chain can be identified when identifying threats to the elements transmitted between the different processes.

As a special software product, patches are normally transmitted from software development and testing to the customer through software packaging and product delivery and transmitted over the network.

In a signaling model, one way to identify threats to the software supply chain is to identify threats to the elements transmitted between different processes. We can use the signaling model with a socio-technical approach to identify threats.

Social Threats We used the SBC model in Figure 3 to itemize threats.

A comprehensive list of the identified social threats (labeled ST1, ST2, and so on, in Figure 7) can be found in our workshop paper;8 we list a few examples here.

Social threats can exist due to deliberate or unintentional human errors or behavior; for example, a supplier might deny having sent a software product, or an ordered software product might not arrive on time owing to nontechnical difficulties, such as quality control problems in software delivery (see the operational level in Figure 3).

Secret information about the outsourced software component, such as a hard-coded key or seed value, might be disclosed unintentionally by internal employees or intentionally, for example, owing to bribery. Unauthorized individuals can obtain access to 36

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

confidential information about an outsourced software component through nontechnical approaches such as spoofing. Such information might be related to hardcoded keys or information about an existing security vulnerability. Internal employees can then expose the confidential information.

Another example of social threats is when internal employees or unauthorized individuals, such as spoofers, destroy or insert malicious code into software source code or an installation package. Moreover, these individuals might manipulate or delete a software user guide.

Security mechanisms deployed in the software product (for example, key length) aren’t in compliance with end user requirements. These examples show how the social threats are at different levels in the SBC reference model, with most threats occurring at the operation level.

Technical Threats Technical threats mainly affect the software supply chain via the three bottom layers of the SBC model— hardware, operating system, and application. Again, a comprehensive list of the identified technical threats (labeled TT1, TT2, and so on in Figure 7) is available in our workshop paper;8 we outline a few here.

One example of a hardware threat is unauthorized individuals destroying outsourced software July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Table 1. Validation interview questions.

Are all key processes that impact software supply chain security represented in the model? If not, what is missing?

General questions regarding the software supply chain (see Figure 6)

Are any identified processes of less importance for modeling threats to the software supply chain?

Are all processes listed at a similar level of abstraction?

What do you think about this model? Coverage? Feasibility? Please choose one box from the coverage/feasibility matrix.

Are all key elements that pass through different processes represented in the model? If not, what is missing?

Are any identified elements of less importance for modeling threats to the software supply chain?

Questions regarding the signaling model for the software How useful is it to model a generic software supply chain system inside a software company using the combination of a process model and a signaling supply chain (see Figure 7) model? Is the system described by the models understandable?

What do you think about this model? Coverage? Feasibility? Please choose one box from the coverage/feasibility matrix.

Are key security requirements related to the software supply chain security (availability, data integrity, source integrity, confidentiality, and nonrepudiation) identified?

Questions regarding threat identification

Are any important threats missing? Please check the threats list.

What do you think about this threat identification approach? Coverage?

Feasibility? Please choose one box from the coverage/feasibility matrix.

Questions regarding the countermeasures identification approach

In terms of illustrated countermeasures, what do you think about using these approaches to identify countermeasures? Coverage? Feasibility? Please choose one box from the coverage/feasibility matrix.

Questions regarding the software supply chain from a socio-technical system perspective (see Figure 4)

Does this socio-technical model give a big picture of software supply chains and their security problems?

Overview questions

Will this model be useful for threat identification in software supply chains in practice?

What do you think about this model? Coverage? Feasibility? Please choose one box from the coverage/feasibility matrix.

Are the threat-modeling results presented in this survey complete enough for software supply chain threat modeling?

Is it easy to adapt the threat-modeling strategy (from system modeling to countermeasures identification) used in this project to meet individual company needs?

components by attacking a storage facility via the storage device or logical partitions. If a software supplier’s operating system is compromised, unauthorized individuals can insert a malicious code over the network into the open source tool, which leads to security defects in the software product.

Another example is hackers breaking into a software repository to obtain confidential information about the outsourced software component, such as a hard-coded key or seed value. If the application an acquirer used to provide remote downloading of software is compromised, unauthorized individuals can insert malicious code into the outsourced software component during delivery. Attackers can prevent network access to the source code or installation package using technical approaches, such as a denial-of-service attack.

The list of technical attacks is extensive, but owing to the SBC model’s layered structure, we can classify these

attacks into a manageable taxonomy for countermeasure analysis.

Countermeasures Similar to the threats categorization, countermeasures can also be divided into social and technical categories.

Social threats can be mitigated by both social and technical countermeasures, and technical threats can be mitigated by social measures. So, for each threat, we should analyze available countermeasures by adopting a more holistic way of thinking both socially and technically.

For example, to deal with social threats when recipients of a software product deny receiving it, a social countermeasure would be to legally require a thirdparty notary to prove that recipients actually received the software product (at the political and legal layer of the SBC stack). A technical countermeasure to deal with the same threat would be the implementation of 37

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Coverage

High

High Low

Feasibility

Figure 8. A coverage versus feasibility template used in an interview with industry experts. The categories in the quadrants might not provide granular feedback; however, at the current stage in our design research approach, we’re more interested in learning whether the experts support or criticize our model.

digital signatures using public-key cryptography (at the application layer of the SBC stack).

Another example of using socio-technical countermeasures is thwarting the threat of malicious code being injected into source code while transmitted over the network. A social countermeasure would be the implementation of a third-party escrow, whereas a technical countermeasure would be the implementation of virtual private networks.

Results Validation For our study, we selected four experts from two major telecom system providers in Sweden and conducted structured interviews to validate the threat modeling results as well as our modeling approaches (see Table 1). During the interviews, we showed the experts the process model in Figure 6, the signaling model in Figure 7, the applied threats identification approach, the list of identified social and technical threats, and the applied countermeasures identification approach, then questioned them on the relevance of our socio-technical framework on software supply chain security assurance.

We asked the experts about each model’s coverage and correctness and the feasibility of applying these approaches in practice. We also asked them to place the models in one of the quadrants of feasibility versus coverage, as Figure 8 shows. We understand that the categories in the quadrants might not provide granular feedback, but at the current stage in our design research approach, we’re more interested in learning whether the experts support or criticize our model.

For the process model, one interviewee suggested we separate software development and testing into two processes, owing to their complexity compared to other processes. However, our reason for representing these two processes as one in the model is that they’re 38

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

recursive and interact a great deal. Also, according to the signaling model, similar information elements (signals) are transmitted in and out from the two processes.

In general, we received positive feedback about the process and signaling models’ coverage and feasibility.

One expert gave higher praise for the process model because it provided a better overview of the system and related threats.

Regarding threat identification, most experts agreed that confidentiality, integrity, availability, and nonrepudiation are a software supply chain’s most important security requirements. They also emphasized other security requirements such as traceability, protection against reverse engineering, and documentation. One common opinion among all the experts was that humans are the biggest threat to software supply chains and the most difficult to manage. Some experts commented that the identified threats must cover other security requirements that might be practically required, such as chain of custody, separation of duties, and code testing and verification. However, all the experts recognized the high feasibility of the framework for threat identification.

Similarly, regarding the applied countermeasures identification approaches’ coverage and feasibility, the experts mentioned several examples in which the socio-technical framework could be applied in practice with their current services and delivery models. However, they also emphasized that for a complex system such as a telecommunications system, the model might need further adaptation to capture situations such as decoupled functionality in the final product.

All in all, the experts recognized the framework’s value. The model was complete enough from a generic perspective and could be adapted easily to individual company needs.

Discussion This framework appears to provide a holistic thinking aid to identify software supply chain security problems and describes relevant technical and social security aspects. However, there are some limitations: because the model addresses a holistic view of a socio-technical system and its interactions with other systems, some experts commented that the framework helped them “see the forest,” but they were unsure which tree “needed to be cut down.”

The software supply chain system modeled in Figure 5 focuses more on the security problems caused by poor coordination between suppliers and customers and unaligned security measures taken at different levels in different systems than common security concerns of software supply chain systems. Poor coordination of requirements between buyers and sellers is, of course, July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

not new; however, the framework is a comprehensive thinking tool for determining the security levels inside and outside a single organization.

To establish a secure software supply chain and ensure that the software product doesn’t introduce any security risks to the end customer, each entity in the software supply chain first needs to perform an analysis of its own socio-technical security posture and then use this to communicate with the next link in the chain, and so on, for the whole supply chain. The ultimate goal is to establish a software supply chain best practice threat analysis model in which all users can perform a comprehensive threat model and communicate it through every change of the evolving global software supply chain.

However, before such a global threat analysis and communication standard can be established, end customers must evaluate, on a case-by-case basis, and select a qualified supplier who can provide a secure software product via a tamper-proof delivery channel. Therefore, each organization must select trusted suppliers for its own sake. In this case, a threat-modeling report provided by suppliers can be useful for organizations establishing selection criteria.

T

he next step is to perform a case study to see if software suppliers and software acquirers can use the model to identify, communicate, and negotiate a common threat model and agree on appropriate countermeasures in a cost-efficient manner. It would be of special interest to perform both the socio-technical approach and a commonly used approach like Stride12 to see if there’s any difference in what’s identified and the work effort needed to perform the threat modeling and countermeasure strategies.

Acknowledgments We thank Xueqin Wang for her contribution in conducting the original case study in her master’s thesis and Ross W. Tsagalidis, senior advisor for cybersecurity with the Swedish Armed Forces.

4.

5.

6.

7.

8.

9.

10.

11.

12.

/it-pro/security-it/man-who-introduced-serious-heartbleed ________________________________ -security-flaw-denies-he-inserted-it-deliberately-2014 ________________________________ 0410-zqta1.html. _________ R.J. Ellison et al., Evaluating and Mitigating Software Supply Chain Security Risks, tech. report CMU/SEI-2010-TN-016, Software Eng. Inst., Carnegie Mellon Univ., 2010.

H. Xu et al., “Application of Security Ontology to ContextAware Alert System,” Proc. IEEE/ACIS 8th Int’l Conf.

Computer and Information Science, 2009, pp. 171–176.

E. dos Santos Moreira et al., “Ontologies for Information Security Management and Governance,” Information Management & Computer Security, vol. 16, no. 2, 2008, pp. 150–165.

E.G. Little and G.L. Rogova, “An Ontological Analysis of Threat and Vulnerability,” Proc. 9th Int’l Conf. Information Fusion, 2006; doi:10.1109/ICIF.2006.301716.

X. Wang, B. Al Sabbagh, and S. Kowalski, “A Socio-technical Framework for Threat Modeling a Software Supply Chain,” Proc. Dewald Roode Workshop on Information Systems Security Research, IFIP WG8.11/WG11.13, 2013, article 17.

S. Kowalski, “IT Insecurity: A Multi-discipline Inquiry,” PhD thesis, Dept. Computer and System Sciences, Univ.

Stockholm and Royal Inst. Technology, 1994.

“Design Science Research in Information Systems,” V. Vaishnavi and W. Kuechler, eds., 2004; www.desrist.org /design-research-in-information-systems. _______________________ C.-C. Hsu and B.A. Sandford, “The Delphi Technique: Making Sense of Consensus,” Practical Assessment Research & Evaluation, vol. 12, no. 10, 2007; http://pareonline .net/getvn.asp?v=12&n=10.

S. Davis, “Threat Modeling with STRIDE,” blog, 16 Apr. 2015; www.webtrends.com/blog/2015/04/threat_________________________ modeling-with-stride. ____________

Bilal Al Sabbagh is a PhD candidate in the Department

of Computer and Systems Sciences at Stockholm University and a senior information and network security consultant at NXme FZ-LLC. His research interests include socio-technical aspects of information systems security. Al Sabbagh received an MS in information and communication systems security from the Royal Institute of Technology, Sweden. Contact him at bilal@dsv.su.se. _________

References 1. H. Bidgoli, The Handbook of Technology Management: Supply Chain Management, Marketing and Advertising, and Global Management, John Wiley & Sons, 2010.

2. M. Ingram, “Here’s Everything You Need to Know about the Heartbleed Web Security Flaw,” blog, 8 Apr. 2014; https://gigaom.com/2014/04/08/heres-everything-you -need-to-know-about-the-heartbleed-web-security-flaw. _______________________________ 3. B. Grubb, “Man Who Introduced Serious ‘Heartbleed’ Security Flaw Denies He Inserted It Deliberately,” Sydney Morning Herald, 11 Apr. 2014; www.smh.com.au

Stewart Kowalski is a professor of information security

and a researcher with the Center for Cyber and Information Security at Gjøvik University College and the cybersecurity officer for Huawei Technology, Norway. His research interests include socio-technical security systems modeling and information security education. Kowalski received a PhD in computer and systems science from the Royal Institute of Technology, Sweden. Contact him at stewart ____ .kowalski@hig.no. ___________ 39

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MALWARE

Toward Anti-fragility: A Malware-Halting Technique Kjell Jørgen Hole | University of Bergen

Unlike robust systems, anti-fragile systems learn to adapt to changes and reduce the impact of future incidents. A combination of software diversity and imperfect malware detection can help achieve networked computing systems that are anti-fragile to malware with unknown and time-varying spreading mechanisms.

T

raditionally, the literature has categorized complex systems as being fragile or robust to classes of incidents. Fragile systems are vulnerable to the impact of these incidents, whereas robust systems can withstand the impact. Essayist and scholar Nassim Nicholas Taleb introduced a third category of complex systems: anti-fragile.1 Unlike robust systems, anti-fragile systems learn from incidents how to function in a changing environment. In fact, anti-fragile systems need incidents to remain welladapted to their environments. Without the ability to learn from incidents, anti-fragile systems become fragile over time. The popular media-streaming company Netflix regularly creates incidents in its production system by inducing local failures.2 It implements design patterns to isolate the impact of these failures, preventing catastrophic failure propagation and allowing the system to learn from and mitigate system weaknesses.

System fragility, robustness, and anti-fragility are best thought of as degrees on a spectrum with fragile systems on the left, robust systems in the middle, and anti-fragile systems on the right. Systems must be robust before they can become anti-fragile, and systems can’t be anti-fragile to all possible incidents.1 By combining automated software diversity3–6 and automated imperfect malware detection and removal, we can achieve a degree of anti-fragility to stop the spreading of various types of malicious software or

40

July/August 2015

SECURITY& PRIVACY

IEEE

malware in networked computing systems. In this article, I propose a malware-halting technique that scales to huge networks because it doesn’t require any tightly coupled interactions or adaptations among groups of devices. This technique is timely and important because malware is an omnipresent and serious security threat.7,8

Malware Spreading Security experts study infectious malware such as computer worms with different spreading mechanisms.

Email malware that spreads via address lists and mobile phone malware that propagates over short-range wireless links generate patterns of infected devices defined by sparse graphs,7 whereas malware that scans Internet Protocol version 4 (IPv4) addresses at random produces dense graphs. I concentrate on sparse-spreading networks in this article. Rather than trying to accurately model the spreading of real malware instances, I analyze worst-case spreading, in which the first attempt to infect a susceptible device always succeeds.

Malware mistakenly downloaded by users, such as Trojan horses, spyware, adware, and ransomware, is considered infectious malware with limited spreading ability. The spreading of malware in a networked computing system is measured by the fraction of infected devices: a system is considered fragile when small outbreaks spread to a large fraction of the devices, and it’s considered robust when the malware spreads to only a

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

small fraction of the devices. A system under repeated attack from malware is considered anti-fragile if it first learns to reduce the fraction of infected devices and then manages to keep the fraction small when the malware’s spreading mechanism changes.

Agent-Based Model To achieve anti-fragility in practice, it’s advantageous to build on existing and planned automated software mechanisms. My proposed model combines compilergenerated diversity, software downloads from application stores, and imperfect malware detection and removal to achieve anti-fragility. An agent-based model randomly adds diversity to a monoculture, creating a software polyculture that greatly reduces the fraction of infected devices. This model demonstrates that periodically removing executable code—including unknown malware—from devices and installing new diverse code drastically increases robustness to malware spreading. If imperfect malware detection is added, the model gains a degree of anti-fragility because it can more quickly remove malware and update vulnerable code to keep the fraction of infected devices very small, even when the malware has unknown and time-varying spreading.

Imagine a system’s software and hardware as a collection of interconnected computing platforms at the OS and application levels.6 Application stores in the cloud, such as Google Play and the iOS App Store, can use compilers with diversity engines to generate binary images for a huge number of platforms, producing many different executable images from a much smaller set of OS and application source codes.4,5 Assume that a program’s many binary images can be divided into classes so that all members of the same class have a common exploitable vulnerability, whereas members of different classes have no common exploitable vulnerability.

Assuming the compilers generate equally large classes, the number of classes indicates the program’s diversity.

Because compiler-driven diversity promises to provide large diversity,4 my model forgoes any notion of central control over the assignment of the software diversity to computing platforms and makes no attempt to minimize the use of diversity. This allows us to study the benefit of software diversity in systems with millions of devices.

Cloud-based compilations allow application stores to support a large number of download requests per day.

A significant fraction of malware infections aren’t discovered by traditional signature-based malware detection because modern malware uses time-varying code obfuscation to avoid detection based on fixed byte patterns.8 Emerging cloud-based antimalware solutions promise to improve automated malware detection.9–11 Servers in the cloud deploy heuristic, behavioral, and signature-based techniques to detect different types of

malware by processing data collected by clients running on user devices. A cloud solution could also incorporate malware knowledge from other sources such as honeypots—computers capturing malware.

Despite the protection promised by cloud antimalware, it’s nearly impossible to keep all computing devices in a networked system free from malware at all times. The difficulty of detecting encrypted malicious traffic and the successful use of rootkits to hide malware suggest that automated malware detection will remain imperfect for the foreseeable future.8 A more realistic goal is to provide a form of “community immunity,” where most devices are protected against malware because there’s little opportunity for outbreaks to spread. Whereas community immunity usually entails “immunization” of nearly all entities in a population, the proposed model deploys compiler-generated software diversity to reduce malware spreading.6 The goal isn’t to force the fraction of infected devices to zero but to keep it very low over time.

I focus on consecutive outbreaks of different malware types, called multimalware outbreaks, because the deployment of multiple malware types is an obvious strategy to counter software diversity. Devices are assumed to automatically remove executable code, including unknown malware, and immediately download new diverse code from application stores on a semiregular basis.

The introduction of imperfect malware detection also allows devices to initiate unscheduled code removal and updates when infections are detected. In extreme cases, trained personnel have to take a device offline to wipe its memory before installing the new software. The following model assumes that the self-repairing and diversityenhancing approach removes all malware. It’s hard to remove advanced malware—especially rootkits—from real systems, but it’s possible to adjust how often the model carries out code removals and updates.

Model Description Multimalware spreading over networked computing devices is modeled using a simple graph—without selfloops or parallel edges—with N nodes and L node types.

At time step t = 0, 1, …, the graph contains D = D(t) of the L node types, where D(0) = 1. The D “active” node types represent binary code classes on the OS or application level of the devices’ computing platforms. In other words, nodes of the same type share an exploitable vulnerability, whereas nodes of different types have no common exploitable vulnerability. The edges represent virtual communication lines. The number of active node types D measures the model’s time-varying diversity.

Two nodes are neighbors if there’s an edge between them. A node’s degree k is its number of neighbors, and ¢k² is the average degree over all nodes. All nodes change type with probability p at each time step to model automated 41

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MALWARE

removal of executable code—including unknown malware—followed by immediate downloads of new diverse code from application stores. One of the L possible node types is selected with probability 1/L, changing the initial monoculture into a polyculture with diversity L. Initially, all nodes are susceptible to malware infections. There’s one malware type per node type. All malware types have the same spreading mechanism. Whereas epidemiological models in the literature tend to model a single malware outbreak, I model systems with many outbreaks. One malware outbreak occurs with probability q at each time step. An outbreak initially infects a single susceptible node selected uniformly at random. A newly infected node infects all its susceptible neighbors of the same type during the next time step. Infected nodes change type with probability r at each time step to model varying degrees of imperfect malware detection, followed by immediate malware removal and installation of new diverse code. Any infected node becomes susceptible to further infection when it changes type.

It’s possible to switch off automated malware detection by setting r = 0; setting p = 0 disables automated software downloads. A small fraction of nodes can be immunized, or made resistant to malware infections. Immunized nodes don’t change type or transmit infections to neighbors. In practice, automated immunization or hardening includes the removal of nonessential software programs, the secure configuration of remaining programs, constant patching, and the use of firewalls and intrusion prevention systems.

Other mitigation techniques, such as control flow integrity,12 that induce code overhead and performance penalties can also be used on selected devices.

Limitations It’s difficult to predict how malware will spread over a networked computing system because propagation depends on the malware’s spreading mechanism, network topology, traffic load, routing and filtering policies, communication protocols, and network failures and misconfigurations. Rather than trying to generate accurate spreading patterns under various network conditions, the model displays very fast worst-case malware spreading, in which an infectious node immediately infects all its neighbors of the same type.

Although the model can’t predict malware spreading in a real networked system, it can demonstrate the usefulness of combining software diversity and imperfect malware detection to halt the spreading. Compared to the model, actual malware is likely to spread more slowly because the first attempt to infect a susceptible computing device won’t always succeed, and not all susceptible devices will be infected because some are unreachable in practice. Hence, it’s reasonable to believe that modeled malware halting translates into halting in real systems.

42

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

I study only sparse-spreading patterns, where the average degree ¢k² is much lower than the number of nodes N. Whereas nodes and edges can be deleted during a model run and new nodes and edges can be added to simulate changes in the malware’s spreading mechanism, nodes can’t change position during a run.

Anti-fragility on Fixed Graphs A worst-case spreading pattern given by a network with average degree ¢k² is homogeneous when all nodes have degrees k ≈ ¢k². To keep the analysis manageable, I first study fixed spreading patterns represented by homogeneous networks and then determine when the system model is anti-fragile to spreading.

Consider a homogeneous network with D(0) = 1 node type at time t = 0. If this monoculture is connected, it’s extremely fragile to malware spreading because a single infected node will infect all nodes as long as nodes don’t change type. The model avoids fragile monocultures by allowing nodes to change type. Each time a node changes type, it selects a particular type with probability 1/L. Consequently, the number of node types D(t) will grow toward the maximum value L. I assume that D(t) = L for t > tc and study the model after the fraction of infected nodes starts to fluctuate around a small time-averaged value f.

During each time step, there’s a probability q that a single susceptible node is seeded with an infection. The probability that no neighboring node has the same type as this “seed” is approximately 1−

k

1 L

.

If we choose a large diversity L > ¢k² such that this probability is large, an infection will almost surely spread only from the seed to the nearest neighbors of the same type.

Ignoring further spreading, each seed infects on average k <1 L of its neighbors. Over a period of T time steps, the expected number of seeds is T ∙ q, and approximately T ⋅q⋅

k L

susceptible neighbors will be infected because the average fraction f of infected nodes is small. The total number of susceptible nodes infected during period T is thus estimated as Q = T ⋅q · 1 +

k .

L

All N nodes in a network change type with probability p to model diverse software downloads. The expected July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Table 1. Proximity networks.* 5

6

7

8

Estimated fraction

0.25 percent

0.26 percent

0.27 percent

0.28 percent

Simulated fraction (max., min.)

0.26 percent

0.27 percent

0.28 percent

0.30 percent

(+0.09, –0.08)

(+0.08, –0.1)

(+0.11, –0.07)

(+0.16, –0.11)

Average degree

* The table shows the estimated average fraction of infected nodes in proximity networks with 500 nodes and increasing average node degree. The corresponding simulated fraction is averaged over 100 runs with the largest observed deviations.

number of type changes over T time steps is p ∙ N ∙ T, and the average number of infected nodes that change type is P = f ∙ p ∙ N ∙ T. Any infected node becomes susceptible when it changes type, even if the infection wasn’t detected because the malware is assumed to be removed. Furthermore, remaining infected nodes at a time step detect their infections with probability r. The expected number of detected infections followed by immediate software downloads is R = r ∙ f ∙ N ∙ T over T time steps.

The number of infected nodes changing type and becoming susceptible must be equal to the number of new infected nodes to maintain a stable time-averaged fraction f of infected nodes. Because the number of nodes N is large and the probabilities p and r are assumed to be small, there’s little overlap between the set of nodes updating their states due to software downloads and the set of nodes changing states due to malware detections.

Hence, for homogeneous networks with diversity L > ¢k², the relation P + R ≈ Q results in the approximation k ] L f≈ . ( p + r )⋅ N q[1 +

(1)

Equation 1 shows that it’s possible to maintain a small fraction f of infected nodes even if malware detection is switched off (r = 0) by adjusting the software download probability p. Note that this property is based on the assumption that malware is removed during the software update process. When malware detection is switched on (r > 0), the fraction of infected nodes reduces further for a fixed probability p. Hence, anti-fragile systems using imperfect detection and removal of new malware further reduce the fraction of infected devices compared to robust systems that merely remove old code and download new diverse code periodically.

Simulations of Anti-fragility on Fixed Networks To validate Equation 1, I consider smartphones and other handheld computing devices that communicate via short-range Wi-Fi and Bluetooth links.7 Infectious malware types can copy themselves to new devices by opening wireless connections. Malware can also

propagate directly between Wi-Fi access points via wireless connections.13 In this model, worst-case spreading patterns are represented by homogeneous proximity networks. _ The system model was programmed in NetLogo (ccl .northwestern.edu/netlogo) _________________ and generates a proximity network with average node degree ¢k² by first placing N nodes uniformly at random on a square. An edge is then added between a randomly chosen node and its closest neighbor in Euclidean distance. More edges are added in the same way until the network reaches the desired average degree. Self-loops and multiple edges between nodes aren’t allowed.

Simulations ran on networks with N = 5,000 nodes, L = 20 node types, and different average degrees ¢k². The outbreak probability was q = 10–2 , the software download probability was p = 10–5, and the malware detection probability was r = 10–3. Table 1 lists the observed average, minimum, and maximum fraction of infected nodes over 100 runs, where each value was averaged over the last 10,000 time steps of a run. The table also contains an estimate of the average fraction obtained from Equation 1. The agreement between the simulated and calculated values shows that the expression can provide good estimates of the average fraction of infected nodes. Other model runs with different parameter values confirm the agreement between simulated and calculated values.

Anti-fragility on Large Fixed Networks Because there is agreement between the average fractions of infected nodes obtained from the simulations and Equation 1, the equation applies to anti-fragility to malware spreading on very large homogeneous networks. The required frequency of software download p and frequency of malware detection r decrease as the size of a network grows because p + r is proportional to 1/N. Hence, anti-fragility to malware occurs on large model networks for “practical” download and detection frequencies.

Consider a homogeneous network with 100 million nodes, such that N = 108, and average degree k L. For outbreak frequency q = 10–2 and an average fraction of infected nodes f = 10–3, Equation 1 shows that 43

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MALWARE

1 Fraction of infected nodes

Spike due to monoculture

Change in spreading mechanism

Immunization of hubs

Malware detection

0 0

40,000 Time steps

Figure 1. Fraction of infected nodes in a changing network with 2,000 nodes, outbreak probability q = 10–2, download probability p = 10–3, and diversity L = 5. The plot illustrates the effects of increasing diversity, malware detection, change in spreading mechanism, and hub immunization.

p+r ≈

q = 10−7 . f ⋅N

If each time step in the model is one second, then there’s a new malware outbreak every 100 seconds on average.

The fraction of infected nodes is maintained when the download frequency is p ≈ 10–7 without malware detection (r = 0); in other words, each device must download and install new software after approximately 116 days.

If malware detection is added to this example, then the average fraction of infected devices is reduced. For r = 10–4 and p = 10–7, f ≈ 10–6. These calculations illustrate that anti-fragility to malware spreading scales to very large homogeneous networks.

Anti-fragility on Time-Varying Graphs I now consider the system model with an unknown and time-varying worst-case spreading pattern that stays sparse over time. Even if the spreading mechanism varies, the spreading is mostly limited to the neighbors of the nodes seeded with infections as long as the spreading pattern remains homogeneous and the diversity remains much larger than the changing average degree.

I therefore study inhomogeneous spreading patterns containing a small fraction of nodes, called hubs, with degrees kh much larger than the time-varying average degree. A hub and its kh neighbors form a star graph with the hub at the center. If all kh + 1 nodes have a uniform distribution of L node types, there are roughly kh/L neighbors with the same type as the hub. A susceptible hub is infected each time one of the neighbors of the same type is infected. A reinfected hub infects all susceptible neighbors of the same type, ensuring a total of kh/L infected neighbors.

44

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

Because a hub’s kh neighbors constitute a significant fraction of all nodes in a worst-case spreading pattern, this case might very well have kh ≫ L. This is why hubs are referred to as super spreaders even in software polycultures.

Note that malware doesn’t have to be infectious for a hub to be a super spreader, for example, a popular website infected by malware that’s inadvertently downloaded by many users accessing the site. Although the noninfectious malware doesn’t spread any further after being downloaded, the number of infected devices kh/L is still large.

Similarly, if the hubs’ neighbors tend to have small degrees, most of the spreading of infectious malware will also be confined to the hubs’ neighbors. Even when a hub changes type regularly, there will still be roughly kh/L neighbors of the same type as long as all nodes have a uniform distribution of types. Over time, a hub that changes type will reinfect many neighbors as long as at least a few neighbors of different types are infected.

If hubs are connected in a small subnetwork and several hubs have the same type, then a large fraction of all nodes in a worst-case spreading network is infected very quickly. Hence, hubs, especially tightly connected hubs, need to be “neutralized” to make the actual spreading pattern more homogeneous so that for any susceptible node with degree k, the expected number of neighbors with the same type is negligible k 1.

L The three following simulations of malware with different spreading patterns show that hubs can be immunized to gain anti-fragility to multimalware spreading. Rather than presenting plots averaged over many runs to obtain smooth curves, the figures plot single runs to better demonstrate how anti-fragile systems would actually behave.

Simulations of Anti-fragility In the first simulation, the NetLogo model generates a time-varying spreading pattern. A model run starts with a proximity network with 2,000 nodes, average degree ¢k² = 4, maximum degree 10, and diversity D(0) = 1.

Initially, malware detection is turned off (r = 0). Figure 1 shows that the fraction of infected nodes decreases from approximately 91 percent to 8 percent as the model changes from a fragile monoculture to a more robust polyculture with diversity L = 5. When malware detection is turned on (r = 10–2), the fraction decreases further to 0.1 percent. The plot confirms the advantage of detecting newly infected devices, immediately removing the malware, and installing new diverse software in addition to the periodically scheduled removal of old software and downloads of new diverse software.

Next, 75 percent of the nodes and their adjacent edges are deleted, and the network is regrown using July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

1 Fraction of infected nodes

the preferential attachment technique14 with each new node connected to three existing nodes. The new nodes have a uniform distribution of node types. This simulated change in spreading mechanism causes the original homogeneous spreading pattern to become an inhomogeneous spreading pattern. The new spreading pattern has average degree ¢k² = 4.8 and maximum degree 33. The fraction of infected nodes increases to roughly 45 percent because the diversity L isn’t large enough to prevent spreading from the new hubs even though malware detection is still turned on. The sharp increase in the fraction of infected nodes illustrates that the malware halting is fragile to changes in the spreading pattern when the diversity is too small.

Finally, the 61 nodes with the largest degrees are immunized. When these hubs and their adjacent edges are ignored because they no longer contribute to malware spreading, the remaining spreading pattern has average degree ¢k² = 3.8 and maximum degree 14. The fraction of infected nodes decreases to roughly 0.6 percent, demonstrating the need to immunize super spreaders in real networks to obtain more homogeneous spreading patterns.

In the second simulation, the NetLogo model starts with an inhomogeneous email network with 1,133 nodes, average degree ¢k² = 9.6, maximum degree 71, and diversity L = 8. The largest hubs are immunized before the model run starts. In Figure 2, the fraction of infected nodes decreases to roughly 1 percent as the monoculture turns into a polyculture. The model then erases 75 percent of all nodes and creates an inhomogeneous network with 2,000 nodes. No new nodes are immunized, and their types are distributed uniformly.

The new subgraph of susceptible nodes has average degree ¢k² = 4.7 and maximum degree 44. Unlike in the first simulation, there’s no large change in the fraction of infected nodes in Figure 2 because the diversity, the remaining immunized nodes, and the malware detection probability (r = 10–2) prevent significant spreading. The plot shows that the malware halting can be made robust to changes in the spreading pattern.

In the third simulation, the NetLogo model utilizes a fixed inhomogeneous spreading pattern with 10,670 nodes, 36 hubs that form a small connected subgraph, and many nodes with low degree k ≈ ¢k² = 4.1. The hubs’ degrees range from 2,312 to 102. In a real network, hub immunization is likely to be imperfect because some infected hubs aren’t detected or because some hardened hubs still become infected. The model utilizes “acquaintance immunization” to simulate imperfect cloud-based detection and immunization of infected hubs.15 This technique chooses a set of nodes uniformly at random and immunizes one arbitrary neighbor per node. Although the original set of nodes is unlikely

Monoculture

Time-varying polyculture 0 0

25,600 Time steps

Figure 2. Fraction of infected nodes in a time-varying email network with outbreak probability q = 10–2, download probability p = 10–3, malware detection probability r = 10–2, and diversity L = 8. The fraction of infected nodes decreases to roughly 1 percent as the monoculture turns into a polyculture. Note that there’s no visible change in the fraction when the spreading network changes.

1

Monoculture

Imperfect hub immunization Polyculture 0 0

7,440 Time steps

Figure 3. Fraction of infected nodes in a fixed network with 10,670 nodes, outbreak probability q = 10–1, download probability p = 10–3, malware detection probability r = 10–3, and diversity L = 14. Imperfect detection and hub immunization reduce the fraction of infected nodes, even when the spreading pattern contains a subnetwork of tightly connected hubs.

to contain the few hubs in the network, the randomly selected neighbors are much more likely to be hubs, as very many edges are adjacent to high-degree nodes.

Figure 3 plots the fraction of infected nodes. The fraction decreases as the diversity grows to L = 14 but stabilizes around 11 percent because the hubs aren’t immunized. When acquaintance immunization selects 2 percent of the nodes, all but three of the hubs are immunized. The fraction of infected nodes decreases to approximately 0.1 percent. Figure 3 illustrates that imperfect detection and hub immunization reduce the fraction of infected nodes, even when the spreading pattern contains a subnetwork of tightly connected hubs.

Additional simulations with varying parameter values confirm the reported model behavior. In particular, simulations using acquaintance immunization 45

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Fraction of infected nodes

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MALWARE

confirm the adequacy of imperfect hub immunization.

The additional simulations further strengthen the claim that combining compiler-generated software diversity, periodic download of software from application stores, and imperfect malware detection and removal provides a networked computing system with a degree of antifragility to multimalware spreading.

Toward Practical Anti-fragility on Large Networks Anti-fragility to malware spreading scales to large networked systems because compiler-generated software diversity and malware detection can be realized as cloud services. Although empirical work is needed to determine the real-world performance of the combined services, it’s encouraging to observe commercial antimalware solutions running in the cloud. According to Michael Franz’s research group, it’s cost effective to compile diverse software in the cloud. 4,5 Furthermore, the software diversity’s impact on the runtime performance is small, and it’s possible to patch diverse software securely. Still, challenges remain.

Although acquaintance immunization is useful for simulating imperfect detection and hub immunization, the strategy isn’t the best choice for real networked systems because many popular websites are potential super spreaders. The challenge is to ensure that owners of potential super spreaders harden their systems. It’s reasonable to assume that cloud-based malware detection discovers popular infected websites because they generate a great deal of malicious activity. Users could be warned to stay away from the infected websites, requiring owners to remove the malware and harden the systems to get their users back. At the time of writing, Google ___ informs users and webmasters of unsafe websites (www .google.com/transparencyreport/safebrowsing).

Netflix’s decision to induce failures in its production system to detect and remove weaknesses and thus increase the robustness to downtime raises the question of whether to use infectious “goodware” to improve the detection of susceptible devices and speed up the learning process leading to anti-fragility to malware spreading. Although ethical questions are associated with this approach, it’s worth investigating.

B

ecause malware halting is fragile to changes in the spreading mechanism when diversity is limited, we need to better understand how much diversity is needed in huge networked systems. There’s already a general lower bound on diversity to ensure that fixed graphs don’t contain a giant component of same-type nodes.6 Although users today regularly download software from application stores, more work is also needed to

46

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

create self-repairing devices that remove malware and install diverse software in a way that’s acceptable to users.

References 1. N.N. Taleb, Antifragile: Things That Gain from Disorder, Random House, 2012.

2. A. Tseitlin, “The Antifragile Organization,” Comm. ACM, vol. 56, no. 8, 2013, pp. 40–44.

3. A. Avizienis et al., “Basic Concepts and Taxonomy of Dependable and Secure Computing,” IEEE Trans. Dependable and Secure Computing, vol. 1, no. 1, 2004, pp. 11–33.

4. M. Franz, “E Unibus Pluram: Massive-Scale Software Diversity as a Defense Mechanism,” Proc. New Security Paradigms Workshop (NSPW 10), 2010, pp. 7–16.

5. P. Larsen, S. Brunthaler, and M. Franz, “Security through Diversity: Are We There Yet?,” IEEE Security & Privacy, vol. 12, no. 2, 2013, pp. 28–35.

6. K.J. Hole, “Diversity Reduces the Impact of Malware,” IEEE Security & Privacy, vol. 13, no. 3, 2015, pp. 48–54.

7. P. Wang et al., “Understanding the Spreading Patterns of Mobile Phone Viruses,” Science, vol. 324, no. 5930, 2009, pp. 1071–1076.

8. C.C. Elisan, Malware, Rootkits & Botnets, McGraw-Hill Osborne Media, 2012.

9. Q. Li and G. Clark, “Mobile Security: A Look Ahead,” IEEE Security & Privacy, vol. 11, no. 1, 2013, pp. 78–81.

10. K. Salah et al., “Using Cloud Computing to Implement a Security Overlay Network,” IEEE Security & Privacy, vol.

11, no. 1, 2013, pp. 44–53.

11. E.G. Amoroso, “From the Enterprise Perimeter to a Mobility-Enabled Secure Cloud,” IEEE Security & Privacy, vol. 11, no. 1, 2013, pp. 23–31.

12. M. Abadi et al., “Control-Flow Integrity Principles, Implementations, and Applications,” ACM Trans. Information and System Security, vol. 13, no. 1, 2009, article 4.

13. J. Milliken, V. Selis, and A. Marshall, “Detection and Analysis of the Chameleon WiFi Access Point Virus,” EURASIP J. Information Security, vol. 2013, no. 2, 2013; doi:10.1186/1687-417X-2013-2.

14. A.-L. Barabási, R. Albert, and H. Jeong, “Scale-Free Characteristics of Random Networks: The Topology of the World-Wide Web,” Physica A, vol. 281, nos. 1–4, 2000, pp. 69–77.

15. R. Cohen, S. Havlin, and D. Ben-Avraham, “Efficient Immunization Strategies for Computer Networks and Populations,” Physical Rev. Letters, vol. 91, no. 24, 2003, article 247901.

Kjell Jørgen Hole is a professor in the Department of

Informatics at the University of Bergen. His research interests include risk management of complex adaptive systems. Hole received a PhD in computer science from the University of Bergen. Contact him at kjell.hole@ii.uib.no. ____________ July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

ACCESS CONTROL

Distance-Bounding Protocols: Are You Close Enough?

Christos Dimitrakakis and Aikaterini Mitrokotsa | Chalmers University of Technology

Distance-bounding protocols can offer protection against attacks on access control systems that require users to both verify their credentials and prove their location. However, tradeoff s among accuracy, cost, and privacy are necessary.

M

any access control problems implicitly require access-seeking parties to not only verify their credentials but also prove their location. For instance, in an automobile’s electronic lock and ignition system, a driver carries a token that activates the lock and the ignition only if the token’s validity and proximity can be verified. Verifying both is necessary; otherwise, the system becomes vulnerable to man-in-the-middle attacks.

Although proximity can be guaranteed through signal fading, it can be easily circumvented through active relays. For example, an adversary could simply use the Internet to relay messages between a token in a user’s house to the vehicle. Many systems are susceptible to this type of attack, as shown by an extensive study of 10 car models from eight manufacturers in which the attack succeeded in all cases.1 In addition to car systems, this type of attack can target bank cards, mobile phones that use near-field technology, and proximity cards such as those used to access buildings. Subtler attacks on wireless ad hoc networks, which rely on location information for routing, are also possible. In this article, we address how and to what extent we can verify the distance of an authenticating party.

A token’s validity can be verified with standard cryptographic authentication protocols, but the problem of establishing proximity remains. The simplest approach is to rely on signal attenuation: as distance increases, the received signal strength decreases.

1540-7993/15/$31.00 © 2015 IEEE

SECURITY& PRIVACY

IEEE

In this article, we briefly explain the principles behind distance-bounding (DB) authentication protocols. Distance bounding creates inherent tradeoffs among verification accuracy, protocol cost, and location privacy. This is of greatest importance in constrained settings, because many wireless devices are limited in power or bandwidth, and distance verification imposes additional limits on computation.

Distance Bounding DB protocols attempt to simultaneously verify the credentials and the proximity of a prover (the party to be authenticated). These protocols are used in settings in which adversaries might want to fool a verifier (the authentication system) into accepting a distant prover.

DB protocols are real-time challenge–response protocols. This means that each time i that the protocol runs, verifier V issues a challenge ci to prover P. P then issues a response ri, which is the result of applying a function fx to the challenges, where x is a secret key. How can attackers defeat this protocol without knowing the secret?

A simple example is a child playing simultaneously against two chess grandmasters.2 By simply relaying the moves of the first grandmaster P to the second grandmaster V, and vice versa, the child can win one of the games. The fraud is detected by prespecifying when each grandmaster should respond, and then checking the timing of the fraudster’s moves.2 Consequently,

Copublished by the IEEE Computer and Reliability Societies

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

47 M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

ACCESS CONTROL

Q

V

(a)

P

A

V

A

V

(b)

P′ (c)

Figure 1. Types of relay attacks: (a) distance fraud, (b) mafia fraud, and (c) terrorist fraud. A masked yellow figure indicates an adversary that knows the secret x. A masked brown figure indicates an adversary that doesn’t know the secret.

although malicious parties could simply relay the messages from V to P to be authenticated, we can apply the same principle, namely timing the fraudsters’ responses.

In this case, we merely want to bound their distance from the verifier; we only wish to ensure that their responses are received within a certain time limit.

However, this isn’t the only type of relay attack.

There are three categories, depending on the nature of the attacker (see Figure 1): ■ Distance fraud. A legitimate prover P knows the secret x and wants to prove to V that it’s close when it’s actually far away.

■ Mafia fraud. A legitimate prover P is far away from legitimate verifier V as well as an adversary A. A relays the challenges of V to P and relays P’s responses back to V. Thus, both the verifier and prover think they’re talking to each other. The attack succeeds if the responses are correct and V doesn’t realize that P is far away.

■ Terrorist fraud. This attack differs slightly from mafia fraud. Prover Pc is a willing accomplice and collaborates with A. A great example compares A to a terrorist who wants to cross the border.2 Prover Pc helps terrorist A answer the questions of an immigration officer V. The formal restriction in this setting is that Pc never reveals the secret x to A.

Distance-Bounding Protocols Current DB protocols rely on the fact that transmission times are fundamentally bounded by the speed of light 48

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

and on the difficulty of forging an authenticated message. Because the speed of light is known, measuring response delay lets us bound the distance of a verifier.

The responses ri can be made hard to forge by making secrets difficult to guess.

DB protocols were first applied in the wireless communication setting to prevent relay attacks against ATM systems. In particular, Stefan Brands and David Chaum wanted to verify the distance between a legitimate ATM and a user’s smart card.3 Many DB protocols have been proposed since then, but they generally comprise three phases: initialization, distance bounding, and verification. In the first phase, the communicating parties establish a session key. During the second phase, challenges and responses are exchanged. In the final phase, the responses’ timing and validity are checked. Figure 2 outlines these phases.

Initialization Phase In most DB protocols, the prover and verifier share a common secret x. However, if the responses ri depend only on the challenges ci and the key x, then every time the protocol is repeated, attackers can learn the correct response for every challenge, thereby enabling a replay attack. For this reason, we generate a secret session key α for each session that’s used to calculate the correct responses in the distance-bounding phase. To create a session key α, the prover and verifier both generate random values, called nonces, which are fed to a pseudorandom function (PRF). The session key is the output of the PRF.

Distance-Bounding Phase The actual distance bounding occurs in this phase, which lasts n rounds. The number of rounds n is a security parameter. In the ith round, V issues challenge ct and observes and stores response rt as well as the time difference Δi between when the challenge was issued and when the response was received. The computations during this phase are extremely simple; to minimize the computational delay, the challenges and responses are single bits.

Verification Phase V checks the correctness of the received responses ri and calculates an upper bound on the distance of the prover P based on the response delay Δi of each challenge ci . Because information can’t travel faster than light, the distance between P and V is upper bounded

by CΔmax/2, where C is the speed of light and Δmax is the maximum allowed delay between sending out bit ci and receiving bit ri .

Finally, verifier V indicates whether prover P is authenticated (OutV = 1) or not (OutV = 0). If the July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Verifier V

Prover P

Shared key x

NV <— {0, 1}m

Shared key x Initialization phase Nv

NP <— {0, 1}m

Np

Generate a session key α using x, Np, Nv

Generate a session key α using x, Np, Nv Distance-bounding phase for i = 1 to n

Pick ci ∈U {0, 1} Start clock

ci ri := fx{ci, ai}

ri Stop clock and store Δi

Verification phase

Verify the responses and that for all rounds Δi ≤ 2Δmax

Outv (...)

Figure 2. Overview of distance-bounding protocol phases: initialization, distance bounding, and verification. (The arrows denote random, uniform sampling.) In the initialization phase, the parameters of the protocol are agreed on. In the distancebounding phase, quick messages are exchanged to guarantee that the distance is sufficiently small. In the verification phase, the messages are verified with a signature scheme.

number of incorrect responses is low and all messages are received within the allowed delay, the protocol succeeds.

ωAP ĉ′i

Noise and Cost Constraints Due to timing constraints, the protocol’s distancebounding phase is performed without any error correction. As such, some of the responses might be incorrect due to noise, even though P knows the shared secret.

(An analysis of the effect of noise can be found in “Location Leakage in Distance Bounding: Why Location Privacy Does Not Work.”4) Thus, a tolerance threshold t for erroneous responses is necessary. Higher thresholds decrease the probability that honest provers will be rejected but increase the probability that adversaries will be accepted. In either case, we can improve accuracy by increasing the number of rounds n.

Unfortunately, each round carries a cost—especially for the prover—which is usually resource constrained owing to low power or bandwidth. We can’t increase the number of rounds arbitrarily to achieve better accuracy.

How should we then optimally design our protocol?

In “Expected Loss Bounds for Authentication in Constrained Channels,” authors Dimitrakakis and Mitrokotsa, along with Serge Vaudenay, formalized the problem in a decision-theoretic framework.5 An optimal solution can be found by specifying a cost not only for the transmissions but also for the two different

P

ωVA c′i

ĉi

r′i

ri(ĉ′i ) ωPA

ci

ri A

r′i

V

ωAV

Figure 3. A mafia fraud attack under noisy conditions.

Attackers communicate with a legitimate prover and a verifier through a noisy channel. Attackers first try to guess the challenges they will receive by communicating with the prover. For any correct guesses, they give the correct response; otherwise, they reply randomly. However, the channel noise makes this attack harder to carry out.

types of authentication errors: accepting adversaries and rejecting legitimate provers. Although the problem is presented generally for authentication in noisy conditions, DB protocols are the most representative example of such authentication protocols. For this category, Dimitrakakis and his colleagues also provided a performance analysis of optimally tuned DB protocols for mafia fraud attacks.5 To illustrate the process, consider Figure 3, which depicts a mafia fraud attack. The communication is performed under noisy conditions; we use ωA,B to denote 49

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

ACCESS CONTROL

P P

dAP

A A

dAV

V

V

(a)

dAP – dAV = c

(b)

Figure 4. Information leakage regarding the locations of the prover and verifier: (a) passive eavesdropping and (b) hyperbole of the attacker’s location. Potential locations for the attacker relative to the prover and verifier are constrained within the hyperbole.

the noise in the communication channel between parties A and B. We use ci to denote the challenge sent during the ith round of the rapid bit exchange and ri to denote the correct response. An attacker might not wait for challenge ci but instead choose a challenge ĉi and send it to legitimate prover P. The prover will finally receive a challenge ĉci, which might differ from the one sent by the adversary due to the noise ωA,B . The same applies for the transmitted responses ri, ˆr i, and ˆr ci .

Location Privacy Constraints Rasmussen and Srđjan Čapkun noted that DB protocols might leak more information than just the fact that the prover is within the maximum allowed distance from the verifier.6 They noticed that when a passive adversary A observes the communication between prover P and verifier V running a DB protocol, A can determine the distance between P and V by observing the arrival times of their exchanged messages. Passive adversaries might also infer their own location relative to those of P and V. Potential locations for attackers relative to the prover and verifier are constrained on a hyperbole, as Figure 4 depicts.

Although Rasmussen and Čapkun (RC) were the first to address the problem of location privacy in DB protocols, their protocol is vulnerable to mafia fraud attacks as well as collision attacks due to nonce repetition.7,8 Nevertheless, privacy can be achieved with the location-private DB protocol proposed in “Mafia Fraud Attack against the RC Distance-Bounding Protocol,” which improves on the RC protocol.7 Mitrokotsa and her colleagues investigated the extent to which location privacy is achievable for both omniscient and limited adversaries.4 Omniscient adversaries 50

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

can measure the transmitted messages’ signal strength and observe all messages’ sending and arrival times.

Limited adversaries are aware only of the time at which they receive messages from other participants.

Location privacy is information-theoretically impossible for either type of adversary.4 In particular, adversaries limited to passive observation might break the location privacy of any polynomial-time DB protocol given unbounded computation time. However, Mitrokotsa and her colleagues showed that carefully chosen parameters enable computational, provable location privacy in practice. They proved that location privacy against limited adversaries minimally requires that the prover and verifier running a DB protocol introduce exponential delays between receiving and sending messages and provide lower bounds for these delays.4

Nonce Space Size As we mentioned, DB protocols use nonces to avoid replay attacks. However, constrained devices such as those used in RFID systems might not be able to use very large nonce spaces. Consequently, collisions might be relatively frequent, enabling replay attacks. In fact, in many DB protocols, attackers can recover the shared secret between a prover and a verifier after some protocol repetitions. More precisely, Mitrokotsa and her colleagues showed that the number of repetitions required is only logarithmic in key length and mainly depends on the length of the nonce.9 Consequently, nonce space size is extremely important for these protocols. Nevertheless, it’s possible to calculate experimental and theoretical bounds that practitioners can use to select appropriate security parameters, including nonce and secret key length, depending on the application scenario. In any case, there exists a countermeasure that can reduce the probability of a successful attack in this setting.

Provable Security of DB Protocols An important question regarding DB protocols is whether their security can be proved formally. Unfortunately, only a few existing protocols are provably secure in a formal sense. Ioana Boureanu and her colleagues were the first to propose a family of provably secure DB protocols called SKI.9,10 They’ve shown that SKI and its variants are provably secure against distance fraud and generalized versions of mafia and terrorist fraud, even under noisy conditions. SKI is designed to be resistant to cases in which the shared secret x is reused outside the computation of the session key. Marc Fischlin and Cristina Onete proposed another provably secure DB protocol in “Subtle Kinks in Distance-Bounding: An Analysis of Prominent Protocols.”11 July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

Vaudenay performed a thorough comparison of the two protocols:12 The main advantages of the FischlinOnete DB protocol are its employment of binary challenges, its resilience to noise at a level of 1/4, and its reliance on standard PRF security.11 Its main drawbacks include its low resistance to mafia fraud, B(n, τ, 3/4), and nonuniform security for distance fraud. The SKI protocol’s main advantages include its uniform security for distance fraud and its better resistance, B(n, τ, 2/3), against man-in-the-middle attacks—that is, a generalization of mafia fraud attacks. SKI’s main drawbacks include its use of nonbinary challenges, its resilience to noise at a level of 16, and its reliance on nonstandard PRF security.

D

istance-bounding protocols are a necessary countermeasure to the emerging threat of relay attacks, especially with the advent of RFID and nearfield technology in credit cards and mobile phones.

They’re also important for applications such as wireless ad hoc networks, where network nodes must be reasonably sure of their neighboring nodes’ proximity to perform network routing.

One of the greatest challenges in DB protocol implementation is that computation in the distancebounding phase must be extremely fast. The RC protocol had a radio-frequency circuit with nanosecond latency;13 more recently, G.P. Hancke proposed a system that achieves low-latency distance bounding through ultra-wideband pulses.14 Because DB protocols must interact in the physical layer, dedicated hardware is necessary for practical implementation.

Consequently, widespread deployment of such protocols must await manufacturer endorsement.

We believe that the time for wide adoption of DB protocols has arrived. The field is now mature, and provably secure protocols exist. Anything would be an improvement on the current state of affairs, which offers no protection against any of these attack modes.

References 1. A. Francillon, B. Danev, and S. Čapkun, “Relay Attacks on Passive Keyless Entry and Start Systems in Modern Cars,” Proc. 18th Ann. Network and Distributed System Security Symp. (NDSS 11), 2011; http://s3.eurecom.fr/docs /ndss11_francillon.pdf. _____________ 2. T. Beth and Y. Desmedt, “Identification Tokens—or: Solving the Chess Grandmaster Problem,” Advances in Cryptography (CRYPTO 90), 1990, pp. 169–177.

3. S. Brands and D. Chaum, “Distance-Bounding Protocols (Extended Abstract),” Advances in Cryptology— EUROCRYPT 93, LNCS 765, Springer, 1994, pp. 344–359.

4. A. Mitrokotsa, C. Onete, and S. Vaudenay, “Location Leakage in Distance Bounding: Why Location Privacy

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

Does Not Work,” Computers and Security, vol. 45, 2014, pp. 199–209.

C. Dimitrakakis, A. Mitrokotsa, and S. Vaudenay, “Expected Loss Bounds for Authentication in Constrained Channels,” Proc. IEEE INFOCOM, 2012, pp. 478–485.

K. Rasmussen and S. Čapkun, “Location Privacy of Distance Bounding,” Proc. 15th ACM Conf. Computer and Comm. Security (CCS 08), 2008, pp. 149–160.

A. Mitrokotsa, C. Onete, and S. Vaudenay, “Mafia Fraud Attack against the RC Distance-Bounding Protocol,” Proc.

2012 IEEE Int’l Conf. RFID-Technologies and Applications (RFID-TA 12), 2012, pp. 74–79.

J.-P. Aumasson, A. Mitrokotsa, and P. Peris-Lopez, “A Note on a Privacy-Preserving Distance-Bounding Protocol,” Proc. 13th Int’l Conf. Information and Comm. Security (ICICS 11), 2011, pp. 78–92.

A. Mitrokotsa et al., “On Selecting the Nonce Length in Distance-Bounding Protocols,” Computer J., 4 Apr. 2013; doi: 10.1093/comjnl/bxt033.

I. Boureanu, A. Mitrokotsa, and S. Vaudenay, “Secure and Lightweight Distance-Bounding,” Lightweight Cryptography for Security and Privacy, LNCS 8162, Springer, 2013, pp. 97–113.

M. Fischlin and C. Onete, “Subtle Kinks in DistanceBounding: An Analysis of Prominent Protocols,” Proc. 6th ACM Conf. Security and Privacy in Wireless and Mobile Networks (WiSec 13), 2013, pp. 195–206.

S. Vaudenay, “On Modeling Terrorist Frauds,” Provable Security, LNCS 8209, Springer, 2013, pp. 1–20.

K.B. Rasmussen and S. Čapkun, “Realization of RF Distance Bounding,” Proc. 19th USENIX Conf. Security (USENIX Security 10), 2010, p. 25.

G.P. Hancke, “Design of a Secure Distance-Bounding Channel for RFID,” J. Network and Computer Applications, vol. 34, no. 3, 2011, pp. 877–887.

Christos Dimitrakakis is a lecturer at Chalmers Uni-

versity of Technology. His research interests include decision theory, reinforcement learning, statistics, and security and privacy. Dimitrakakis received a PhD in machine learning from the École Polytechnique Fédérale de Lausanne, Switzerland. Contact him at chrdimi@chalmers.se. _____________ Aikaterini Mitrokotsa is an assistant professor in the

Department of Computer Science and Engineering at Chalmers University of Technology. Her research interests include information and network security, privacy preservation, machine learning for security, and applied cryptography. Mitrokotsa received a PhD in network security from the University of Piraeus, Greece. She’s an associate editor of IEEE Communications Letters and the Computers and Security Journal.

Contact her at aikmitr@chalmers.se. _____________ 51

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

DATA LEAKS

Can Content-Based Data Loss Prevention Solutions Prevent Data Leakage in Web Traffic?

David Gugelmann | ETH Zurich Pascal Studerus | AdNovum Informatik AG Vincent Lenders | armasuisse Bernhard Ager | ETH Zurich

The effectiveness of data loss prevention (DLP) solutions remains obscure, especially with respect to the types of attacks they claim to protect against. A systematic analysis of HTTP data leakage vectors and a test of three content-based DLP solutions give insight into these solutions’ vulnerabilities.

D

ata leakage is a serious threat to many organizations. A recent study of 350 companies estimated that the average total cost of a data breach incident is US$3.8 million.1 To address this threat, companies have deployed perimeter protection mechanisms such as firewalls to control and block undesired dataflows.

However, attackers have quickly adapted their data leakage techniques to evade these protection schemes.

For example, today’s malware often relies on HTTP as a command-and-control channel to hide its activities in the regular flow of Web traffic.2 Because blocking HTTP traffic at the perimeter would prevent users from accessing the Web, this communication channel is often left open. Therefore, perimeter security alone is no longer sufficient to address modern data leakage threats, and defense-in-depth concepts are required. This means that organizations expect weaknesses in their protection measures and employ layered defenses that prevent attacks as well as incorporate attack detection and mitigation measures.3 Data loss prevention (DLP) solutions—systems that analyze dataflows and enforce data policies to safeguard sensitive information—promise to help. Due to the variety of technical approaches that can address data leakage threats, it’s not surprising that many DLP

52

July/August 2015

SECURITY& PRIVACY

IEEE

solutions are currently on the market.4,5 More than a dozen security companies promote their solutions as a silver bullet against leakage. However, selecting a proper solution isn’t a trivial task as there is little transparency in this area: vendors are generally reluctant to reveal how their systems work in detail and provide only highlevel descriptions in their product documentation. In addition, threat models aren’t defined explicitly, so the DLP solutions’ effectiveness against particular threats remains unjustified.

In an attempt to fill this gap, we provide a systematic evaluation of content-based DLPs to examine whether they effectively prevent data leakage in Web traffic.

Although the DLP solutions we tested received very high marks in other evaluations, our tests demonstrate that these solutions protect against accidental data loss but can be bypassed by attackers. We specifically focus on HTTP, as it’s the dominant Internet protocol and many organizations allow Web browsing.

Data Leaks over HTTP HTTP is a stateless request–response protocol that browsers use to access the Web. It can be tunneled over an encrypted SSL/TLS channel (HTTPS). DLP solutions employ mechanisms to access the clear text

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

Data leakage via distorted HTTP request or other protocol

Data leakage via valid HTTP request Request line (URL)

Body

Plain

M q M q

M q

MqM q

Obfuscated

Plain

Obfuscated

Header field

Plain

Covert channel (obfuscated)

Obfuscated

Threat models

Plain

Obfuscated

T1, T2, T3

T2, T3

Figure 1. HTTP attack vectors. An attacker can follow the protocol semantics (left tree) or send arbitrary traffic over the HTTP port (right tree).

In addition, malicious insiders and malware (threat models T2 and T3) might obfuscate data before inserting it into HTTP requests. T1, T2, and T3 refer to different threat models with different privileges and intentions in terms of data exfiltration.

of HTTPS traffic, which allows them to handle HTTPS traffic similarly to HTTP traffic. Thus, we treat HTTP and HTTPS as functionally equivalent and refer to both as HTTP. HTTP can be used in many different ways to exfiltrate data. Figure 1 illustrates a systematic categorization of the possible data leakage vectors. At the top level, attackers might choose to follow the HTTP protocol syntax and semantics or send arbitrary traffic that violates the HTTP specification. Violating this specification might be a tempting strategy for leaking data as DLPs might not capture all possible types of distortion. However, it’s generally easier to detect and block malicious traffic that violates the protocol semantics by simple protocol checking. In this article, we explain and analyze the challenges facing HTTP version 1.1. The upcoming HTTP version 2 faces similar issues, as it’s semantically compatible.

HTTP/1.1 requests consist of the request line stating which resource is being requested from the server, a message header with additional information, and an optional body. Attackers following the HTTP protocol basically have four options to transmit data in HTTP requests: request body, request line, header fields, and covert channels.

Leakage in Request Body This is the most straightforward case, as transmitting text input and files in the request body of POST requests is the recommended way to upload data to servers. According to the HTTP specification, uploaded data is either URL encoded or submitted as multipart/form data. A DLP solution can easily revert these encodings and access the original payload. Therefore, we refer to this subcategory as plain (as shown in Figure 1). Figure 2a shows an example submission of text input, and Figure 2b shows a file upload to a webpage.

However, attackers might obfuscate the data they’re trying to exfiltrate. They could achieve this by using other types of unforeseen encodings in the body or by

(a)

(b)

(c)

Figure 2. Example HTTP requests uploading secret=one, two, three to the server. Note that one%2C+two%2C+three is the URL-encoded representation of one, two, three. (a) POST request with URL-encoded data in the body. (b) POST request with multipart/form data file upload in the body. (c) GET request with URL-encoded data in the URL.

using encryption. We combine encryption and unusual encodings in the obfuscated category in Figure 1.

Leakage in Request Line In HTTP/1.1, the requested URL is split into two parts.

The Internet host part of the URL (example.com) is specified in the host header field, and the path of the resource on the server, including additional parameters, 53

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

DATA LEAKS

is contained in the request line. Figure 2c shows an example GET request that uploads text input in the URL.

Leakage in Header Fields In this slightly subtle case, attackers exploit a field of the request header to upload data. For example, data might be hidden in the Accept-Language field. Instead of defining the languages, attackers insert sensitive data.

Attackers might include data as plain text or obfuscate leaked data through different encodings or encryption.

Leakage via Covert Channels Covert channels are the most challenging strategy to defend against because leaked data isn’t directly transmitted in the URL, header, or body. Instead, leaked data is hidden by a secret coding scheme that relates, for example, to the presence or order of header fields in transmitted requests, the timing of requests, or the selection of particular links when browsing.6–9 Unless an organization is aware of such side channels, this type of leakage is extremely hard to prevent. In fact, we aren’t aware of any DLP solution on the market that claims to be able to deal with this threat.

Data Leakage Threats The ability to abuse different parts of the HTTP protocol for data exfiltration depends greatly on an attacker’s access privileges and intentions. We take this into account by introducing three threat models (T1, T2, and T3) with different privileges and intentions in terms of data exfiltration. For each threat model, we show which parts of the HTTP protocol request messages could be exploited to leak data.

T1: Data Loss through Regular Web Browsing In this threat model, sensitive data is intentionally or unintentionally leaked through regular Web browsing.

We assume that data leaked this way isn’t obfuscated before being transmitted over HTTP. As highlighted in Figure 1, the leakage vectors for T1 are limited to plain body and plain URL, as these are the two mechanisms expected to transmit user data to a server. For example, employees might accidentally leak sensitive information by uploading a file to their company’s Facebook account. Such unintentional data leaks are very common: negligence and system glitches account for about half of all data breaches.1

T2: Malicious Data Exfiltration Using the Web Browser In this threat model, we assume that attackers try to leak sensitive data using the browser. Attackers might exploit all browser features for exfiltration (such as JavaScript) but don’t have the privileges to access the Internet with 54

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

any other software. This is a typical scenario in securityaware organizations with managed workstations.

Therefore, T2 represents the threat model of frustrated employees who try to leak confidential information.

Under T2, we have to assume that attackers can exploit all HTTP attack vectors. By creating a website with built-in JavaScript, it’s trivial to obfuscate the data to be transferred by encoding it in the browser. With JavaScript, it’s also possible to alter HTTP header fields and create modified or even invalid HTTP. This type of threat model might require programming skills to stage an attack, but the cost can be considered low because software exploits aren’t necessary.

T3: Unconstrained Data Exfiltration In this threat model, we assume that attackers or malware have gained root privileges on a workstation. Thus, attackers have unconstrained access to the workstation and could generate arbitrary traffic with a program of their choice. Data transfers might be obfuscated by means of encryption, side channels, or invalid HTTP. This is the highest threat level of the three. Because attackers or malware with root privileges could start and stop any processes on the workstation, we must consider the possibility that attackers might even disable a DLP agent running on the workstation.

DLP Solution Testing There are two main types of DLP solutions: host based and network based. Host-based solutions must be installed on every workstation and include a combination of control mechanisms to disable or monitor network I/O, removable media, copy and paste, screen capturing, and printing as well as control which applications are executed or get network access.10 Network-based DLP solutions analyze network streams on a gateway. We use the term security control point to refer to the locations of systems’ control mechanisms.

When controlling dataflows at a security control point, the DLP solution might either block all dataflows to a destination (such as disabling copying to USB flash drives) or selectively identify sensitive information and block corresponding data transfers. Content-based DLPs identify sensitive information by searching the content of transferred information for patterns and keywords, often relying on predefined patterns for typical sensitive information—such as Social Security numbers, email addresses, or credit card numbers—and configurable keywords, such as “internal” or “classified.” DLPs include parsers for a wide range of document formats— including Microsoft Excel, Word, and PowerPoint documents or PDF documents—and file containers (such as .zip, .gzip, and .7z) to extend their search to the contents of these documents and containers.

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

Testing Methodology As is the case for many proprietary security tools, DLP vendors rarely provide documentation of their tools’ internal implementations, much less the source code.

This makes it difficult to determine whether a solution can protect against a particular attack. One possible approach is to reverse-engineer the executable. However, reverse engineering takes an exceptional amount of effort that would be annihilated by every software update released by the vendor. Thus, we developed a dedicated penetration-testing tool—called Data Exfiltration Tester (DXT)—that implements several data exfiltration methods over HTTP. Our black-box testing has an advantage over reverse engineering in that any system administrator can quickly detect flaws in a DLP solution’s implementation or configuration by simply testing whether it prevents a particular kind of data leakage.

To analyze the implemented security controls, we took advantage of the fact that DLP solutions log or explicitly inform users when an action has been blocked. By correlating DLP alerts with actions conducted by DXT, we could gain insight into the kind of security controls used by a DLP solution. For example, if a DLP solution triggers an alert while our tool is reading a file from the file system, we can deduce that the file system I/O is being monitored. If the alert occurs only when a file is already loaded or when copied and pasted data is sent over the network, we can deduce that the network I/O is being monitored.

By combining multiple tests, we can deduce with high confidence which security controls are in place without reverse engineering. Thus, black-box testing provides an excellent tradeoff between analysis efficiency and confidence in the results.

To evaluate DLP solutions’ capabilities, we installed each solution on a workstation and used DXT to exfiltrate test data from the workstation to the Internet over HTTP. DXT contains a component written in JavaScript (jsDXT) that obfuscates and uploads data directly in the Web browser as well as a Python-based stand-alone client (pyDXT) that obfuscates and uploads files independently from the Web browser—similar to how malware works.

Both components implement methods to transmit plain and obfuscated data according to the attack vectors.

We used the same rules (R) to test all DLP solutions: ■ R1—block documents containing any email addresses using a predefined pattern provided by the DLP, ■ R2—block documents containing the keyword VH2uJKLi, and ■ R3—block documents containing the keyword c/M!=&zu.

R1 tests one of the predefined patterns, whereas R2 and R3 test the ability to find keywords in data. We used

two tests to evaluate whether URL encoding of text fields—the most prevalent parameter encoding (see Figure 2)—was properly handled. The keyword in R2 can be recognized even if URL-encoded values aren’t decoded. To match R3, URL-encoded values must be properly decoded due to the special characters contained in the keyword. For each rule, we created matching documents of different types (.txt, .docx, .xlsx, and .pdf) as well as compressed versions of the documents (.gzip, .zip, and .7z). To evaluate a DLP solution according to threat models T1 and T2, we opened jsDXT in the Web browser and uploaded these documents to a Web server. In addition, we copied and pasted text to a form field and uploaded the text using jsDXT. For threat model T3, we submitted the documents using pyDXT.

Evaluation of DLP Solutions We applied our testing methodology to three major vendors’ host-based DLP solutions (referred to here as DLP A, B, and C). The vendors and solutions remain anonymous, as our goal isn’t to pinpoint individual products but to showcase different DLP mechanisms and discuss their effectiveness. SC Magazine and Network World evaluated some of the solutions and gave them high ratings. However, those evaluations reflect case studies that lack an exhaustive attack vector classification and a clearly defined threat model. The vendors are also listed in market reports by Gartner and the Radicati Group.4,5 Tables 1 and 2 show details of the applied attack vectors with exfiltration of copied and pasted text via a Web form (Table 1) and with exfiltration of different file types via a Web form/script (Table 2). Each row represents an HTTP attack vector with the vertical labels (T1 to T3) showing the corresponding threat models. The columns show the tested DLPs and uploaded file types.

A green circle means data loss was prevented (all three rules were effective). A white circle means data loss was partially prevented (R1 and R2 were effective but R3 failed). A red square means there was complete data loss (all rules failed).

In Table 1, attack vectors 1 through 4 (threat model T1) submitted the plain data. In attack vectors 5 through 9 (threat model T2), the HTTP requests were slightly obfuscated by placing data at unexpected positions in the HTTP request (attack vectors 5 and 8) or by base64 encoding the data prior to submission. In Table 2, attack vectors 10 through 12 (threat model T3) submitted the data using pyDXT, first plain, then base64 encoded, and finally plain but as distorted HTTP without any request line or header. Most benign websites transfer text input as shown in attack vector 1 and file uploads as shown in attack vector 2. Thus, these two attack vectors are most relevant with respect to threat model T1.

55

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

DATA LEAKS

Table 1. Exfiltration of copied and pasted text via Web form for three data loss prevention (DLP) solutions.

Threat model

DLP A

DLP B

DLP C

T1

Attack vector no. Attack vector type 1

Body plain: URL encoded in POST body

●

■

●

T1

2

Body plain: multipart/form data in POST body

●

■

●

T1

3

Body plain: raw data in POST body

●

■

●

T1

4

URL plain: URL encoded in request line of GET request

●

■

●

T2

5

URL plain: URL encoded in request line of POST request

●

■

●

T2

6

Body obfuscated: base64 and URL encoded in POST body

●

■

●

T2

7

Body obfuscated: base64 encoded in multipart/ form data POST body

●

■

●

T2

8

Header field plain: in additional header field of GET request

●

■

●

T2

9

Header field obfuscated: base64 encoded in additional header field of GET request

●

■

●

● : data loss prevented (all rules effective) ■ : complete data loss (all rules failed)

Results Our results show that DLP A was able to successfully block all data leakage where adversaries tried to copy and paste data to a text field in the Web browser. HTTP requests with files in multipart body containers were blocked when submitted through the Web browser or through pyDXT. However, 7z-compressed files didn’t seem to be supported. Obfuscated file uploads weren’t blocked at all. This suggests that DLP A controls the clipboard and network I/O on the workstation. However, as experiment 3 showed, the network analyzer didn’t support files that were transmitted without encapsulation in multipart containers.

DLP B did not prevent any data leakage through text form fields. File leakage using attack vectors 2, 3, and 7 was prevented as the Web browser was denied access to files matching the rules. However, pyDXT can read and upload all files. These results strongly suggest that DLP B controls only the Web browser’s file system I/O. The implementation seems to be flawed, which leads to failure in enforcing R3 with .docx and .xlsx files. Controlling files when they’re read by an application instead of controlling the HTTP requests sent by the application is advantageous in that obfuscation in the application has no effect on the DLP solution. Thus, DLP B can prevent plain and obfuscated file uploads through the Web browser. However, pyDXT isn’t monitored by DLP B, and accidental copying and pasting of sensitive information can’t be prevented because the clipboard and the network traffic don’t seem to be monitored.

DLP C blocked all data leakage attempts in which 56

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

adversaries tried to copy and paste data to a text field in the Web browser. Data leakage through the most commonly used upload mechanisms (attack vectors 1 and 2) was also blocked. Raw file uploads in POST requests (attack vector 3) were partially blocked. The unobfuscated data submitted by pyDXT was successfully blocked. Obfuscated file uploads weren’t blocked at all. These results suggest that DLP C monitors the clipboard and network I/O on the workstation.

To summarize, we found that all three DLP solutions can prevent accidental file uploads over HTTP if attack vector 2 is used. However, none of the solutions can prevent data leakage under threat models T2 and T3.

For example, to bypass DLP A and DLP C, T2 adversaries can simply obfuscate a document in JavaScript; to bypass DLP B, T2 adversaries can copy and paste the contents to a text input field in the Web browser.

Furthermore, we found flaws in the implementation of security control points that resulted in problems enforcing a rule, even when a control point was in place.

Security Control Points Here we look at the flawed combinations of security control points implemented by the evaluated DLPs and present security control combinations that would more efficiently prevent leakage.

Implemented Security Control Points By correlating DLP alerts with actions conducted by our black-box testing tool, we found three security control points (shown in Figure 3): July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Table 2. Exfiltration of different file types via Web form/script.

Attack vector type

.txt

.docx

.xlsx

.pdf

.txt zip

.txt gz

.txt 7z

.txt

.docx

.xlsx

.pdf

.txt zip

.txt gz

.txt 7z

.txt

.docx

.xlsx

.pdf

.txt zip

.txt gz

.txt 7z

DLP C

Attack vector no.

DLP B

Threat model

DLP A

T1

2

Body plain: multipart/ form data in POST body

●

●

●

●

●

●

■

●

●

●

●

●

●

●

●

●

●

●

●

●

●

T1

3

Body plain: raw data in POST body

■

■

■

■

■

■

■

●

●

●

●

●

●

●

●

●

●

■

●

■

●

T2

7

Body obfuscated: base64 encoded in multipart/form data POST body

■

■

■

■

■

■

■

●

●

●

●

●

●

●

■

■

■

■

■

■

■

T3

10

Unknown application body plain: multipart/ form data in POST body

●

●

●

●

●

●

■

■

■

■

■

■

■

■

●

●

●

●

●

●

●

T3

11

Unknown application body obfuscated: base64 encoded in multipart/form data in POST body

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

T3

12

Unknown application distorted HTTP: raw data without a request line or header

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

■

● : data loss prevented (all rules effective) ● : data loss partially prevented (rules 1 and 2 effective) ■ : complete data loss (all rules failed)

■ File system I/O on the workstation (FILE_I/O)— control of files that are read from disk or network storage.

■ Copy and paste on the workstation (CLIPBOARD)— control of data that is copied between applications via the clipboard.

■ Network I/O on the workstation (NET_HOST)— control of HTTP and HTTPS network traffic. By installing a hook in the network API, a DLP solution can inspect and block requests prior to leaving the workstation.

For comparison, we also considered a security control point that corresponds to a network-based product: ■ Network I/O on a network gateway (NET_GATEWAY)—control of HTTP and HTTPS network traffic. Network-based DLP solutions employ an HTTP proxy server to monitor data streams. The proxy server

intercepts SSL/TLS-encrypted data streams to provide clear-text access to HTTPS requests. Detected data leaks can be avoided by terminating the corresponding network stream.

Table 3 summarizes which of these security control points appear to be implemented by the evaluated DLP solutions. The security control point labels correspond to Figure 3.

Mandatory Security Control Points Here we relate the three threat models to a minimum set of security control points required to prevent data leakage over HTTP. In other words, assuming that a DLP policy for email addresses has been defined, we show for each threat model which security control points must be used to prevent leaking a document containing email addresses. Table 4 shows mandatory security controls to protect against a particular threat model as 57

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

DATA LEAKS

User input User

Network I/O

Network I/O

NET_HOST File system I/O FILE_I/O

Storage

NET_GATEWAY Internal organization network

Copy and paste CLIPBOARD

Web browser

Clipboard Workstation Organization perimeter

Figure 3. Security control points for enforcement of DLP rules for Web browsing. Data can be provided to the Web browser as user input via the keyboard, the file system, or the clipboard. Security control points FILE_I/O and CLIPBOARD monitor the file system I/O and the clipboard, respectively. NET_HOST controls network traffic by monitoring the workstation’s network API, and NET_GATEWAY monitors the organization’s network traffic.

Table 3. Security control points.

Solution

Implemented security controls (derived from black-box testing and product documentation)

DLP A

CLIPBOARD and NET_HOST

DLP B

FILE_I/O (only for supported applications)

DLP C

CLIPBOARD and NET_HOST

Table 4. Mandatory security controls.

Threat model

Prevention

Detection and forensics

T1: regular Web uploads

FILE_I/O and CLIPBOARD or NET_HOST or NET_GATEWAY

NET_HOST or NET_GATEWAY

T2: malicious exfiltration using Web browser

FILE_I/O and CLIPBOARD

NET_HOST or NET_GATEWAY

T3: malware

There is no sufficient combination of the identified security control points

NET_GATEWAY

well as security control points for network-based leakage detection and forensics.

For threat model T1, the Web browser either doesn’t transform the data before uploading to a Web server (multipart/form data) or it transforms the data as specified by well-defined standards (URL encoded) so that network-based monitoring solutions can reverse the data transformations. As a result, employing a network-based solution, such as NET_HOST or NET_ GATEWAY, is sufficient. Alternatively, a DLP can rely on a combination of FILE_I/O and CLIPBOARD to control file uploads and copied and pasted data.

For threat model T2, data can be obfuscated in the 58

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

Web browser before being sent to a Web server. Because obfuscations can be arbitrary, it generally isn’t feasible to know which obfuscations have been applied or to reconstruct the original data from network traffic. For example, in attack vector 7 in Table 2, we applied base64 encoding to files as a very simple means of obfuscation.

Yet, this was enough to allow our HTTP requests to pass through the network I/O security control points of DLPs A and C. Attackers could use much more sophisticated obfuscation or even encrypt (http://code .google.com/p/crypto-js) and obfuscate data in JavaScript. Thus, internal attackers could use a website with JavaScript to evade purely network-based DLP solutions July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

(such as NET_HOST or NET_GATEWAY). For effective data leakage prevention, a DLP solution must employ security control points that can inspect data before obfuscation, such as FILE_I/O and CLIPBOARD, which control data before it’s processed by the Web browser. Note, however, that even this combination of security control points can’t prevent data leakage if another application has already obfuscated the data before it’s copied onto the clipboard or read from the file system.

Malicious software in threat model T3 can run with root privileges on a compromised workstation. The malware might tamper with or disable any DLP solution running on a workstation, rendering the security control points FILE_I/O, CLIPBOARD, and NET_ HOST ineffective. However, malware on the workstation can’t disable a security control point in the network.

Therefore, NET_GATEWAY can prevent unobfuscated data leakage, but as soon as the malware obfuscates data transfers, NET_GATEWAY is also rendered ineffective.

As a result, once malware with root privileges is running on a workstation, it’s generally not feasible to enforce a DLP policy. Even though NET_GATEWAY can’t prevent obfuscated data leakage, it can still provide useful connection information for detection and forensics.

W

e found that DLPs A, B, and C all implement security control points that can prevent accidental file uploads over HTTP. However, none of the solutions employ the correct combination of the mandatory security control points to prevent basic data leakage caused by internal attackers or malware. Thus, these solutions can’t protect against T2 or T3. By following our recommendations regarding the correct combination of mandatory security controls, these DLP solutions would be significantly improved.

Our study suggests that this deficiency is related to the lack of fundamental security controls at the required vantage points. Only one of the evaluated DLP solutions transparently reports this severe constraint. One vendor even explicitly states that its product addresses data leakage caused by internal attackers or malware, which we found to be false. We invite organizations to test their DLP solutions to make sure they meet the promises found in vendors’ product descriptions.

Acknowledgments This work was partially supported by the Zurich Information Security Center at ETH Zurich and represents only the views of the authors.

References 1. “2015 Cost of Data Breach Study: Global Analysis,” Ponemon Inst., May 2015; www-03.ibm.com/security/data-breach.

2. “Top Banking Botnets of 2013,” Dell SecureWorks, 3 Mar. 2014; www.secureworks.com/cyber-threat-intelligence /threats/top-banking-botnets-of-2013/. ______________________ 3. “Defense in Depth: A Practical Strategy for Achieving Information Assurance in Today’s Highly Networked Environments,” Nat’l Security Agency; www.nsa.gov/ia /_files/support/defenseindepth.pdf. ____________________ 4. “Magic Quadrant for Endpoint Protection Platforms,” Gartner, 2 Jan. 2013; www.gartner.com/doc/2292216 /magic-quadrant-endpoint-protection-platforms. ___________________________ 5. “The Radicati Group Releases ‘Content-Aware Data Loss Prevention Market, 2013–2017,’” Radicati Group, 11 Mar. 2013; www.radicati.com/?p=9377.

6. N. Feamster et al., “Infranet: Circumventing Web Censorship and Surveillance,” Proc. 11th USENIX Security Symp., 2002, pp. 247–262.

7. S. Cabuk, C.E. Brodley, and C. Shields, “IP Covert Timing Channels: Design and Detection,” Proc. 11th ACM Conf.

Computer and Comm. Security (CCS 04), 2004, pp. 178–187.

8. N. Schear et al., “Glavlit: Preventing Exfiltration at Wire Speed,” Proc. 5th Workshop Hot Topics in Networks (HotNets 06), 2006, pp. 133–138.

9. K. Borders and A. Prakash, “Quantifying Information Leaks in Outbound Web Traffic,” Proc. 30th IEEE Symp.

Security and Privacy, 2009, pp. 129–140.

10. A. Shabtai, Y. Elovici, and L. Rokach, A Survey of Data Leakage Detection and Prevention Solutions, Springer, 2012.

David Gugelmann is a doctoral student at ETH Zurich. His

research interests include network forensics and privacy measurements. Gugelmann received an MS in electrical engineering and information technology from ETH Zurich. Contact him at gugelmann@tik.ee.ethz.ch. ________________ Pascal Studerus is a software engineer at AdNovum

Informatik AG. Studerus received an MS in computer science focusing on information security from ETH Zurich. Contact him at pascalstuderus@gmail.com. _________________ Vincent Lenders is a research program manager at arma-

suisse, where he is responsible for the Cyberspace and Information research program. Lenders received a PhD from ETH Zurich and was a postdoctoral researcher at Princeton University. He serves as the industrial director of the Zurich Information Secu____ rity Center at ETH Zurich. Contact him at vincent .lenders@armasuisse.ch. _______________ Bernhard Ager is a senior researcher and lecturer at

ETH Zurich. His research is driven by his desire to understand large and complex systems such as the Internet. Ager received a PhD from TU Berlin and was a researcher at Deutsche Telekom Laboratories (T-Labs). Contact him at bager@tik.ee.ethz.ch. _____________ 59

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

OS FINGERPRINTING

Packet Inspection for Unauthorized OS Detection in Enterprises Rohit Tyagi and Tuhin Paul | Indian Space Research Organisation B.S. Manoj | Indian Institute of Space Science and Technology B. Thanudas | Vikram Sarabhai Space Centre

Many recent malware implementations employ virtual machines to carry out malicious activities. These are hard to detect because antivirus software running in the native OS can’t detect virtual machines’ system states. An approach that uses TCP SYN packets for OS fingerprinting can detect the presence of unauthorized OSs.

M

any modern malware implementations carry out their activities using virtual machines to escape detection from antivirus software running on the host OS.1 This malware can also be used as part of a botnet to transfer information from the infected machine to a command-and-control center. Such malware is hard to detect because the context data and state of the programs run by a virtual machine can’t be accessed by antivirus software installed on the native OS. Due to increasing OS vulnerabilities, enterprise network administrators must regularly perform OS audits.

These help determine various services running on different systems and identify OSs with flaws that might cause vulnerabilities in the enterprise network. Audits also help in configuring network-based intrusion detection systems and maintaining an adaptive enterprise security policy. If OSs’ virtual machines differ from the native OSs, these malicious OSs can be identified and the infected machines can be cleaned. Although various RFCs specify definitions and interpretations of different TCP/IP packet fields,2–7 many fail to specify a standard set of initial values for these fields. As a consequence, developers of various OSs implement the protocol stack with different initial values for these fields.

Similar to the way a human fingerprint serves as a tool to uniquely identify a person, an OS can be uniquely identified on a network by its packet fingerprint. Packet fingerprints are derived from the implementation

60

July/August 2015

SECURITY& PRIVACY

IEEE

dissimilarities of various OSs’ communication protocols. By analyzing initial values of certain protocol flags, options, packet fields, and data in the packets that a host sends over a network, we can determine the OS installed in a host. If the OS determined from the packet generated by an enterprise host differs from the original OS installed in the host machine, an unauthorized OS is likely present. An unauthorized OS might have been installed either by a malicious user without obtaining permission from a network administrator or by a virtual machine installed by malware over a native host OS. In this article, we present an approach to passively fingerprint OSs using the information available from TCP packets as well as a system to use this approach to detect unauthorized OSs in an enterprise network. For more information on related work in OS fingerprinting, see the sidebar.

Proposed Unauthorized OS Detection System for Enterprises Figure 1 shows the outline of our proposed system. The enterprise network connects to the Internet through a firewall that is typically combined with a router. The enterprise router captures outgoing TCP SYN packets from the network, which are forwarded to an OS fingerprinting analyzer that extracts and analyzes their headers. Our OS fingerprinting analyzer applies a Euclidean distance estimation algorithm using specific

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

header fields and determines the OS of the machine generating the packet. The system also contains an OS database that includes information about all the authorized OSs installed in an enterprise’s machines.

Such OS databases might be created through the OS and software audit process. The classifier compares the database’s OS information with the OS fingerprinting analyzer’s results and makes a decision about the presence of an unauthorized OS.

Unauthorized OS present Classifier

OS fingerprint analyzer

Unauthorized OS not present

OS Database

OS Fingerprinting Strategy Our system’s OS fingerprinting approach uses a TCP SYN packet’s TCP/IP headers to identify the OS generating the packet. IP is used to assign logical addresses to users to enable communication, and TCP ensures reliable packet delivery. OSs must conform to the rules specified in the relevant RFCs to implement these protocols.

Again, many RFCs fail to specify a standard set of initial values for the TCP header fields; thus, OS developers implemented the TCP/IP protocol stack and TCP/IP header fields with differing initial values. For example, a TCP/IP header’s window size and time to live (TTL) fields don’t require any specific initial values.

The concept of passive OS fingerprinting arises from the disparity among OSs in the implementation of these protocol fields’ initial values. In our application, we use TTL, total length, window size, and TCP options fields.

Enterprise network

Internet

Firewall part of a TCP/IP router

Figure 1. Proposed system for packet-based unauthorized OS fingerprinting in enterprise networks. The OS fingerprint analyzer utilizes TCP header information to detect the OS that originated the packet. The classifier verifies the detected OS with the authorized enterprise OS database. Deviation of the detected OS from the authorized OS indicates the presence of an unauthorized OS.

Table 1. Popular OSs’ time to live (TTL) and window size values.

OS

TTL

Linux 2.4 and 2.6

64

5,840

Google customized Linux

64

5,720

Time to live. The TTL field indicates the maximum time

Linux kernel 2.2

64

32,120

a datagram can remain in the Internet system. If the TTL field contains the value zero, the datagram must be dropped by a router. Table 1 shows typical TTL values for various OSs.

FreeBSD

64

65,535

OpenBSD, AIX 4.3

64

16,384

Windows 2000

128

16,384

Windows XP

128

65,535

Total length. The total length field indicates the length

of the packet and includes the TCP header, IP header, and payload. This field plays an important role in passive OS fingerprinting. For a SYN packet, the total length is OS specific. Each OS sets its own TCP NOP (No Operation) options, which affect the total length, making each OS unique in total packet length. In some cases, we can identify an OS by the SYN packet length alone.

Every OS uses at least one TCP option field, typically maximum segment size, so the TCP SYN packet length should be at least 44 bytes. Thus, a 40-byte TCP SYN packet is a strong indication of a crafted packet.

Window size. This field tells us how many octets of

data the receiving computer is ready to accept. Once a connection is established and data is transferred, the window size changes depending on how much data occupies the receiver buffer. The initial value for window size can be different for each OS. Table 1 also shows popular OSs’ initial window size.

Window size (bytes)

Windows 7, Vista, and Server 8

128

8,192

Cisco Router IOS 12.4

255

4,128

Solaris 7

255

8,760

64

65,535

MAC

TCP options. At times, the TCP options field alone can

identify OSs. Typically, it includes the following: ■ Maximum segment size. This field communicates the maximum receivable segment size at the TCP endpoint sending this segment.

■ Time stamp options. The time stamp options field carries two four-byte time stamp fields. The time stamp value (TSval) field contains the current value of the time stamp clock of the TCP endpoint sending the option.

The time stamp echo reply (TSecr) field is valid only if the ACK (acknowledgment) bit is set in the TCP header. If TSecr is valid, it echoes the TSval sent by the 61

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

OS FINGERPRINTING

Table 2. Popular OSs’ packet size, IP header, and TCP options fields.

OS

TTL

Packet size (bytes)

NOP (No Operation)

Selective acknowledgment (SACK OK)

Don’t Fragment (DF)

Time stamp

Linux

64

60

1

1

1

1

OpenBSD

64

64

1

0

1

1

64

44

0

0

1

0

Windows 2000

AIX 4.3

128

48

1

1

1

0

Windows 7

128

52

1

1

1

0

Windows 8

128

52

1

1

1

0

Cisco IOS

255

44

0

1

0

0

Solaris 7

255

44

0

1

1

0

64

60

1

0

1

1

MAC

Table 3. Numeric transformation of field vectors.

Vector

Transformation

TTL

64 = 1, 128 = 2, 255 = 3

Window size

log10 (window size)

Packet size

log4 (packet size)

NOP

Unchanged

SACK OK

Unchanged

DF

Unchanged

Time stamp

Unchanged

remote TCP endpoint in the TSval field of a time stamp option field. When TSecr isn’t valid, its value must be zero. The TSecr value will generally be from the most recent time stamp option received; however, there are exceptions as explained in the RFCs.2–7 ■ SACK OK (selective acknowledgment). Selective acknowledgment informs the sender of all received data so that the sender can retransmit only data that wasn’t received.

■ NOP. NOPs are one byte in size and used to pad the TCP options field to increase the packet length to a number divisible by four.

The OSs are fingerprinted based on the data shown in Tables 1 and 2. The data is limited to specific versions of popular OSs; most of today’s enterprise networks work only with these sets of OSs. Different versions of Linux vary only in window size. Therefore, if we fail to obtain an exact version of Linux, our system just estimates the OS to be Linux. Similarly, multiple versions of the same OS vendor might have identical fields; in this case, our system returns the names of both versions.

We obtained the data for different versions of Windows, Linux, MAC, Cisco IOS, and AIX 4.3 through 62

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

manual examination of their TCP SYN packets. The rest of the data was obtained from RFC 1323.6 To identify OSs based on Tables 1 and 2, we used Euclidean distance to obtain the deviation from the different OSs’ defined values. The Euclidean distance for an n-dimensional vector is

d(x , y) =|| x − y ||=

∑

n

(xi − yi)2 ,

i=1

(1)

where d is the distance between two vectors x and y, and i varies from 1 to n where n is the size of the vector. The OS fields in Tables 1 and 2 are used as an n-dimensional vector, for example, x = [TTL, packet size, window size, NOP, SACK OK, Don’t Fragment (DF), time stamp], n = 7, and any new outgoing TCP SYN packet from the enterprise network is used to create another n-dimensional vector using the same fields. We compute the Euclidean distance between two vectors using Equation 1 for every set of OSs and choose the one with the least distance (ideally zero) as the OS generating that TCP SYN packet.

To give equal importance to all fields in the distance calculation, we mathematically transform certain fields’ values to comparable values (see Table 3). For example, we take the packet length’s logarithm to the base 4 and window size logarithm to the base 10, and we transform the TTL value to 1, 2, or 3 for the values 64, 128, and 255, respectively. We don’t transform the other fields.

We compare the fingerprinted OS result with the listed OS in the enterprise database of OSs; if a match fails to occur, an alert is generated indicating the presence of an unauthorized or unregistered OS in the enterprise. The alert could be used for manual verification of a possibly malware-controlled virtual machine.

Performance Results Here we describe the OS fingerprinting results we obtained by applying the algorithm in Figure 2 to a large July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

M q M q

M q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

MqM q THE WORLD’S NEWSSTAND®

1. For every TCP SYN packet: 2. Create a vector ai from the fields listed in Tables 1 and 2 … i = 1, 2, … n after mathematical transformation of certain fields.

3. For every OS: 4. Compute bi, where bi is the mathematically transformed field’s vector of known fingerprints of a different OS. 5. Find a set of Euclidean distance using Equation 1 between ai and bi.

6. End for.

7. Find the minimum distance among the set obtained in line 5.

8. If(minimum distance is unique): 9. The machine OS is the one corresponding to the minimum distance.

10. End if.

11. If(minimum distance not unique): 12. Multiple OSs returned corresponding to each distance.

13. End if.

14. Compare the OSs returned in lines 9 or 12 with the enterprise’s OS database.

15. If(match occurs): 16. Unauthorized OS not present.

17. Else: 18. Unauthorized OS present.

Figure 2. Packet-based OS fingerprinting algorithm. Our algorithm first extracts the TCP SYN packet fields and then estimates the Euclidean distance with known OSs’ TCP SYN fields. The closest distance is used for detecting the OS. Once a detection is made, it is verified against the enterprise’s OS audit database to confirm the presence of an unauthorized OS.

set of network packets. We installed a virtual machine using Windows OS on a machine with a native Linux OS. We captured and analyzed packets generated from both the OSs using our proposed technique (see Table 4). A group of packets generated from the Linux OS was successfully identified as being generated from Linux, and the packets generated from the Windows 7 machine were identified as being generated from either Windows 7 or Windows 8, as the initial values of the TCP header fields used by both OSs are almost the same.

We also applied our proposed solution to a subsection of a university’s network. Using our algorithm, we analyzed 2,000 SYN packets captured from a common router. We correctly fingerprinted 95.5 percent of the systems. Approximately 86.3 percent of the systems exactly matched the reference OSs from the enterprise database and were determined with zero distance; 9.2 percent were determined correctly using the minimum distance approach even though there wasn’t an exact match.

Approximately 4.5 percent of the results were incorrect. Because we captured packets from a common router, some packets were from mobile phone OSs whose information wasn’t incorporated in our database.

Others were packets whose OSs were incorrectly identified. Table 5 summarizes the results.

Our OS fingerprinting approach can also be used to detect the OS distribution in enterprise networks or public network environments. For example, Figure 3 shows

Table 4. Fingerprinting results from a machine using a virtual OS. No. of packets

Original OS

Fingerprinted OS

15

Linux

Linux

15

Windows 7

Windows 7 or Windows 8

Table 5. OS fingerprinting results in an enterprise scenario.

OSs successfully determined with exact match (%)

OSs successfully determined with minimum distance approach (%)

False positives (%)

86.3

95.5

4.5

the OS distribution of a typical university dormitory building, where close to 85 percent of OSs reported were identified as Windows. Furthermore, the fingerprinted OS can be verified against the database of the OSs in the enterprise to evaluate the results.

Although our system can be useful, it can also be attacked by ■ poisoning the database of an enterprise’s authorized OSs, ■ spoofing packet field contents to avoid detection by our solution, or 63

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

OS FINGERPRINTING

Related Work in OS Fingerprinting

I

Percentage of OSs

n “HTTP Fingerprinting and Advanced Assessment Techniques,” Saumil Shah proposed a solution to distinguish HTTP server software and the OS by using information included in the HTTP responses.1 In “Remote OS Detection via TCP/IP Stack Fingerprinting,” Gordon “Fyodor” Lyon used the Network Mapper (Nmap; www ___ .nmap.org) to fingerprint servers.2 The Nmap tool has a remote OS fingerprinting function; it sends probe packets to target devices and monitors the response. The application then determines the target’s OS from the response packets. Because Nmap usually initiates a sequence of conversations with the target system, the target system can detect such attempts and disable the responses. Nmap utilizes a database of OS fingerprints that helps identify the hosts’ OSs. Active OS detection wastes bandwidth and reveals information about the testing process. In comparison, our solution is a passive OS detection approach using a nonexact matching of fingerprints.

In “Passive OS Fingerprinting by DNS Traffic Analysis,” Takashi Matsunaka and his colleagues utilized characteristics of DNS queries specific to each OS, such as unique domain names, query and patterns, and time interval.3 The Passive OS Fingerprinting Tool (www.stearns.org/p0f) estimates the number of devices with each OS from the number of queries by using the characteristics of the time interval patterns. This tool has limited passive OS fingerprinting capability. Similarly, tools such as Ettercap (http://ettercap .github.io/ettercap) ___________ and Siphon (https://github.com/unmarshal /siphon) ____ are passive OS fingerprinting tools that depend on exact matches of fingerprint values.

80 70 60 50 40 30 20 10 0

In “Passive Operating System Identification from TCP/IP Packet Headers,” Richard Lippmann and his colleagues proposed a solution to identify OSs by using TCP header fingerprinting.4 Their solution relies on an exact matching of the header’s fingerprinted value with the example values. In contrast, our work focuses on OS fingerprinting by looking into enterprise networks’ TCP packet header fields, transforming the default header values, and comparing them using a Euclidean distance method to avoid the need for exact matches. Furthermore, in our solution, when a TCP packet is detected as originating from a particular OS, this information is verified with the enterprise’s OS database to determine the presence of an unauthorized OS. References 1. S. Shah, “HTTP Fingerprinting and Advanced Assessment Techniques,” BlackHat, 2003; www.blackhat.com/presentations/bh -asia-03/bh-asia-03-shah/bh-asia-03-shah.pdf. __________________________ 2. G.F. Lyon, “Remote OS Detection via TCP/IP Stack Fingerprinting,” Nmap, 2011; http://nmap.org/book/osdetect.html.

3. T. Matsunaka, A. Yamada, and A. Kubota, “Passive OS Fingerprinting by DNS Traffic Analysis,” Proc. 27th IEEE Int’l Conf.

Advanced Information Networking and Applications (AINA 13), 2013, pp. 243–250.

4. R.P. Lippmann et al., “Passive Operating System Identification from TCP/IP Packet Headers,” Proc. Workshop on Data Mining for Computer Security (DMSEC 03), 2003; www.ll.mit.edu/mission /cybersec/publications/publication-files/full_papers/03_POSI ____________________________________ _Lippmann.pdf. _________

come across malware that manipulates TCP headers generated by a virtualized OS to match the native OS in order to evade OS fingerprinting. Finally, because it’s a passive monitoring system, it’s less vulnerable to DDoS attacks.

Windows 7 or 8

Windows 7 or Vista OS vendor

Linux

Figure 3. OS distribution in a university network. Our technique can be used to identify the distribution of the OSs in a campus network or a public network.

■ performing a distributed denial-of-service (DDoS) attack on the computer that runs our solution.

Existing database authorization solutions can protect the database we need to use for authorized OS information in enterprises. Furthermore, we haven’t 64

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

O

ur system can be deployed for unauthorized OS detection in large enterprise networks. The solution requires information on authorized OSs installed on different enterprise network machines. Because OS audits in enterprises are becoming an important process, our system can be very effective for enterprise network security.

References 1. S.T. King and P.M. Chen, “SubVirt: Implementing Malware with Virtual Machines,” IEEE Symp. Security and Privacy, 2006; doi:10.1109/SP.2006.38.

2. RFC 791—Internet Protocol, Sept. 1981; www.faqs.org /rfcs/rfc791.html. __________ July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

3. RFC 792—Internet Control Message Protocol, Sept.

1981; www.faqs.org/rfcs/rfc792.html.

4. RFC 793—Transmission Control Protocol, Sept. 1981; www.faqs.org/rfcs/rfc793.html.

5. RFC 1122—Requirements for Internet Hosts—Communication Layers, Oct. 1989; www.faqs.org/rfcs/rfc 1122.html. ______ 6. RFC 1323—TCP Extensions for High Performance, May 1992; www.faqs.org/rfcs/rfc1323.html.

7. RFC 1349—Type of Service in the Internet Protocol Suite, July 1992; www.faqs.org/rfcs/rfc1349.html.

Organisation. His research interests include computer network security using network analysis. Paul received a B.Tech in avionics from the Indian Institute of Space Science and Technology. Contact him at tsagar50@gmail.com. _____________ B.S. Manoj is an associate professor in the Avionics Depart-

ment of the Indian Institute of Space Science and Technology. Manoj received a PhD in computer science and engineering from the Indian Institute of Technology Madras. Contact him at bsmanoj@ieee.org. ___________

Rohit Tyagi is a scientist/engineer at the Indian Space

B. Thanudas is a scientist and engineer at Vikram Sarab-

Research Organisation. His research interests include designing technologies for information security.

Tyagi received a B.Tech in avionics from the Indian Institute of Space Science and Technology. Contact him at __________________ rohit01011992@yahoo.com.

hai Space Centre and a research scholar at Dr. M.G.R. Educational and Research Institute University. Thanudas received an M.Tech in computer science from the Indian Institute of Technology Madras. Contact him at b_thanudas@vssc.gov.in. _______________

Tuhin Paul is a scientist/engineer focusing on micro-

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

wave remote sensors at the Indian Space Research

Take the CS Library wherever you go!

IEEE Computer Society magazines and Transactions are now available to subscribers in the portable ePub format.

Just download the articles from the IEEE Computer Society Digital Library, and you can read them on any device that supports ePub. For more information, including a list of compatible devices, visit

www.computer.org/epub

65

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

EXPLOIT DEVELOPMENT

Bypassing EMET 4.1 Jared DeMott | Bromium

As part of Microsoft’s ongoing effort to make computing more trustworthy, the company released the Enhanced Mitigation Experience Toolkit, a Windows tool that can stop common exploit patterns and techniques. But how difficult is it to bypass this tool’s protections?

I

n 2012, Microsoft released the Enhanced Mitigation Experience Toolkit (EMET) as an extra layer of defense against malware attacks. Because it’s freeware, it was intended for use after the firewall but before antivirus software. In the years since EMET’s debut, Microsoft researchers Neil Sikka (www.youtube .com/watch?v=Vyi8b3VOw9M#!), Elias Bachaalany (http://recon.cx/2013/slides/Recon2013-Elias%20 Bachaalany-Inside%20EMET%204.pdf#!), and oth__________________________ ers have given excellent technical talks on the product, which can stop common exploit patterns and techniques, such as return-oriented programming (ROP). Many of EMET’s ROP protections came from the secondplace winner (htt ps://ropguard.googlecode.com/svn ________________________ /trunk/doc/ropguard.pdf) of the 2012 BlueHat con________________ test (www.microsoft.com/security/bluehatprize).

But how difficult is it to breach EMET? Researchers at Bromium, an endpoint security company, were able to bypass EMET’s protections in both example code and with a real-world browser exploit. Our findings indicate that technologies operating on the same plane of execution as potentially malicious code offer little lasting protection.

ROP Background ROP is an evolution of the return-to-libc code reuse attack, which triggers existing code rather than injecting exploit code (htt p://seclists.org/bugtraq/1997 _____________________ /Aug/63). ROP expands on the idea of executing code ______ 66

July/August 2015

SECURITY& PRIVACY

IEEE

that already exists in memory by tying together snippets of this existing code. Figure 1 illustrates the process, in which the program is maliciously redirected to one or more instructions just above a return. Each small series of instructions before a return is called a gadget, with the return at the end of a gadget calling the next gadget and so on, until an ROP chain comprised of connected gadgets emerges, essentially containing a set of addresses and whatever data the gadgets require.

Because the ROP portion is difficult to write, the typical attack approach is to minimize it and instead use a standard payload called a shellcode. Thus, the ROP portion either adds execute permissions on the memory page containing the shellcode or allocates a new executable page and copies the shellcode there.

But first, a pivot is required—that is, the stack pointer (SP) must be adjusted such that it points to attackercontrolled data to initiate the ROP chain. This pivot is the first step, and the difficulty depends on the nature of the exploit. For stack bugs, it’s a matter of adjusting the SP; for heap bugs, the SP must point to user-controlled “malicious” input on the heap.

Figure 2 gives an overview of a complete stack pivot plus ROP. To execute a standard exploit payload as shown, the current heap page permissions need to be modified (or newly allocated) so that they’re also executable. By default on modern systems, code is readable and executable, whereas data is read and write. Figure 2 shows the SP being pivoted off the

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

stack to the heap before the ROP payload executes to change page permissions—then a standard payload can be run. This is the typical approach, although several variations exist.

EMET’s ROP Protections To prevent these types of attacks, EMET offers five ROP protections—LoadLibrary, Caller, SimExecFlow, MemProt, and StackPivot—that can be enabled and disabled for each protected application. Figure 3 shows EMET’s GUI. While a protected program executes, EMET considers more than 50 functions to be “critical.” These functions are hooked, meaning that at the start of each one, stub code jumps to do security checks before the real function is allowed to execute. However, sneaky attackers can find a way to disguise the payload or bypass the check altogether—for example, they might jump around the hooks instead of using the normal API. If any of these security checks detects an attack, EMET either closes or crashes the application and attempts to write a log message that can be viewed through Windows Event Viewer. Because EMET is an injected code module (a Windows DLL), reporting isn’t always possible—if the process memory is corrupted beyond repair, the process crash happens before the inprocess logging code can execute. This is a fundamental weakness of in-process protections.

The first check we investigated was LoadLibrary.

Loading a library is a common need in attacker-created shellcode because it pulls in extra functionality. Thus, the LoadLibrary function is hooked by EMET. The added check ensures that a shellcode doesn’t use remote file paths.

Caller is the name of the function calling check, which ensures functions are called as they should be (not returned into as in ROP). Before one of the more than 50 important APIs is allowed to run, EMET walks up and down the call stack to verify that the target is properly called and not returned or jumped into.

SimExecFlow is the reverse of Caller: after a critical API completes, this protection simulates the execution as it moves forward to ensure that the code following the protected function looks normal (and isn’t ROP).

The first return address is given on the stack, and subsequent return addresses are deduced by simulating instructions that modify the stack/frame pointer. Each return address must be preceded by a call instruction to appear normal. SimExecFlow and Caller work together to make sure that critical APIs are called as expected.

MemProt checks memory protection functions such as VirtualProtect to make sure they aren’t trying to mark stack memory as executable (which enables shellcode to be run).

Program flow sub5to8 Instruction Instruction Instruction Instruction return

Attacker controlled 5 6 7 8

sub12to20 Instruction Instruction Instruction Instruction Instruction Instruction Instruction Instruction Instruction return

sub9to11 Instruction 9 Instruction 10 Instruction 11 return

12 13 14 15 16 17 18 19 20

Figure 1. Return-oriented programming (ROP) visualization. The program is maliciously redirected to one or more instructions just above a return.

Stack Code reuse attack

Heap

Original stack pointer

Current stack frame

Pivot stack pointer Malicious input

ROP Standard shellcode

Figure 2. Pivot and ROP before payload. To execute a standard exploit payload as shown, the current heap page permissions must be modified (or newly allocated) so that they’re also executable.

StackPivot is the stack register’s sanity check. Upon entering a critical function, EMET checks to ensure that the SP is within the thread’s upper and lower specified stack limit. This guards against pivoting the SP to, say, attacker-controlled heap memory—a common technique in some exploits.

Bypassing EMET via Sample Programs To see if we could bypass EMET’s five ROP protections, we installed EMET 4.1 on our test computer, which included the 32-bit version of Internet Explorer (IE) 9 on a 64-bit Windows 7 platform. We then created a simple program (vuln_prog.exe) that had a trivial stack buffer-overflow vulnerability via a file read. (Later, we created a sample program with a heap overflow to test the StackPivot protection.) For the vulnerability, we assumed that the attacker had 67

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

EXPLOIT DEVELOPMENT

Export Export Selected File Mitigations All

Memory

Add Add Application Woldcard Add/Remove ROP

Remove Show Full selected Path Options

Stop on exploit

Deep Hooks

Audit only

Banned Functions

Default Action

Anti Detours

Mitigation Settings

Other Find

Clear

Figure 3. Enhanced Mitigation Experience Toolkit (EMET) GUI showing all 12 protections. The checkmarks indicate the protections enabled for each application. Usually, all protections are enabled, but sometimes select ones must be disabled for the application to run normally.

■ control over the input to trigger the bug, ■ an additional memory leak or information disclosure bug to find a useful DLL’s base address, and ■ a pregenerated ROP chain that can be rebased at runtime.

Because these are fairly common, we simulated them in our example programs: ■ We used msvcr71.dll as our discovered DLL because it has been so often abused in the past.

■ We didn’t spend much time fiddling with a real memory leak bug or searching for gadgets (these weren’t the focus of our study).

In some cases, we disabled address space layout randomization (ASLR) on the main binary and added gadgets to the .exe if proper gadgets weren’t immediately obvious in msvcr71.dll. Our assumption is that useful gadgets can usually be found, as recent research shows (https://www.usenix.org/system/files/conference _________________________________ /woot12/woot12-final9.pdf). __________________

LoadLibrary EMET can catch a stock Metasploit payload. EMET alerted in a LoadLibrary call while the Metasploit shellcode was executing, but not because of the LoadLibrary itself. The LoadLibrary rule simply checks whether a remote DLL load was attempted. But, in this case, all the code modules were on the local file system. Actually, it was the Caller check triggering again. As before, we could bypass EMET if we properly called LoadLibrary, 68

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

rather than jumping it. After this upgrade, EMET still caught our Metasploit payload, but only after the attack succeeded. As Figure 5 shows, EMET triggered one of the seven non-ROP rules (there are 12 total, as shown in Figure 3) called export address filtering (EAF), which protects the export address table (EAT) of code modules.

To do something “useful,” shellcode generally needs to call Windows APIs. However, to call an API, shellcode must first find the address where that API has been loaded. To do this, most shellcode iterates through the EAT of all loaded modules, looking for modules that contain useful APIs. Typically, this involves kernel32 .dll or ntdll.dll. Once it finds an interesting module, the shellcode can then figure out the address where an API in that module resides. This mitigation filters accesses to the EAT, allowing or disallowing the read/write access based on the calling code. With EMET in place, most of today’s shellcode will be blocked when it tries to look up the APIs needed for its payload.

Rather than use a typical Metasploit payload (which EMET might catch), we created our own shellcode that performs similar actions. This new shellcode includes the (protected) LoadLibrary and GetProcAddress functions, just as in real payloads, except we call them instead of jumping, as Metasploit does. For our bypass example, we displayed a message box (see Figure 6).

Caller The Caller check decides if a protected function is properly called or illegally returned into. Using Python to generate input, we created a simple ROP chain exploit July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Figure 4. Caller security check. EMET blocks the attack and generates a log entry.

Figure 5. Export address filtering (EAF) stops an exploit. EAF prevents shellcodes from scanning the export address table.

and delivered it to our test program, and EMET successfully blocked it. As Figure 4 shows, the Microsoft event log reports “CallerCheck Failed” and gives some details.

Although there are multiple ways to bypass this check, the simplest would be ■ using an API other than VirtualProtect/VirtualAlloc, if such a useful API exists (in other words, using a nonprotected function) or ■ finding existing code that does a valid call to one of those two APIs and then using that code instead of returning into the API. We chose the latter approach, found a call to VirtualAlloc inside msvcr71.dll, and updated our ROP chain

to call this location, rather than directly returning to the VirtualAlloc address. This exploit contained a stock Metasploit reverse shell payload. Metasploit is a common tool used by penetration testers, and reverse shell simply means that the compromised host connects back to the awaiting attacker machine and provides command-level access to the victim.

SimExecFlow SimExecFlow attempts to verify a proper return from a critical function by checking to see if legitimate calls were used rather than returning to the API. Because we returned to code that legitimately calls the critical function (VirtualAlloc location to bypass the Caller check), we bypassed this security check without extra effort.

69

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

EXPLOIT DEVELOPMENT

A critical question is whether such gadgets can really be found. We experimented with a real-world problem to find out.

Real-World Example

Figure 6. EMET bypass example, which displays a message box.

MemProt The MemProt rule triggers when VirtualProtect is called, checking to see if the attacker is changing execution permissions on stack pages. Because we used VirtualAlloc rather than VirtualProtect to mark stack pages with our shellcode, we inherently bypassed this rule in our sample exploits.

StackPivot We did a pivot in our attack, but because the attackercontrolled data was on the stack, this check passed without issues. To make things more interesting, we constructed another program in which our input was on the heap—specifically, we changed the bug to be a function pointer overwrite on the heap. In light of the current threat landscape—with better operating system mitigations and fewer simple bugs—this type of attack is more common than stack overflows.

To bypass the StackPivot check, we first used a relocation copy loop to move our ROP chain from the heap to the stack. In this version of the attack, the copy code is all one assembly code, but in reality, the copy code would be a series of ROP gadgets chained together to achieve a similar result. Next, we called VirtualAlloc and copied our custom shellcode to the readable, writable, and executable (RWX) address that it returned.

Finally, we executed a payload.

Simple Example: Lessons Learned From our simple example, we can summarize the lessons we learned: ■ We didn’t directly return into critical APIs and thus bypassed the Caller and SimExecFlow rules.

■ We avoided remote file paths to bypass the LoadLibrary rule.

■ We didn’t attempt to use VirtualProtect on stack pages, thus bypassing the MemProt rule.

■ We avoided the StackPivot rule by copying and running our core ROP chain on the stack and then jumping to actual shellcode in a newly allocated memory page.

70

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

CVE-2012-4969 is a use-after-free (UAF) bug in Microsoft’s IE browser, reported on 14 September 2012 by Eric Romang (www.cve.mitre.org/cgi-bin /cvename.cgi?name=CVE-2012-4969). _________________________ A public exploit for it in Metasploit depends on the presence of a non-ASLR module; by default, EMET forces all modules to use ASLR. Accordingly, we used a more sophisticated exploit.

Our upgraded exploit is better because it dynamically finds ntdll.dll’s base address and adds that address to each gadget’s relative offset to construct the ROP chain at runtime. Our new exploit also contains a custom payload. The base address is determined via pointers to a shared data address, and these pointers were set only on 32-bit code running on 64-bit Windows. This Windows shared data bug has since been fixed, but the same UAF IE flaw could be modified to leak a DLL base address in another way, so the fact that the original technique is now patched doesn’t solve the problem. After some minor tweaks to the ROP chain, the exploit worked perfectly in our environment.

We tried the exploit again, but with EMET 4.1 installed. EMET blocks the exploit via the StackPivot check because the exploit is attempting to use VirtualProtect to mark the heap as executable, whereas the stack register is pointing to the heap rather than the legitimate stack (because of the required stack pivot before ROP).

As part of this research, we were curious to see if the exploit could be enhanced to bypass EMET 4.1, using the techniques previously discussed. We especially wanted to see if we could develop an EMET bypass technique for the StackPivot check because this protection hasn’t been publicly bypassed to our knowledge.

Although we later learned that someone had done something similar, it wasn’t against EMET (http://vuln factory.org/blog/2011/09/21/defeating-windows-8 -rop-mitigation); __________ other researchers have talked about ideas or techniques to bypass some of the other protections (see the “Related Work in Protection Bypassing” sidebar).

Our StackPivot bypass idea is simple (and similar to the example problem previously discussed): ■ Pivot the SP to the heap as normal.

■ Use a first-stage ROP chain to “pop-copy” a secondstage ROP chain to the stack. Pop-copy is a way to use ROP gadgets to copy data from one place to another.

■ Unpivot back to the stack and execute the second July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Related Work in Protection Bypassing

M

y team at Bromium wasn’t the first to show that the Enhanced Mitigation Experience Toolkit (EMET) could be bypassed, but we’re the first to do so with so much technical detail. For more code-level detail about our work, visit http://bromiumlabs .files.wordpress.com/2014/02/bypassing-emet-4-1.pdf. The following is a partial list of other researchers who have conducted similar EMET research: ■ ■

■

■

Gal Badishi summarizes export address filtering bypasses at http://badishi.com/tweaking-metasploit-modules-to-bypass-emet-part-2.

Shahriyar Jalayeri bypassed EMET 3.5 by resolving the ZwProtectVirtualMemory system call via a shared-memory pointer, marking his shellcode readable, writable, and executable. Once his shellcode was running, he disabled EMET as his primary bypass technique. He released an exploit for CVE-2011-1260 (http://news.softpedia.com/news/Expert-Bypasses-EMET-3-5-ROP-Mitigations-Microsoft -Responds-286424.shtml). ______________ Aaron Portnoy showed how to bypass EMET 4.0 during a Nordic Security Conference talk (www.exodusintel.com/files/Aaron _Portnoy-Bypassing_All_Of_The_Things.pdf). __________________________ 0xdabbad00 released a paper called “EMET 4.1 Uncovered,” in which he explained EMET and discussed some hypothetical strengths and weaknesses of EMET protections (http://0xdabbad00.com/wp-content/uploads/2013/11/emet_4_1_uncovered.pdf).

stage, which uses VirtualProtect to mark the heap as executable.

■ For the final stage, jump off the stack, go back to the heap, and execute an EMET-friendly exploit payload.

As Figure 7 shows, the pop-copy gadgets are from ntdll.dll. The hex number in front of each pictured ROP gadget is the offset added to ntdll.dll’s base address to achieve the runtime gadget address.

Pop-copy works by popping 32 bits from the input data and copying them to the desired location (in this case, the stack) via a dereferenced move. Then the destination pointer (the stack) is incremented (by four for a 32-bit system). This particular pop-copy isn’t space efficient as it produces a total ROP chain that’s six times the original size. This didn’t matter in our example, and work could likely be done to find a more efficient popcopy technique.

The second-stage ROP chain in Figure 8 uses an undocumented system call from ntdll.dll (NtProtectVirtualMemory) to mark the relevant heap page RWX. We found that NtProtectVirtualMemory is hooked only when “deep hooks” are enabled. Deep hooks add more functions to the protected list (more than 50 by default). The reason they can be enabled separately is because enabling them might lead to more compatibility issues (for example, applications crash when not under attack). After we presented our findings to the EMET team, that team’s members decided to turn on these extra hooks for upcoming EMET releases, because most of those compatibility issues have now been worked out. However, VirtualAlloc could be used here rather than VirtualProtect to bypass MemProt, so even after this fix, successful attacks are possible. Finally,

Figure 7. Pop-copy. The pop-copy works by popping 32 bits from the input data and copying them to the desired location (in this case, the stack) via a dereferenced move. Then the destination pointer (the stack) is incremented (by four for a 32-bit system).

the second-stage ROP chain jumps back to the start of the shellcode that’s on the executable heap page.

Once we bypassed the StackPivot protection, EMET blocked our exploit with the EAF check (as in our earlier Metasploit payload example), so we added stub code that we modified for Windows 7 (http://piotrbania.com/all /articles/anti_emet_eaf.txt). The bypass works by call_________________ ing NtSetContextThread to disable the current thread’s hardware breakpoints, which is how EMET detects that a shellcode is attempting to resolve functions via the export table. To bypass the EAF check, we simply compiled the EAF stub with the nasm assembler (www.nasm.us) and added that blob of bytecode to an existing shellcode.

We were able to bypass or ignore all 12 EMET protections with this exploit. We focused on bypassing ■ StackPivot protection by using a pop-copy to the stack, a second pivot to the stack to execute critical ROP code, and a final jump back to an EMET-friendly payload; ■ EAF filtering by clearing the debug registers, which are key to this protection; and ■ the remaining checks by calling an unprotected 71

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

EXPLOIT DEVELOPMENT

Stack

version of VirtualProtect, although VirtualAlloc can be used if VirtualProtect is deeply hooked.

Heap Gadget 1 Gadget 2 Store gadget

Disable data execution protection Jump to shellcode

Second stage gadget 1 Copy gadget Store gadget Second stage gadget 2

Run ROP on stack

Copy gadget Store gadget Export address filtering stub Shellcode

Now executable

Figure 8. Overview of final attack. The second-stage ROP chain uses NtProtectVirtualMemory to mark the relevant heap memory page as readable, writable, and executable.

stay connected.

To summarize how EMET works: there are 12 total protections. Of those 12, five are focused on ROP. One or more of the five ROP protections could be triggered any time one of the more than 50 protected functions are executed (or more if deep hooks are enabled). Jump hooks allow the ROP checks to occur before the protected functions run.

W

e provided our work to Microsoft long before writing about these weaknesses publicly, to provide the company with an opportunity to address shortcomings and make computers running Windows safer. In particular, we stated that addressing the following weaknesses would help: ■ Hook NtProtectVirtualMemory by default. Microsoft took this advice and has enabled deep hooks in EMET 4.1 update 1 and in 5.0 tech preview, but it’s still possible to change approaches and use VirtualAlloc.

■ Create a new EAF protection scheme, although it wouldn’t stop shellcode that doesn’t use export address resolution.

■ Check more than one call deep to see if code was returned into.

■ Expand ROP mitigations to cover 64-bit code.

Even with these fixes, many of the weaknesses we found are unlikely to be sufficiently addressed in user-space protection technologies such as EMET, which doesn’t even protect against kernel vulnerabilities. Similar technologies likely suffer from the same problem—mitigations that run on an even playing field with malicious code can be bypassed, given sufficient attacker interest. Protections that offer an architectural advantage, such as hypervisor-based isolation, make for a better solution.

Jared DeMott is a principal security researcher for BroKeep up with the latest IEEE Computer Society publications and activities wherever you are. | @ComputerSociety | @ComputingNow | facebook.com/IEEEComputerSociety | facebook.com/ComputingNow | IEEE Computer Society | Computing Now | youtube.com/ieeecomputersociety

mium and a security consultant who has spoken at security conferences such as DerbyCon, Black Hat, Defcon, ToorCon, Shakacon, DakotaCon, and GRRCon. He teaches an application security course for companies and at conferences and coauthored the book Fuzzing for Software Security Testing and Quality Assurance (Artech House Publishing 2008). DeMott received a PhD in computer science from Michigan State University. Contact him at ______________ jdemott@vdalabs.com.

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

72

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

IT ALL DEPENDS Editors: Mohamed Kaâniche, ______________ mohamed.kaaniche@laas.fr | Aad van Moorsel, _____________ aad.vanmoorsel@ncl.ac.uk

The Importance of Security Cases: Proof Is Good, But Not Enough John Knight | University of Virginia

W

hen a system is said to be secure, should we believe that claim? Probably not, for at least two reasons: we know that absolute security is impossible, but we don’t know what exactly is meant by “secure,” and we haven’t been given a reason to believe the claim. Security cases—which document the rationale for believing that a system is adequately secure1—are intended to address both these issues.2 Many powerful security technologies, such as proof3 and highentropy artificial diversity,4 have been developed in response to genuine concerns about damage caused by cyberattacks and the vulnerabilities likely remaining in deployed systems. These technologies deal with major technical challenges, but applying them must be part of a comprehensive defense approach.

Cyberattacks tend to succeed because of the obscure items that we forget, rather than the breakdown of techniques like proof and artificial diversity.

We can’t make our defenses comprehensive with absolute certainty. But a security case can help us identify issues that we might otherwise miss.

1540-7993/15/$31.00 © 2015 IEEE

Copublished by the IEEE Computer and Reliability Societies

SECURITY& PRIVACY

IEEE

Anatomy of a Security Case By examining a security case, any interested party should be able to understand why the various stakeholders, including system developers, owners, and regulators, believe in the system’s security.

Figure 1 illustrates the primary components of a security case, which include a security claim, the context within which the system will operate, the assumptions that are made about the system, a body of evidence about the system and its development, and a rigorous argument linking the body of evidence to the security claim.

A security case makes explicit the key items that should be documented when considering security for any system. The security claim documents the property we

desire. The property must be stated precisely using terms for which clear, agreed-on definitions exist, because the rest of the security case will argue that the system possesses this property. “The system must be secure” isn’t enough—instead, a property could be stated in terms of the threats that will be tolerated and the classes of vulnerabilities that have been eliminated, for example, “all buffer-overflow vulnerabilities have been eliminated from the software.”

The context includes the threat model, a detailed explication of the environment in which the system will operate, the expected user population, and the system’s implementation details. Such material is always considered in system development but rarely comprehensively assembled and addressed.

The assumptions that are made about a system’s security include propositions that are assumed to be true without further elaboration.

It might be necessary, for example, to assume that the developers don’t pose an insider threat and that the target hardware comes from a trusted supply chain.

The body of evidence includes the results of the analysis conducted on the system, along with details of the development process. System analysis could include testing, static analysis, proof, and inspection. Details of the process could include the software methodology (for example, spiral or agile), process audit tools, and configuration management approach.

Finally, a rigorous argument documents how the developers think the evidence justifies their security July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

73 M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

IT ALL DEPENDS

Assumptions made about the system

Security claim

Context of the claim

Rigorous argument linking the body of evidence to the security claim

Body of evidence about the system including details of development, analysis, testing, and so on

Figure 1. A security case for a specific system consists of a security claim, the context within which the system will operate, the assumptions that are made about the system, a body of evidence about the system and its development, and a rigorous argument linking the body of evidence to the security claim.

claim and, in doing so, exposes their rationale to scrutiny. The argument should be valid (without fallacy), complete (cover all concerns), and compelling (evoke a strong degree of belief).

Rigorous Arguments The least familiar component of a security case is the rigorous argument. The other components are within the scope of traditional development, although they have to be more carefully prepared, scrutinized, and assembled in a security case.

The structure of a rigorous argument is a hierarchy of claims in which belief in a claim at one level is justified by belief in the claims at the level below it, using a documented strategy that links the levels. Starting with the original security claim, claims are progressively refined until they can be established with evidence. Consider a claim that a set of vulnerabilities to which a system might be subject has been eliminated. The argument might use a strategy of arguing separately for each type of vulnerability, with a separate claim for each type at the next level of refinement. The next 74

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

level might argue that suitable detection methods for each vulnerability type have eliminated all instances of that type. Finally, evidence could be added for each detection method that a technique such as static analysis has successfully located the vulnerabilities. John Goodenough and his colleagues present a more detailed example.5 A major benefit of rigorous argument lies in the identification of questions that raise concern and force us to examine both the system under development and the techniques being used. Using the same example that all vulnerabilities have been found, such questions could include the following: ■ Were all possible types of vulnerabilities identified, or might other unidentified vulnerability types exist?

■ Are the various detection methods used technically capable of identifying each type of vulnerability, or could some escape detection? Could the application of parallel, redundant techniques provide more confidence?

■ Were the detection methods

applied completely and correctly?

Were the results interpreted, stored, and reported correctly?

Could an important warning have been missed?

Questions such as these often derive from circumstances in which human judgment is called upon to make a technical decision. The first question can rarely be answered by purely technical means, and experts are frequently called upon to help answer it. Human judgment is often wrong, and these types of questions allow us to focus on difficult areas, such as decisions based on human judgment.

The Benefits of a Security Case Developing a security case is quite an undertaking, in addition to the other work involved in building a system. A security case itself doesn’t make a system more secure, but it can provide the following benefits: ■ Helps generate questions about the comprehensiveness of every step of development and analysis. These questions often reveal previously unaddressed potential weaknesses.

■ Allows reasoned judgment about the adequacy and suitability of the evidence that will be generated during development and of the technology generating that evidence.

■ Documents rationale for belief in the security claim, thereby offering the opportunity for scrutiny of that rationale. This scrutiny can take place throughout development as part of the engineering process, allowing higher degrees of confidence that the development will be successful.

Separately, the availability of the rationale for belief in the security claim provides an ideal basis for an approval process by a regulatory organization.

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

Figure 2 illustrates a development process in which a security case has been integrated.

To gain the maximum benefit, a system’s security case needs to be developed along with the system and viewed by all stakeholders as an asset. Planning a security case at the outset of system development can influence many aspects of the system, including the methods and technology to be used, the architectural means of addressing security concerns, and the analysis of the system that will be conducted.

Reviewing all aspects of the evolving security case provides the opportunity to question the adequacy of the security mechanisms being used. Systematic examination of the way in which vulnerabilities have been identified and dealt with, guided by the rigorous argument, allows the technology to be refined (and the security case to be revised) during development.

P

owerful new technologies bring many benefits, but new technologies alone aren’t enabling the development of systems that we can trust. The fault doesn’t lie in the technologies but in our failure to recognize their limitations and to identify weaknesses such as reliance on human judgment. Security cases are not a panacea but an orthogonal approach that directs our attention to issues that tend to slip through the cracks. Developing security cases for security-critical systems is a lowrisk, high-payoff path to follow.

Functional and security requirements

Development planning

System development

Security case

Security-critical system

Engineering scrutiny

Approval or audit

Figure 2. Building a security case along with the system requires that engineers document the rationale for the selected technology as it is chosen and used. As the security case evolves, this rationale is available for scrutiny by any legitimate stakeholder, including approval agencies or audit authorities, and provides feedback to allow “midcourse” correction in the development process.

Kelly, Security Assurance Cases: Motivation and the State of the Art, tech. report CESG/TR/2011/1, Dept. of Computer Science, Univ. of York, 2011.

3. G. Heiser, T. Murray, and G. Klein, “It’s Time for Trustworthy Systems,” IEEE Security & Privacy, vol. 10, no. 2, 2012, pp. 67–70.

4. J. Hiser et al., “ILR: Where’d My Gadgets Go?,” IEEE Symp. Security and Privacy, 2012, pp. 571–585.

5. J. Goodenough, H.F. Lipson, and

C.B. Weinstock, “Arguing Security— Creating Security Assurance Cases,” Build Security In, US Dept.

Homeland Security, 4 Jan. 2007; https://buildsecurityin.us-cert.gov /articles/knowledge/assurance ____________________ -cases/arguing-security-creating ____________________ -security-assurance-cases. ______________ John Knight is a professor in the

Computer Science Department at the University of Virginia. Contact him at ______________ knight@cs.virginia.edu.

References 1. B.D. Rodes et al., “A Case Study of Security Case Development,” Proc.

23rd Safety-Critical Systems Symp., M. Parsons and T. Anderson, eds., 2015; http://scsc.org.uk/paper_129 /Rodes%20-%20A%20Case%20 ____________________ Study%20of%20Security%20Case ____________________ %20Development.pdf?pap=968. __________________ 2. R. Alexander, R. Hawkins, and T.

_____________________

75

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY & PRIVACY ECONOMICS Editors: Michael Lesk, _______ lesk@acm.org | Jeffrey MacKie-Mason, _________ jmm@umich.edu

Ideas Ahead of Their Time: Digital Time Stamping Michael Lesk | Rutgers University

O

lder ideas sometimes come back to life. Early ideas might be too far ahead of their time, be phrased in obsolete language, or suffer from bad luck in the exploitation process. Security applications face an additional barrier: the original idea might involve something secret, so others might be unaware of the whole strategy or even the actual idea. Thus, we ought to keep our eyes open for older ideas that might now be practical and useful.

The public tends to believe, perhaps from movies about Thomas Edison, that an inventor has a stroke of inspiration, and the idea moves immediately through the lab to production and worldwide acceptance.

76

July/August 2015

SECURITY& PRIVACY

IEEE

However, real technological change involves a great deal of uncertainty in development, production, and marketing. The people who take an idea and turn it into something with impact should get at least as much credit as those who originated it.

From Innovation to Acceptance It takes longer than we think for innovations to move into practice. As a nontechnical example, consider the discovery of penicillin. Many people are familiar with some variant of the story that Alexander Fleming left a petri dish of bacteria in his lab and went on vacation, leaving the lab door Copublished by the IEEE Computer and Reliability Societies

open. A bit of mold blew into the room and killed the bacteria. The world owes a great deal to Fleming’s failure to tidy up before he left the lab and his attention to the dish when he returned. This happened in 1928, and Fleming published a paper about it in 1929.

Almost nobody paid attention.

It wasn’t until 1938 that two biochemists, Howard Florey and Ernst Chain, followed up. Looking at the prospect of another world war, they decided to postpone basic research in favor of looking for something that would help treat infections. Chain went to the library to look for papers about anything that killed bacteria and spotted Fleming’s paper. By luck, he discovered during a teatime conversation that somebody at Oxford had a sample of the bacteria-killing mold. This sample was critical because Fleming— not a mold specialist—had misidentified the species of mold.

Florey and Chain managed to get enough penicillin to treat a few sick mice in May 1940 and then a sick human in January 1941. The gap between Fleming’s discovery and the attempt to exploit it was nine years.

Digital Time Stamping What called this to my mind was a story in the New York Times about using Bitcoin technology to certify the existence of documents.1 Paper documents can sometimes be 1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

analyzed chemically or microscopically to detect tampering, but establishing that a particular document existed on a particular date is much harder with bitstrings. Paper documents can be notarized, and some companies offer such an electronic service. But two Bellcore researchers, Stuart Haber and Scott Stornetta, had a new idea 25 years ago.

Suppose you have a good hash function. In this case, establishing that a document’s hash existed is equivalent to knowing that the document existed, because you can’t find an alternative string that will hash into the same result. The Haber-Stornetta insight was that if you had many documents and hashed each of them, and then hashed all the hashes together, you had one short string that depended on every original document. You could then publish that one string in a public place—originally the New York Times public notices section— and use it to confirm the existence of the documents you started with.

This was patented as “Method for secure time-stamping of digital documents,” US patent 5136647, filed in August 1990 and granted in 1992.

The algorithm was also published in 1991 in a well-written paper beginning with a Shakespeare quote.2 Originally, the inventors attempted to market the idea as a certification service through a new company they founded, Surety. The idea received minimal public attention,3 although the company ran a weekly public notice in the New York Times to publicly establish the combined hash value (see Figure 1).4 I have no idea whether Times readers ever noticed these.

The time-stamping paper is cited in the original 2008 white paper, Bitcoin: A Peer-to-Peer Electronic Cash System by Satashi Nakamoto (believed to be a pseudonym).5 The same idea is behind the “blockchain” that secures Bitcoin transactions. The original Bellcore patent

would have expired in 2009, before Bitcoin became popular.

But in 2015, the New York Times noted that people had rediscovered the time-stamping idea as separate from Bitcoin and wanted to use it.

So, there was an 18-year delay from invention to use in Bitcoin, and it was another seven years before somebody argued publicly and loudly for extracting the original algorithm and using it independently.

Other Old Ideas Many other examples exist. In the early 1980s, I built a system to give driving directions, but a combination of management and marketing problems and the lack of GPS technology meant that it went nowhere. Fifteen years later, I received a call from somebody asking if the old code could be dug out and made to work. By that time, it was too late—a lot of other people had done it better.

Ideas can also expire. Data compression was once a popular research topic but seemed of little practical importance until CDs were invented and getting as much data as possible onto a CD-ROM was significant for marketing. Then, distribution in physical form faded.

If you look at a Google Ngram search for “data compression” you’ll see interest in it peak from 1994 to 1996, then decline.

When ideas mature has as much to do with technology as marketing and fads. Figure 2 is a Google Trends plot of interest in Bitcoin matched with the price of bitcoins over the same time period. Correlation doesn’t imply causation, of course, but one does wonder whether Bitcoin investors are driving the hype.

If old thoughts are a good source of research ideas, are there other good security-related ideas that could be mined from the literature?

Almost certainly, yes. One of my early advisors was known for going to conferences in the 1980s and

Public Notices

5100

3:34 pm 2/14/92 Bellcore TimeLink Digital Time-Stamp Marker: cb4b62bd61406b80bc53ef9e83b560 These markers prevent computer document tampering. 445 South St. Morristown, NJ, 07960, Rm. 2Q-346.

Figure 1. Publicly establishing the value of the hash of a chain of time-stamped documents in the New York Times.

1990s and claiming after each presentation that he’d thought of that idea in the 1960s.

Security History Security is an area in which it’s likely that good ideas exist but aren’t widely known. All innovators run the risk that their work will be overlooked, perhaps because they don’t publish prominently enough or because their papers don’t happen to be read by anybody. The antibacterial effect of penicillin had been noticed by a French researcher in 1895, but he died young and nobody else carried on his work.

Yet another obstacle with security research is that it’s often done in secret. So mining old ideas for solutions to present-day problems is probably particularly relevant in the security area. There are many examples of hidden and older work that later became important.

For example, today we’re aware that the concepts behind public-key cryptography were discovered by the British security research group Government Communications Headquarters a few years before the better-known announcements by Whitfield Diffie and Martin Hellman. However, the UK researchers didn’t have the idea for digital signatures, nor did they anticipate many of the applications for asymmetric cryptography. The UK work wasn’t disclosed until 1997, and work done in secret doesn’t attract attention or exploitation.

77

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY & PRIVACY ECONOMICS

E D Interest

G

C

F (a)

2011

2012

2013

2014

B A 2015

1,000

US$

750 500 250 0 CoinDesk BPI in effect (b)

2011

2012

2013

www.coindesk.com

2014

2015

Figure 2. Interest in Bitcoin versus the price of bitcoins. (a) Google Trends plot of “Bitcoin” from 2010 to present. (b) Price of bitcoins from 2010 to present.

Similarly, work on code breaking during World War II inspired at least one thought of language translation. In 1947, Warren Weaver of the Rockefeller Foundation wrote a letter suggesting machine translation, saying “one naturally wonders if the problem of translation could conceivably be treated as a problem in cryptography.”6 When I first read this, I of course didn’t know that its author had been aware of computer decryption during the war. I had been taught that the first computers did numerical calculations, especially artillery firing tables. Not until 1974 was it generally known that the first electronic computer did a kind of text processing. Could research have moved faster if we had known of these precedents? Vannevar Bush’s famous Memex essay inspired the whole field of information retrieval,7 and Bush had been working in secret on code breaking with microfilm for years. If others had known of this work, perhaps somebody else might have been inspired to study this area.

78

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

Unearthing Security Innovations So what might we look for in old security articles? Once upon a time, there was a lot of discussion about back doors into encrypted data and about key escrow systems to help the intelligence community—and these ideas are resurfacing. I can see the back door idea coming back, but not key escrow. Admittedly, the situation is much more confusing today, with national security issues sometimes tangled up with entertainment industry antipiracy crusades.

I had fun reading, for this article, a 1994 National Security Agency report (declassified in 2012) about the NSA’s early activities.8 The writer exults that almost none of the Eurocrypt 1992 papers were of any interest to the NSA efforts because the outside researchers seemed to have moved to theoretical topics, away from cryptanalysis or cryptosystem design. To me, it’s strange for the NSA staff to be so dismissive of a thriving field of work, but it seems they were hoping that their ciphers wouldn’t

be at risk from outside efforts. And, in the spirit of looking for old ideas, that document reprints the recommendations for improving security that were made after the investigation of the Pearl Harbor attack. Many of these recommendations for improving security via better information exchange, more awareness of the need for alertness and proper management, and being aware of new risks would still be valid today.

At times, once we’ve solved a problem, we might want to look for other applications for the solution.

For example, some early cybercrime articles discuss phone phreaking (the manipulation of the telephone system using tones that open a path to the switch controls to avoid being charged for calls and frustrating attempts to track calls—both of importance to bookies), which has been solved by moving telephone calls’ metadata outside the voice path accessible to the sender. Why can’t we do the same with email? It’s been proposed many times, but its problems are administrative rather than technical, such as determining who would write its rules or validate the email systems. But I’m getting very tired of the daily phishing emails claiming to be from my Rutgers email service but easily observed to be from Africa or Asia.

Proving systems correct and building sandboxes were also popular topics at one time. Should these be resurrected? Provably correct code would be of use for not only security against attacks but also reliability in the face of accidents.

Sandboxes could minimize the impact of flaws or break-ins. What about micropayments? Denial-ofservice attacks rely on flooding a computer with a vast number of messages; if the bad guys had to pay for the messages, the attacks would be much harder to perform.

At one time, attackers engaged in war dialing—calling random numbers hoping to find modems July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

attached to unprotected dial-in systems. That isn’t an issue today, but random probing of computers on Wi-Fi networks is. There are now appropriate recommendations for dealing with this threat, but attackers have been placing receivers on roadsides to grab information from the electronic systems of passing cars. For example, Ishtiaq Rouf and his colleagues demonstrated how to use the RF signals from tire pressure monitoring to either track cars or falsely turn on a dashboard warning light.9 Defenses to such attacks will be similar to what we already know.

Rouf and his colleagues suggest several: software checking, encryption, and packet sequencing.

A common theme decades ago, and still important, is training: Do we educate enough people in cybersecurity defense? Similarly, management issues were important then and are now. As organizations decide to share information, they might find that their systems’ least-defended part offers access to everything. Before 9/11, the US government partitioned the information on terrorist threats among its intelligence agencies; criticism of that approach led to greater information sharing, which then enabled a relatively low-level employee (Bradley Manning, now Chelsea Manning) to have access to, and betray, a wide range of information. Edward Snowden, by contrast, seems to have gained wider access via “social engineering” (that is, schmoozing); see previous sentences about management and training issues.

How might we enjoy the advantages of big data with fewer risks?

Research in this area includes performing distributed database queries using limited access to each part of the database (viewing the data only through a specified kind of query—a sort of narrow hole into the data). Topics such as performing queries on encrypted data are regaining popularity.

A higher-level question is whether people would trade features for security. How many people using the Adobe PDF viewer know that it includes a 3D viewer?

Would they give up this program for a simpler one with fewer vulnerabilities? In this case, users have a choice, because there are other PDF viewers. Operating system users’ options are scarce. In addition, all users are at a disadvantage because evaluating security is difficult and there are no published ratings for the security of systems.

A

s with other ideas, new thoughts on improving security might require an appropriate technical environment, effective productization, suitable marketing, and good luck but might face the obstacle of confidentiality. Security suggestions are often made and tested in secret, and data regarding their effectiveness is often hard to come by. This is understandable; we don’t want to publicize techniques to help hackers. But it also means that ideas aren’t explored by a wider community and might be overlooked. Digital time stamping as a way to certify transaction dates, although not kept secret, was given a second life thanks to Bitcoin. Solutions to other current security problems might be hiding in old technologies; let’s hope we can find some.

References 1. S. Ember, “Data Security Is Becoming the Sparkle in Bitcoin,” New York Times, 2 Mar. 2015, p. B1.

2. S. Haber and W.S. Stornetta, “How to Time-Stamp a Digital Document,” Advances in Cryptology, A.J. Menezes and S.A. Vanstone, eds., LNCS 537, Springer, 1991, pp. 437–455.

3. J. Markoff, “Technology: Experimenting with an Unbreachable Electronic Cipher,” New York Times, 12 Jan. 1992.

4. New York Times, 16 Feb. 1992, p. L49.

5. S. Nakamoto, Bitcoin: A Peer-to-Peer Electronic Cash System, white paper, Bitcoin, 2008.

6. W. Weaver, Memorandum, Rocke___ feller Foundation, 1949; www .mt-archive.info/Weaver-1949.pdf. ___________________ 7. V. Bush, “As We May Think,” Atlantic Monthly, vol. 176, no. 7, 1945, pp. 101–108.

8. Cryptolog, National Security Agency, vol. 20, no. 1, 1994; ____ https:// www.nsa.gov/public_info/_files /cryptologs/cryptolog_126.pdf. __________________ 9. I. Rouf et al., “Security and Privacy Vulnerabilities of In-Car Wireless Networks: A Tire Pressure Monitoring System Case Study,” Proc.

19th USENIX Security Symp., 2010.

Michael Lesk is a professor at Rut-

gers University. Contact him at lesk@acm.org.

_________

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

GET MORE, FOR LESS.

Digital magazine subscriptions now available.

computer.org/subscribe 79

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

BUILDING SECURITY IN Editor: Jonathan Margulies, ____________ jonathan@qmulos.com

Garage Door Openers: An Internet of Things Case Study Jonathan Margulies | Qmulos

E

arly last year, my garage door opener’s motor died. While researching potential replacement units, I focused on Chamberlain’s products because they had a reputation for high quality. Once I settled on a model, I noticed another option: for a little more money, Chamberlain would include the MyQ Internet Gateway, its new system for monitoring and controlling the opener via the Internet. Curiosity got the best of me, so I went for it.

After installing the opener, the MyQ languished in my closet for months. I loved the idea of getting an alert if I left my garage door open— I can’t count the number of times I’ve turned around five minutes after leaving the house to double-check that it was closed—but I felt sure there would be a security flaw in the MyQ that would make me worse off. The emergence of the Internet of Things (IoT) has turned trusted, long-standing companies into unwitting network attack vectors.1 But the MyQ is different from— and more impactful than—other IoT devices: it controls access to my house. That got me thinking: What if I wanted to solve this problem from the ground up? How would I design an Internet-connected garage door opener (“IoT opener”) to be adequately secure? Is it possible?

The Ground Rules First, I define “adequately secure” to mean no less secure than the traditional rolling-code garage door 80

July/August 2015

SECURITY& PRIVACY

IEEE

Copublished by the IEEE Computer and Reliability Societies

opener. Because that’s the system the MyQ is replacing, it seems like the right standard.

Second, due to space constraints, I focus only on design. The usual implementation caveats—for example, the need for well-written code and correct use of encryption libraries—still apply, but I won’t address them in detail.

Third, I assume the same basic set of features the MyQ offers: a user can open or close a garage door via the Internet from a smartphone or computer and receive emails or push notifications when the door’s status changes.

Standard Garage Door Openers If the standard is the security of rolling-code openers, we first need to understand how those openers work.

The most popular rollingcode implementation is a product called KeeLoq, a lightweight block cipher that generates codes based on a cryptographic key and a counter (www.webcitation .org/6ZZYZpH2n). When a user syncs a remote control with a garage door opener, the remote control begins to generate the same codes, in the same order, as that opener.

Thereafter, when a user pushes the “open” button, the remote control increments its counter, generates a new code, and broadcasts that code wirelessly. When the opener receives a code, it checks the code against the next 256 codes in its 1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

queue. (Checking against so many are those intruders? Why aren’t they the opener on its own private virpossible codes helps ensure that the taking advantage of the universally tual LAN and having my wife use a remote control and the opener don’t weak security of modern suburban virtual private network to reach it, lose sync when a user presses the homes? As podcaster Roman Mars because who would trust a garage button outside the opener’s receiv- eloquently observed, “locks have door opener company with network ing range.) If the code is a match, become a social construct as much access control? But I was naive.

The IoT industry has clearly the opener increments its counter as they are a mechanical construct” to just above the matching code (http://99percentinvisible.org decided that having a central serGarage vice act as a clearinghouse for and opens the door. In addition to /episode/perfect-security). ________________ using remote controls, some users door openers only need to be secure authentication, authorization, and mount keypads in front of their enough to let passersby know we commands is a must, and it’s easy to see why: it frees them from havgarages that similarly sync with the don’t want them to come in. ing to worry about configuring openers; these keypads broadcast a home routers, setting up dynamic code when a user correctly enters a Openers and the DNS for when customers’ IP numeric password.

Internet of Things The simplest way for attackers Exposing garage door openers to the addresses change, or having access to open a rolling-code garage door Internet might make them such easy to all of a user’s relevant data when opener is to sync it to a new remote targets as to pose a real risk. What if those inevitable tech support calls control. Replacement remote con- an attacker could indiscriminately come in. The problem is that the trols are available at just about any send open commands to any opener? cloud service opens another attack surface, and a big one: hardware store, and syncinstead of having to hack ing them requires only a a single IoT opener at a few minutes alone in the Exposing garage door openers to time, attackers can try garage. A similarly easy the Internet might make them such to hack them all through option is to go after the the cloud service. It’s a keypad by spying on the easy targets as to pose a real risk. single point of failure for user, or deduce or bruteauthentication, integrity, force the code. A third and availability. Indeed, option is a physical attack.

Most openers include an emergency What if every time an email account MyQ experienced an unplanned release rope just inside the door. If is hacked, the hacker is given a clear four-hour disruption in late April an attacker can slip a wire hanger path to find the user’s home address that affected all users, and I doubt it above the door and latch onto that and credentials to open that user’s will be the last. rope, a skilled tug can unlock the garage? What if an attacker managed door. The final option for attacking to download a whole database of user Cloud Service traditional openers is to go after the credentials for IoT openers? Any of Authentication rolling-code mechanism itself. Over these possibilities would make home Using a cloud service as a central the past decade, several researchers intrusions so easy as to be inevitable. hub isn’t the security decision I have developed methods to derive a We’re starting to see this evolution would have made, but it seems like KeeLoq key given access to a work- with cars that use Bluetooth keys; it’s such a foregone conclusion that ing, synced remote control.2–4 A become so easy and cheap to break I’ll treat it as an assumption for the simpler but less effective approach into some of them that insurers have remainder of this article. Authentiis to sniff a code over the air from started demanding additional secu- cation at both the cloud service and a remote control by pushing the rity measures.5 the opener is of paramount concern. “open” button outside the opener’s But before I can delve deeper The obvious way for the cloud serrange, and then using that code into how those types of attacks vice to authenticate to the opener is before the owner comes home (at might happen on IoT openers, I with a certificate, but what happens which point, that code will expire). must first address a key architec- when attackers compromise the All these attacks require close tural question: Will the opener private key behind that certificate? proximity to either the garage or the authenticate and authorize the user, This could allow attackers to send remote control and are sufficiently or will a cloud service do so on the arbitrary commands to any opener, difficult that virtually all intruders opener’s behalf? When I bought the or something much worse: if the prefer to break a window, force a MyQ, I hoped it would be the for- certificate can be used to force overdoor open, or pick a lock. But where mer, as I had grand plans for putting the-air software updates, attackers 81

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

BUILDING SECURITY IN

Chamberlain Statement The following statement was sent to the author on 12 June 2015: Chamberlain built its MyQ technology—as we have all of our products for more than 40 years—on a foundation of safety and security. We have an aggressive product roadmap that includes continuous security updates, including the feature recommendations noted in the IEEE [Security & Privacy] story.

Specifically, Chamberlain will upgrade password requirements in the second half of this year [2015], and is looking at the best ways to implement two-factor authentication based on our user needs and their usage scenarios. We also plan to introduce multi-user access so that account administration details and log-in credentials are secure to only one account owner, while allowing garage door access to other users. This will include assigning different levels of permissions based on user access level, times of day, days of week, etc.

In addition to the specific updates noted above, we combine our own team’s expertise in security technologies with reputable thirdparty security firms to audit our systems on an ongoing basis. Our continuous security updates and processes include using industry standard encryption, applying the latest security techniques, and periodic security testing with respected outside services. We also recommend IoT technology leaders continuously advise their customers on how to maximize the security of their home Wi-Fi network, which are critical gateways to device security for consumers. Chamberlain’s brand and reputation are built on a heritage of delivering safe and secure products to consumers; we take the safety and security of the smart home very seriously.

could gain control of the whole system. For this reason, software updates should be user initiated and openers should regularly check for certificate revocation.

Authentication at the cloud service is more complicated. Perhaps the worst-case scenario is when attackers download the password database, as in the famous attacks on Sony’s PlayStation Network (www.cnet.com/news/playstation -______________________ net work- sti l l-of f line-af ter -suspected-attack) and LinkedIn ___________ (http://money.cnn.com/2012/06 /06/technology/linkedin-password ______________________ -hack). ___ If the passwords aren’t sufficiently complex to stymie bruteforce attacks or aren’t encoded by an adequate key derivation function (such as bcrypt), user account takeover becomes trivial.

What about password reset, the issue behind attacks on countless celebrities’ email and Apple iCloud accounts? Google recently published research suggesting that security questions are insufficient to protect accounts.6 As of this writing, MyQ uses email for 82

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

password reset, which, in this case, seems like a terrible idea: any time attackers hijack an email account, they can search for emails containing the term “MyQ” to determine whether the user has a MyQ account, and then search for shipping information to determine the user’s home address. Ironically, just about the only personal information the MyQ website asked for was my home address, which is the information they should least want to have on file. This combination of information allows attackers to build databases of locations of vulnerable openers.

An opener is actually an interesting case from a password reset perspective in that it has an unusual security feature: it never moves.

That means a user can’t lose it, and it would therefore be reasonable for Chamberlain to require users to electronically prove they have possession of it (for example, the opener could display a code that rolls every few seconds). Such a feature would also help new homebuyers prove transfer of opener ownership.

Potential Security Improvements The other side of this discussion is the security improvements networked openers might offer. One improvement is two-factor authentication (2FA). Many of the problems outlined in this article can be mitigated by 2FA and, in a system that’s already so reliant on smartphones, users are already carrying the obvious second factor in their pockets.

A second improvement is policybased access control. This can be useful in several ways: ■ allowing multiple user accounts to control the door, but only one to administer it; ■ allowing administration from specific devices only; ■ restricting certain accounts (such as caregiver or contractor) to operating the door only during business hours; and ■ creating time-limited guest accounts.

These policies could be easy to administer through a Web interface.

July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

A third improvement is a more granular alerting system. I’d be much more interested in knowing that a new user or remote control was given access to my opener, or that the door opened in the middle of the night, than in knowing that my wife opened the door at 5 pm.

C

M q M q

M q

MqM q

onvenience will continue to drive companies that lack information security expertise to build IoT devices, and consumers to buy them. Ideally, this new IoT world would be built on a few competing platforms by people who understand and can address the security risks—Apple, Google, and Facebook have all recently launched the beginnings of IoT platforms—rather than inexperienced companies rolling their own authentication, authorization, and communication code. Although the risks and functions change from one IoT device to another, they all need the same basic security infrastructure: a way for users to authenticate, two-way mapping between users and devices,

access policy creation and enforcement, logging and alerting capability, and secure communication.

These problems are largely solved; they just need to be made easy for IoT developers to use. When that happens, and the right security features are in place, I think IoT openers could become difficult enough to attack that no one would bother with them, just like the openers most of us have today.

In the meantime, my MyQ is going back in the closet.

3.

4.

5.

Author’s Note I contacted Chamberlain after writing this article, and a representative responded with an outline of plans for addressing some of the concerns in this article. See the sidebar for the response.

6.

References 1. J. Fontana, “Belkin Patches Vulnerabilities in WeMo Devices,” ZDNet, 19 Feb. 2014; www.zdnet.com/article /belkin-patches-vulnerabilities-in ____________________ ________ -wemo-devices.

2. S. Indesteege et al., “A Practical Attack on KeeLoq,” Proc. 27th Ann. Int’l Conf. Theory and Applications of

Cryptographic Techniques (EUROCRYPT 08), 2008, pp. 1–18.

A. Bogdanov, “Cryptanalysis of the KeeLoq Block Cipher,” Cryptology ePrint Archive, report 2007/55; htt ps://epr int.iacr.org/2007 /055.pdf. _____ I. Sheetrit and A. Wool, “Cryptanalysis of KeeLoq Code-Hopping Using a Single FPGA,” Cryptology ePrint Archive, report 2011/242; ____ https:// eprint.iacr.org/2011/242.pdf.

H. Osborne, “Thieves Target Luxury Range Rovers with Keyless Locking Systems,” Guardian, 27 Oct. 2014; www.theguardian.com/money /2014/oct/27/thieves-range-rover ____________________ -keyless-locking. _________ J. Bonneau et al., “Secrets, Lies, and Account Recovery: Lessons from the Use of Personal Knowledge Questions at Google,” Google Research Archive, report 43783; http://research.google.com/pubs /pub43783.html. _________

Jonathan Margulies is the chief tech-

nology officer at Qmulos. Contact him at ___________ jonathan@qmulos .com ___ or follow him on Twitter @unsaltedhash.

IEEE Pervasive Computing explores the many facets of pervasive and ubiquitous computing with research articles, case studies, product reviews, conference reports, departments covering wearable and mobile technologies, and much more.

Keep abreast of rapid technology change by subscribing today!

www.computer.org/pervasive 83

www.computer.org/security

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

IN OUR ORBIT Editors: Angela Sasse, __________ a.sasse@cs.ucl.ac.uk | Alessandro Acquisti, _____________ acquisti@andrew.cmu.edu

Mass-Marketing Fraud: A Growing Concern Monica T. Whitty | University of Leicester

A

growing international concern for end users is massmarketing fraud, which exploits mass communication techniques (such as email, instant messenging, bulk mailing, and social networking sites) to target victims for financial profit. The Internet has opened the floodgates to fraud, given that criminals can use it to target many more potential victims. Mass-marketing fraud is a serious, complex, and often organized crime. Examples include foreign lotteries and sweepstakes fraud, in which victims believe they have won money and are told to pay a fee to receive the funds; 419 scams or advance fee fraud, in which victims believe that for a small amount

84

July/August 2015

SECURITY& PRIVACY

IEEE

of money they will make a large fortune; and romance scams, in which victims send money to someone posing as a romantic interest on an online dating site.

In 2010, the International MassMarketing Fraud Working Group reported that this type of crime has gradually transformed from a predominantly North American problem into a pervasive global criminal threat.1 Since 2012, in the UK, approximately 2.6 million adults have fallen for these scams, with about 500,000 adults falling victim to a dating or romance scam; around 900,000 being conned by a boiler room scam (where victims are pressured into buying fake, Copublished by the IEEE Computer and Reliability Societies

overpriced, or worthless stocks); 700,000 by a charity scam; 900,000 by a “need funds for an emergency” scam; 700,000 by an inheritance scam; and 800,000 by a lottery scam. In 2012 alone, some 800,000 UK adults were victims of this kind of fraud. Of further concern is that nearly a quarter of victims are scammed at least twice.2 Reporting bodies in the UK, such as Action Fraud, estimate that less than 10 percent of the population actually report this type of crime, partly because of the shame and embarrassment associated with becoming a victim, lack of knowledge of where and how to report the crime, and the awareness that it’s unlikely that criminals will be located or prosecuted. In the US, the Internet Crime Complaint Center (IC3) reported that in 2014, 123,684 victims reported financial losses as a result of Internet crimes, many of which were mass-marketing fraud. The most common complaints included auto fraud, government impersonation, intimidation or extortion, real estate scams, and romance scams.

It’s estimated that only 10 percent of victims report the crime to IC3.3 In Australia, just over AUS$94 million was reported lost by victims of mass-marketing fraud in 2012.4

Costs of Cybercrime Researchers have yet to accurately measure the amount lost to individuals. As Dinei Florencio and 1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

Cormac Herley point out, we need to take a cautious approach to survey results about losses, because outliers can distort the results.5 For example, “a single individual who claims [US]$50,000 in losses, in a 1,000 person survey, is all it takes to generate a $10 billion loss over the population. One unverified claim of $7,500 in phishing losses translates into $1.5 billion.”5 Ross Anderson and his colleagues argue that our response to cybercrime is so disproportionate that we overspend protecting against it.6 They note that “traditional offences such as tax and welfare fraud cost the typical citizen in the low hundreds of pounds/Euros/ dollars a year; transitional frauds cost a few pounds/Euros/dollars; while the new computer crimes cost in the tens of pence/cents.” However, indirect and defense costs “are much higher for transitional and new crimes. For the former, they may be roughly comparable to what the criminals earn, while for the latter they may be an order of magnitude more.” These points are important when considering the costs of cybercrime; however, in addition to financial losses, we must consider the costs to the psychological well-being of victims as well as family members and friends who are often impacted by mass-marketing fraud.7

Psychological Impact The psychological effects of massmarketing fraud can sometimes outweigh the financial impact, even when large sums of money are lost.8,9 The Office of Fair Trading reported that “in contrast to other types of crime, scams are a ‘silent’ crime for which victims may receive little support. Because there is usually not a visible perpetrator, it can also be difficult for victims to get closure.”10 Psychological harm can include shame, guilt, embarrassment, depression, suicidal feelings, grief, anxiety, and loss of trust.11

Unlike most other crimes, massmarketing fraud victims are often blamed for their victimization. Family and friends, who are typically an important source of support for victims, often blame the victims, pointing out how they too have been affected (such as losing their inheritance). Victims are often viewed by society and depicted in the media as stupid or naive. This makes recovery from this crime even more difficult, especially given that victims aren’t likely to have their funds recovered.

Furthermore, preliminary findings suggest that aftercare for victims needs to be long term, which can be too expensive for victims.8,11 Websites and online support groups such as www.romancescams .org __ and www.scamsurvivors.com have emerged where victims can support one another, although the effectiveness of this type of support is yet to be empirically tested.

Vulnerability and Persuasive Techniques Research on the types of people most likely to be victims of scams has mostly focused on “get rich quick” schemes.12 Nonetheless, these studies provide some important insights into the sorts of individuals who are more vulnerable to mass-marketing fraud. Steve Furnell argued that greedy and naive individuals are more likely to be conned.13 Jinkook Lee and Horacio Soberon-Ferrer found that victims of fraud tend to be older, poorer, less educated, and single.12 Kristy Holtfreter and her colleagues looked broadly at consumer fraud victimization and found that victims were more likely to have low self-control.14 In contrast, it’s been found that those who are considered sensation seekers aren’t more likely to be scammed by the online dating romance scam; rather, for those who score high on the Romantic Belief Scale, idealization is more associated with the likelihood of being a victim.7,11

In explaining the success of mass-marketing fraud, scholars mostly draw from social psychological theories and apply them to these scams. An Office of Fair Trading report offers the most comprehensive explanation of the psychology of scams.15 The report argues that “falling for a scam comes down to errors in decision-making,” and that “scammers create situations (with their scam offers) that increase the likelihood of poor decision-making.”

The authors found that both cognitive (overconfidence in a specific topic) and motivational (the scam triggers positive emotions) processes explained the psychological reasons for responding to scams. Across their studies, the most consistent reasons people were drawn into scams were “appeals to trust and authority” (the use of people or institutions of authority to make the scam appear legitimate) and “visceral triggers” (triggers employed to make potential victims focus on huge prizes and imagined positive future emotional states).

However, other persuasive techniques are worth considering. For example, scarcity has been found to play a role; that is, presenting offers that have time limits.9 Classic marketing strategies such as the foot-in-the-door technique (asking for small amounts of money first and gradually increasing the requests as the person continues to comply) and the door-in-theface technique (asking for large amounts of money and gradually decreasing the amount until the person is willing to comply) have also been found effective.9 Frank Stanjano and Paul Wilson analyzed face-to-face cons and found that attackers use a limited number of techniques to manipulate their victims.16 In addition to classic persuasive techniques, I would argue that it’s 85

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

IN OUR ORBIT

equally important to focus on the role the Internet plays in persuading individuals to part with their money. In the past, researchers have found that individuals develop very trusting, intimate relationships online.17 Given this, we might postulate that the Internet is the ideal place for criminals to target strangers to earn their trust.

I’ve found that one-sided, hyperpersonal relationships often develop between victims and scammers.9 Victims idealize the fake persona, believing they have found the perfect romantic partner. Given that much of the communication is asynchronous, criminals have time to develop the illusion of the ideal person, making it easier to trick victims into falling in love with them.

Joseph Walther also found that when people are communicating via the Internet, they become much more focused on their online communications, blocking out other environments.17 I’ve found that victims of the romance scam report continuously re-reading emails and looking forward to the next email or chat session. When communicating online, a record is kept for individuals to revisit whenever they want, reinforcing the romantic messages being sent by criminals.

Therefore, I believe online communication helps strengthen the perceived emotional bond, making it difficult for victims to escape the relationship. The same idea could apply to other types of scams in which criminals spend time communicating with victims before asking them for money.

A Stage Theory Approach Thinking of scams as a series of stages is a useful method of gaining greater insight into their anatomy.

I’ve outlined six stages involved in the romance scam, which could potentially be applied to other types of mass-marketing fraud: 86

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

■ stage 1—potential victims need to be motivated to find the “ideal” partner; ■ stage 2—potential victims are presented with the profile of an ideal partner and promised an exclusive relationship; ■ stage 3—potential victims are groomed to trust and love the criminal, who then gauges whether potential victims are ready and willing to part with their money; ■ stage 4—the criminal employs techniques to persuade potential victims to send money (such as a narrative about a crisis where money is urgently needed or the foot-in-the-door technique); ■ stage 5—the criminal employs further techniques to keep the scam alive, such as inventing another crisis or employing the door-in-the-face technique; and ■ stage 6—victims might believe the scam is over and are subsequently revictimized (the criminal admits to the scam but claims to be in love and then asks for more money, or victims receive an email from the criminal pretending to be law enforcement asking for a small amount of money to return their funds).9

Prevention Preventing mass-marketing fraud is a difficult task. Catching and prosecuting attackers is challenging given that the criminals often live in a different country than the victims; the methods the criminals use make them difficult to trace; and prosecution is very time consuming, owing to the large amounts of online data that needs to be analyzed to establish evidence and gain intelligence on the criminals’ whereabouts and operating tactics.

Other methods are needed to help prevent these crimes.

Law enforcement and others have attempted to prevent these crimes by using disruption tactics.

Users of dating sites, for instance, have been encouraged to report known fake profiles to help reduce the number of criminal profiles on these sites. Law enforcement has also tried to influence a change in the way money is transferred via money transfer companies, such as Western Union and MoneyGram, so that users are traceable. Doing so would make catching criminals easier and could deter other criminals. Moreover, these transfer companies have been encouraged to enter suspicious activity reports for any transaction that appears to have been conducted by a scammer. However, even if such strategies were successful, criminals could potentially find a loophole (for example, persuading victims to unknowingly transfer money into other victims’ accounts, making the money more difficult to trace back to criminals as well as involving victims in money laundering).

Unfortunately, awareness of mass-marketing fraud doesn’t necessarily prevent individuals from becoming victims. Researchers have found that many victims who fall for mass-marketing frauds have heard of these scams, and some even have detailed knowledge of them.11,15 The 2009 Office of Fair Trading report argues that detailed knowledge of a scam can, in fact, increase vulnerability because these individuals often develop an “illusion of invulnerability.”15 Moreover, once victims are hooked into a scam, it’s very difficult to make them believe it was a hoax. I’ve found that even when authority figures (such as police, law enforcement, and bank managers) alert people to the fact that they are victims of a romance scam, victims often have difficulty believing it. Moreover, when victims question criminals about their authenticity, criminals will employ persuasive techniques to convince victims otherwise.18 Given that July/August 2015

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

THE WORLD’S NEWSSTAND®

knowledge about a scam might not be enough to prevent individuals from becoming defrauded, other types of interventions are needed.

Stajano and Wilson contend that because attackers consistently use a small number of well-established tricks, system designers could do more to prevent exploitation: “Our message for the system-security architect is that it is naive to lay blame on users and whine, ‘The system I designed would be secure, if only users were less gullible.’ The wise security designer seeking a robust solution will acknowledge the existence of these vulnerabilities as an unavoidable consequence of human nature and actively build safeguards that prevent their exploitation.”16

A

s cybercriminals develop and refine their techniques, a greater proportion of the population will likely be susceptible to mass-marketing fraud. Research to date has provided some detail as to the anatomy of these scams and the techniques criminals use to persuade victims to part with their money. New methods of tracing criminals are needed, as is cooperation across international boundaries. Among the technical solutions advocated by Stajano and Wilson is that computer scientists should develop programs to help law enforcement trace criminals online (akin to technologies developed to trace pedophiles) or software that detects fake profiles and deceptive communication and alerts users in real time.16 In addition, new methods are needed to help prevent victims from sending money. In Germany, for instance, automatic detection techniques and bank and post office staff training has dramatically reduced the number of successful 419 attacks. Many of these scams require mules to monetize the proceeds of attacks, so detection

and increasing awareness of typical mule recruitment techniques might be fruitful. Moreover, because an increasing number of scams are conducted via mixed channels—online and telephone—and involve impersonation of bank or law enforcement staff, we urgently need reliable ways to signal users when they are talking to a legitimate person, rather than an imposter.

References 1. “Mass-Marketing Fraud: A Threat Assessment,” Int’l Mass-Marketing Fraud Working Group, 2010; www ___ .ice.gov/doclib/cornerstone/pdf /immfta.pdf. _______ 2. M.T. Whitty, Mass-Marketing Fraud in the UK, report, 2013.

3. “2014 Internet Crime Report,” ann. report, Internet Crime Complaint Center, 2014.

4. “Targeting Scams: Report of the ACCC on Scam Activity 2012,” Australian Competition & Consumer Commission, 2013; www ___ .accc.gov.au/system/files/Targeting %20scams%202012.pdf. _____________ 5. D. Florencio and C. Herley, “Sex, Lies and Cyber-Crime Surveys,” Economics of Information Security and Privacy III, B. Schneier, ed., Springer, 2013, pp. 35–53.

6. R. Anderson et al., “Measuring the Cost of Cybercrime,” Economics of Information Security and Privacy, R. Böhme, ed., Springer, 2013, pp. 265–300.

7. M.T. Whitty and T. Buchanan, “The Online Dating Romance Scam: The Psychological Impact on Victims— Both Financial and Non-financial,” to be published in Criminology Criminal Justice, Sage Publications, 2015.

8. M. Button, C. Lewis, and J. Tapley, “Not a Victimless Crime: The Impact of Fraud on Individual Victims and Their Families,” Security J., vol. 27, no. 1, 2014, pp. 36–54.

9. M.T. Whitty, “The Scammers Persuasive Techniques Model: Development of a Stage Model to Explain the Online Dating Romance Scam,”

10.

11.

12.

13.

14.

15.

16.

17.

18.

British J. Criminology, vol. 53, no. 4, 2013, pp. 665–684.

“Helping People Affected by Scams: A Toolkit for Practitioners,” Office of Fair Trading, 2010.

M.T. Whitty and T. Buchanan, “The Psychology of the Online Dating Romance Scam Report,” Univ. of Leicester, 2012; www2.le.ac.uk ________ /depar tments/media/people ____________________ /monica-whitty/Whitty_romance ____________________ _scam_report.pdf. __________ J. Lee and H. Soberon-Ferrer, “Consumer Vulnerability to Fraud: Influencing Factors,” J. Consumer Affairs, vol. 31, no. 1, 2005, pp. 70–89.

S. Furnell, “Internet Threats to EndUsers: Hunting Easy Prey,” Network Security, vol. 2005, no. 7, 2005, pp. 5–9.

K. Holtfreter, M.D. Reisig, and T.C. Pratt, “Low Self-Control, Routine Activities and Fraud Victimization,” Criminology, vol. 46, no. 1, 2008, pp. 189–220.

“Provoking and Committing Errors of Judgment,” Office of Fair Trading, 2009; http://fraudresearchcenter .org/2012/03/the-psychology-of -scams-provoking-and-committing ___________________ -errors-of-judgment. ___________ F. Stajano and P. Wilson, “Understanding Scam Victims: Seven Principles for Systems Security,” Comm.

ACM, vol. 54, no. 3, 2011, pp. 70–75.

J.B. Walther, “Computer-Mediated Communication: Impersonal, Interpersonal and Hyperpersonal Interaction,” Comm. Research, vol. 23, no.

1, 1996, pp. 3–43.

M.T. Whitty. “True Romance?,” PoliceProfessional.com, 19 July 2012; www.policeprofessional.com /news.aspx?id=14829. ____________

Monica T. Whitty is a cyberpsycho-

logist and professor of media and communications at the University of Leicester. Contact her at mw229@le.ac.uk. ___________ Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

87

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

LAST WORD

The Right to Be Unobserved

T

here are two ways to define privacy, and neither involves the squishiness that begins “a reasonable expectation of ... .”

The first is what privacy means as a civil construct—what associate justice Louis Brandeis described as “the right to be left alone—the most comprehensive of rights, and the right most valued by civilized men.”1 The second is what privacy means at its operational core: the effective capacity to misrepresent yourself with de minimus side effects.

According to the first definition, privacy is something that society—meaning you— gives the individual—meaning me. When privacy isn’t given and is thus not available, secrecy is something I can take for myself; secrecy is a functional backstop for the absence of the civil construct.

If privacy is a gift and secrecy is something that is taken, then the possibility of privacy is inversely proportional to the numbers of those who must do the giving for the state of privacy to prevail; hence, privacy is inversely proportional to interconnectedness. This is consistent with a view of risk as proportional to dependency in which dependency, in turn, is proportional to nonoptional interconnectedness. This is where the all-wired world’s “information wants to be free” is most robustly antiprivacy.

The second definition—“privacy is the power to selectively reveal oneself to the world”2—means that in choosing what to reveal, however idiosyncratically, we demonstrate our liberty. As if that weren’t enough, “philosophical and legal analysis has identified privacy as a precondition for the development of a coherent self,”3 which asks the question of whether a person whose life has been without privacy can even comprehend the desire of those who prefer privacy. Raising the young to not expect privacy foreordains that when it’s their turn to run society, they will be just as happy despite privacy’s absence and will legislate accordingly.

If I reject the transparent society4 because I acknowledge my inability to sufficiently police its stronger members, then the most important thing I can do is to protect my privacy.

Daniel E. Geer Jr. In-Q-Tel

88

July/August 2015

SECURITY& PRIVACY

IEEE

The loss of privacy is irreversible, for information is never unrevealed. Privacy is therefore the paragon of David Hume’s conjecture: “It is seldom that liberty of any kind is lost all at once.”5 In the face of the snowballing bigness of the institutions of globalized human life, we must reserve privacy rights explicitly so that we can misrepresent ourselves to those against whom we have no other defense, against those for whom our name is a label on data collected without our consent.

This has nothing to do with whether you have anything to hide.6 If for no other reason, we must make it understood that just as “there is nothing sinister in so arranging one’s affairs as to [minimize] taxes,”7 neither is there anything sinister in arranging one’s affairs as to minimize observability.

References 1. Olmstead v. United States, US Reports, vol.

277, 1928, p. 438.

2. E. Hughes, “A Cypherpunk’s Manifesto,” Mar. 1993; www.activism.net/cypherpunk /manifesto.html. _________ 3. P. Agre, “The Architecture of Identity,” Human–Computer Interaction Seminar, 1998; http://hci.stanford.edu/courses/cs547 /speaker.php?date=1998-05-01. __________________ 4. D. Brin, The Transparent Society: Will Technology Force Us to Choose between Privacy and Freedom?, Perseus Books, 1998.

5. D. Hume, Essays: Moral, Political and Literary, 1742.

6. D.J. Solove, Nothing to Hide: The False Tradeoff between Privacy and Security, Yale Univ.

Press, 2011.

7. Commissioner of Internal Revenue v. Newman, Circuit Court of Appeals, 159 F.2d 848, 1947.

Daniel E. Geer Jr. is the chief information secu-

rity officer for In-Q-Tel and past president of the Usenix Association. Contact him at dan@geer.org. ________ Selected CS articles and columns are also available for free at http://ComputingNow. computer.org.

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

PURPOSE: The IEEE Computer Society is the world’s largest association of computing professionals and is the leading provider of technical information in the field.

MEMBERSHIP: Members receive the monthly magazine Computer, discounts, and opportunities to serve (all activities are led by volunteer members). Membership is open to all IEEE members, affiliate society members, and others interested in the computer field.

COMPUTER SOCIETY WEBSITE: www.computer.org Next Board Meeting: 15–16 November 2015, New Brunswick, NJ, USA

EXECUTIVE COMMITTEE President: Thomas M. Conte President-Elect: Roger U. Fujii; Past President: Dejan S. Milojicic; Secretary: Cecilia Metra; Treasurer, 2nd VP: David S. Ebert; 1st VP, Member & Geographic Activities: Elizabeth L. Burd; VP, Publications: Jean-Luc Gaudiot; VP, Professional & Educational Activities: Charlene (Chuck) Walrad; VP, Standards Activities: Don Wright; VP, Technical & Conference Activities: Phillip A. Laplante; 2015–2016 IEEE Director & Delegate Division VIII: John W. Walz; 2014–2015 IEEE Director & Delegate Division V: Susan K. (Kathy) Land; 2015 IEEE Director-Elect & Delegate Division V: Harold Javid

BOARD OF GOVERNORS Term Expiring 2015: Ann DeMarle, Cecilia Metra, Nita Patel, Diomidis Spinellis, Phillip A. Laplante, Jean-Luc Gaudiot, Stefano Zanero Term Expriring 2016: David A. Bader, Pierre Bourque, Dennis J. Frailey, Jill I. Gostin, Atsuhiro Goto, Rob Reilly, Christina M. Schober Term Expiring 2017: David Lomet, Ming C. Lin, Gregory T. Byrd, Alfredo Benso, Forrest Shull, Fabrizio Lombardi, Hausi A. Muller

EXECUTIVE STAFF Executive Director: Angela R. Burgess; Director, Governance & Associate Executive Director: Anne Marie Kelly; Director, Finance & Accounting: Sunny Hwang; Director, Information Technology Services: Ray Kahn; Director, Membership: Eric Berkowitz; Director, Products & Services: Evan M. Butterfield; Director, Sales & Marketing: Chris Jensen

COMPUTER SOCIETY OFFICES Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C. 20036-4928 Phone: +1 202 371 0101 • Fax: +1 202 728 9614 Email: hq.ofc@computer.org ___________ Los Alamitos: 10662 Los Vaqueros Circle, Los Alamitos, CA 90720 Phone: +1 714 821 8380 • Email: __________ help@computer.org Membership & Publication Orders Phone: +1 800 272 6657 • Fax: +1 714 821 4641 Email: __________ help@computer.org Asia/Pacific: Watanabe Building, 1-4-2 Minami-Aoyama, Minatoku, Tokyo 107-0062, Japan • Phone: +81 3 3408 3118 Fax: +81 3 3408 3553 • Email: tokyo.ofc@computer.org _____________

IEEE BOARD OF DIRECTORS President & CEO: Howard E. Michel; President-Elect: Barry L. Shoop; Past President: J. Roberto de Marca; Director & Secretary: Parviz Famouri; Director & Treasurer: Jerry Hudgins; Director & President, IEEE-USA: James A. Jefferies; Director & President, Standards Association: Bruce P. Kraemer; Director & VP, Educational Activities: Saurabh Sinha; Director & VP, Membership and Geographic Activities: Wai-Choong Wong; Director & VP, Publication Services and Products: Sheila Hemami; Director & VP, Technical Activities: Vincenzo Piuri; Director & Delegate Division V: Susan K. (Kathy) Land; Director & Delegate Division VIII: John W. Walz

Executive Committee (ExCom) Members: Christian Hansen, President; Dennis Hoffman, Jr. Past President; Jeffrey Voas, Sr. Past President; W. Eric Wong, VP Publications; Alfred Stevens, VP Meetings and Conferences; Marsha Abramo, VP Membership; Shiuhpyng Winston Shieh, VP Technical Activities; Scott Abrams, Secretary; Robert Loomis, Treasurer Administrative Committee (AdCom) Members: Marsha Abramo, Scott Abrams, Loretta Arellano, Lon Chase, Joseph Childs, Pierre Dersin, Lance Fiondella, Carole Graas, Lou Gullo, Christian Hansen, Dennis Hoffman, Samuel Keene, Pradeep Lall, Zhaojun (Steven) Li, Robert Loomis, Pradeep Ramuhalli, Rex Sallade, Shiuhpyng Shieh, Alfred Stevens, Jeffrey Voas, and W. Eric Wong

http://rs.ieee.org The IEEE Reliability Society (RS) is a technical society within the IEEE, which is the world’s leading professional association for the advancement of technology. The RS is engaged in the engineering disciplines of hardware, software, and human factors. Its focus on the broad aspects of reliability allows the RS to be seen as the IEEE Specialty Engineering organization. The IEEE Reliability Society is concerned with attaining and sustaining these design attributes throughout the total life cycle. The Reliability Society has the management, resources, and administrative and technical structures to develop and to provide technical information via publications, training, conferences, and technical library (IEEE Xplore) data to its members and the Specialty Engineering community. The IEEE Reliability Society has 23 chapters and members in 60 countries worldwide.

The Reliability Society is the IEEE professional society for Reliability Engineering, along with other Specialty Engineering disciplines. These disciplines are design engineering fields that apply scientific knowledge so that their specific attributes are designed into the system / product / device / process to assure that it will perform its intended function for the required duration within a given environment, including the ability to test and support it throughout its total life cycle. This is accomplished concurrently with other design disciplines by contributing to the planning and selection of the system architecture, design implementation, materials, processes, and components; followed by verifying the selections made by thorough analysis and test and then sustainment.

Visit the IEEE Reliability Society website as it is the gateway to the many resources that the RS makes available to its members and others interested in the broad aspects of Reliability and Specialty Engineering.

revised 5 June 2015

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

DIGITAL EDITION

SUBSCRIBE FOR $2995 Protect your network Further your knowledge with in-depth interviews with thought leaders Access the latest trends and peer-reviewed research anywhere, anytime

IEEE Security & Privacy is the publication of choice for great security ideas that you can put into practice immediately.

No vendor nonsense, just real science made practical.

—Gary McGraw, CTO, Cigital, and author of Software Security and Exploiting Software

www.qmags.com/SNP SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®