Cybersecurity for Financial Industry: An Analysis of the Cyber Resilience Assessment Framework

CYBERSECURITY FOR FINANCIAL INDUSTRY: AN ANALYSIS OF THE CYBER RESILIENCE ASSESSMENT FRAMEWORK

1

CONTENTS

CONTENTS

PREFACE

4

EXECUTIVE SUMMARY

5

1. THE CYBER RESILIENCE ASSESSMENT FRAMEWORK (C-RAF) Introduction Cybersecurity Assessment C-RAF and Other Cybersecurity Assessment Frameworks

13 16 18

2. INHERENT RISK ASSESSMENT (IRA) Overview Survey Result Survey Result – by size Indicator and Sub-domain Analysis Analysis by Service Provision Hierarchical Clustering Factor Analysis Inherent Risk Analysis: Summary

22 24 26 29 32 34 37 39

3. MATURITY ASSESSMENT (MA) Overview Survey Result Controls and Sub-Domain Analysis Responses IRA: Low Cybersecurity Staffing Risk for High-risk AIs Maturity Assessment: Summary

42 46 47 50 50

4. INHERENT RISK AND MATURITY Overview Domain Comparison Inherent Risk and Maturity Analysis Summary

53 56 60

5. C-RAF AND REAL-LIFE MEASURES SSL Certification Adoption Suboptimal Certification Adoption Time-series Analysis C-RAF and Real-life Measures: Summary

62 64 67 69

CONCLUSION

69

APPENDIX

70

2

This report documents the findings and insights from research conducted by HKUST Business School as part of the Fintech Theme-based Research Project, “Contributing to the Development of Hong Kong into a Global Fintech Hub” (Project No. T31-604/18-N), funded by the Research Grants Council (RGC).

Editorial: Fintech Research Project, HKUST Business School Artwork and Design: Andrew Tang @JamFactory Print production: Media Technology and Publishing Center, HKUST Enquiries: fintech@ust.hk

About HKUST Business School Founded in 1991, the HKUST Business School is young, innovative and committed to advancing global business knowledge. The School has forged an international reputation for world-class education programs and research performance, and has received many top global rankings.

It is one of the first Asian business schools accredited by both AACSB and EQUIS. The School strives to contribute to the economic and social advancement of the region by developing future leaders who possess an innovative and entrepreneurial spirit as well as a strong sense of responsibility. We also take active steps to promote knowledge advancement in many significant business areas. For more information, please visit www.bm.ust.hk.

About Fintech Research Project The Fintech Research Project is funded by the Research Grants Council (RGC) under the Theme-based Research Scheme 2018-19, titled “Contributing to the Development of Hong Kong into a Global Fintech Hub”. The project aims to provide a roadmap for transforming Hong Kong into a global fintech hub through the delivery of policy recommendations, scholarly contributions, and industrial impact. A team of researchers from HKUST and other universities with expertise spanning finance, information systems, statistics, computer science, accounting, and economics are tackling eight major research tasks that cover blockchain, cybersecurity, risk preference, robo-advising, artificial intelligence / machine learning, systemic risk, financial innovation policy, and manpower development. © February 2022 HKUST Business School, The Hong Kong University of Science and Technology All Rights Reserved.

3

PREFACE

PREFACE Cyberattacks have been on the rise globally. The COVID-19 pandemic saw many organizations across the world shifting to remote work, giving ways to new opportunities for cybercrime, along with other cybersecurity threats such as phishing and ransomware.

Financial institutions such as banks and credit agencies continue to be popular targets.

Many governments and organizations have proposed initiatives to strengthen cybersecurity such as filtering tools which limit access by malicious content. Their effectiveness, however, remains unclear. In 2016, the Hong Kong Monetary Authority (HKMA) launched a Cybersecurity Fortification Initiative (CFI), a key component of which is the Cyber Resilience Assessment Framework (C-RAF), where financial institutions (FIs) are required to assess its cybersecurity risk and determine the adequacy of its cybersecurity measures. While FIs should formulate necessary plans and strengthen their security to address any gaps, are their investments worth the cost?

To understand the effectiveness of the security measures undertaking by FIs under the C-RAF and to assess the impact of C-RAF adoption on security in the industry, we have conducted a comprehensive research over the course of 24 months. This report documents our findings on the detailed assessment and analysis of the C-RAF measures that would provide additional views on how these measures can expose the state of security development of financial institutions.

We hope the findings presented in this report provide useful insights for financial institutions, practitioners, and regulators on whether their current security measures, policies and regulations are sufficient and worthy of investments so that we can continue to develop good practices and cybersecurity protection schemes for the financial institutions in Hong Kong.

Kai-Lung Hui School of Business and Management The Hong Kong University of Science and Technology Wei Thoo Yue Department of Information Systems City University of Hong Kong

4

EXECUTIVE SUMMARY The Hong Kong Monetary Authority (HKMA) has developed the Cybersecurity Fortification Initiative (CFI) to strengthen the cyber resilience of authorized institutions (AIs) in Hong Kong.

One of the CFI’s components is the Cyber Resilience Assessment Framework (C-RAF). Between January 2019 and June 2021, we compiled a detailed list of Cyber Resilience Assessment Framework (C-RAF) measures, surveyed 22 AIs and collected public data on the participating banks for a comprehensive assessment using different models of analysis to study the effectiveness of the C-RAF and the banks’ cybersecurity performance relative to industry peers.

Through a survey conducted with the participating AIs, we assessed the two processes of the C-RAF – Inherent Risk Assessment (IRA) and Maturity Assessment (MA). In the IRA process, AIs performed self-assessment under five domains to reflect their cybersecurity threat level, which is mapped to their expected maturity level of cyber resilience. In the MA process, they assessed cybersecurity controls under seven domains to determine their maturity level.

I. KEY FINDINGS OF INHERENT RISK ASSESSMENT (IRA) The surveyed AIs selected the most appropriate inherent risk level (Low, Medium or High) for each risk indicator in the five domains of IRA: Technologies, Delivery Channels, Products and Technology Services, Organizational Characteristics and Tracked Records on Cyber Threats. An AI’s overall risk level is determined by the most common risk level. The inherent risk level is identified to be low for the majority of AIs (45%), implying they tend to face lower cybersecurity risk. On the contrary, 14% of the surveyed AIs are identified as high-risk AIs with the highest risk exposure among the three classes.

To provide a better understanding of the correlation of AIs’ risk profile with their business operations and underlying causes of their cyber risk, we conducted detailed analyses from various dimensions.

Inherent Risk Score We calculated a risk score for each AI by converting its survey responses on an ordinal scale to quantify the results.

Our observations: 1. The Technologies domain contributes to the biggest difference between high- and mediumrisk AIs. 2. The Products and Technology Services domain sees the largest gap between low- and medium-risk AIs. 3. As indicated by the Tracked Records on Cyber Threats domain, the inherent risk related to past cyberattacks is similar to AIs at all the risk levels.

5

EXECUTIVE SUMMARY

Inherent Risk Assessment - Risk Score Comparison by Domain Low

Medium

Technologies

High

Delivery Channels

Products and Technology Services

Organizational Characteristics

Tracked Records On Cyber Threats

Percentage Risk Score As the number of indicators (security controls) in each domain is not the same, we used a percentage risk score to provide a standardised view of risk propensity independent of the number of indicators.

Inherent Risk Assessment – Percentage Risk Score by Risk Class by Domain Tracked Records On Cyber Threats

High

Organizational Characteristics Products and Technology Services Delivery Channels

Medium

Technologies

Low

Our observations: 1. The Technologies domain remains to have the most noticeable gap between high- and medium-risk AIs. 2. The Delivery Channels domain appears to pose the highest inherent risk to both high- and medium-risk AIs. Each has about a 15 percentage point difference to its second riskiest domain.

3. Low-risk AIs do not appear to suffer from the risk posed by the Delivery Channels domain.

Organizational Characteristics is the most prominent risk for them.

6

Correlation of Risk with Equity of AIs We grouped the AIs by their size (equity) to assess their risk level. The hypothesize is that larger AIs will show a higher inherent risk.

Our observations: 1. Despite showing high risk in the Organizational Characteristics domain, all low-risk AIs are small in size, i.e. with total equity of less than HK$10 billion.

2. As we expect, the AIs’ inherent risk level increases as their equity level goes up.

3. While we expect large AIs to show high risk in the Technologies domain due to the potential of having more difficulties in managing IT systems, it turns out not the case.

4. The group with the highest total equity reported to have a lower level of suboptimal software systems and reliance on third parties than some of its smaller counterparts, hence a lower risk in the Technologies domain.

5. The risk from the Products and Technology Services domain is more salient for large- and medium-sized AIs. The high risk appears to be caused by the wide range of services these AIs offer.

6. The narrow gap in the Organizational characteristics domain between low-risk AIs and their peers were partially caused by the lower number of cybersecurity staff.

Risk Score Comparison between Equity Size and Risk Class Size

Risk Class

61

89

<$10bn (Low)

$10bn - $30bn (Medium)

92

>$30bn (High)

Risk Indicator and Sub-domain Level Each domain comprises a number of risk indicators that reflect various business and operational aspects of AIs. We found that the AIs’ risk level depends partly on how widely applicable the risk indicators are. For risk indicators such as Prepaid Card Risk and Cloud Computing Services Risk, most AIs responded with “Not Applicable”, hence a low-risk score.

We further analyzed risk indicators in details to see how the risk varies between risk classes. To simplify the analytical process, we have grouped related indicators within a domain into various sub-domains.

Top "Not Applicable" Risk Indicator Prepaid Card Risk

20

Cloud Computing Services Risk

20

Host IT Services Risk

19

Merchant Acquier Model Risk

16

Merchant Acquier Merchant Risk

16 0

7

5

10

15

20

25

EXECUTIVE SUMMARY

Our observations: 1. When measured by percentage risk score, high- and medium-risk AIs have a high risk in Delivery Channels (domain) while low-risk AIs do not. The gap is mostly driven by a discrepancy in their mobile presence (sub-domain risk indicator), indicating that high- and medium-risk AIs offer a much wider range of mobile banking services than low-risk AIs. 2. The large gap in Technologies (domain) between high- and medium-risk AIs is mainly prompted by third-party related risk (sub-domain risk indicator). This perhaps shows that firms with high-risk classification use more third-party software/hardware or outsource various duties to vendors, making risk control more challenging.

3. The higher the risk class, the lower the cybersecurity staffing risk (sub-domain risk indicator).

High-risk AIs outperform both low- and medium-risk peers, suggesting that AIs with higher risk might be more aware of their own cybersecurity risk and have hired appropriate professionals to mitigate their risk.

Correlation of Risk with Service Offerings by AIs At the sub-domain level, we found that service provisions account for a large gap between different risk groups. Hence, we conducted further analysis to investigate how service provision impacts risk level.

Our observations: 1. Half of the surveyed AIs reported that they are offering either payment card or ATM service or both services at the same time.

2. Many of the AIs provide both services at the same time, therefore risk score in both services is closely linked together. For the service-providing AIs, we found a high risk in their mobile presence.

Risk Class Proportion By Service Provided 100%

9%

60%

Medium 64%

82%

64%

18%

18%

40% 20%

High

18%

80%

27%

0% Payment Card

Non-Payment Card

73%

Low

18% 9%

ATM

Non-ATM

These observations indicate that many risk indicators are interconnected and prompted our study with exploratory factor analysis, which is discussed in more details in a later section of this report.

Correlation of Risk with AIs’ Business Nature and Target Customers To identify any hidden characteristics among the AIs, which are not directly captured in the IRA, we identified three groups of AIs using hierarchical cluster analysis. Group A is composed of low- and medium-risk AIs, Group B contains five medium risk and one high-risk AIs , and Group C consists of AIs from all three risk classes.

8

Our observations: 1. AIs in each group appear to have a similar risk score. While the risk classifications are mixed, inherent risk scores and percentage scores within a group are mostly similar.

2. Upon examining the key characteristics between the groups, we found that the overarching difference among the three groups is on their service provision and overall size – perhaps in terms of the number of staff, assets or total amount of client deposits.

3. While we cannot conclusively suggest how the type of AIs impacts their risk level, the clustering result reveals that the AIs’ overall risk might be closely related to their business nature and target audience.

II. KEY FINDINGS OF MATURITY ASSESSMENT (MA)

The inherent risk levels of the AIs identified in the IRA process are mapped to their required maturity levels of cyber resilience: Baseline, Intermediate and Advanced. They assessed applicable security controls according to their corresponding maturity level.

INHERENT RISK LEVEL Low Medium High

MATURITY LEVEL Baseline Intermediate Advanced

Not All AIs Reach Required Maturity Levels Overall, we found that not all the AIs have reached their required maturity level. Gaps are identified in their cybersecurity measures and there are areas of improvement for them to enhance cyber resilience.

The AIs conducted assessment under the seven domains of MA: Governance, Identification, Protection, Detection, Response and Recovery, Situational Awareness and Third-party Risk Management.

Our observations: 1. High-risk AIs have adopted almost all security controls and hence high attainment rate of cybersecurity measures across the three maturity levels.

2. Medium-risk AIs perform less well. Their underperformance mostly comes from the Governance, Protection and Third-party Risk Management domains.

3. Low-risk AIs have mixed results. They are mostly on target in terms of Protection, Situational Awareness, Response and Recovery and Third-party Risk Management. However, they do not meet the minimum standard in three other domains, which are related to governance and internal environment.

9

EXECUTIVE SUMMARY

Common Strengths and Weaknesses Each domain comprises a number of components. We have identified the best and worst performing maturity components to reflect the strengths and weaknesses of AIs in taking cybersecurity measures.

Best Performing Components Data Security

99.1%

Incident Management

97.3%

Escalation And Reporting

96.7%

Access Control

96.5%

Threat Monitoring And Analysis

96.3%

Response Planning

95.8%

Worst Performing Components Audit

90.8%

Anomalies Activity Detection

90.8%

Cyber Risk Identification And Assessment

89.1%

External Connections

88.2%

Ongoing Monitoring On Third-party Risk

84.9%

Third-party Management

82.9%

Our observations: 1. Data security is the best performing component with over 99% attainment rate, showing that all surveyed AIs have taken as many controls as possible to safeguard their sensitive information and data.

2. Overall, the AIs do the best under the Response and Recovery domain in which three of its components (Incident Management, Escalation and Reporting, and Response Planning) are among the best performing ones. This shows that most AIs perform well in implementing follow-up measures in case of cybersecurity incidents.

3. The average maturity attainment rate is over 80% for all of the worst performing components, which is an encouraging finding.

III. OVERALL RELATIONSHIP BETWEEN INHERENT RISK LEVEL AND MATURITY LEVEL

1. After a thorough investigation and analysis, we found a positive relationship between the AIs’ inherent risk score and their maturity level. A high overall risk tends to mean a higher maturity score in each domain.

2. Many medium-risk AIs have failed to reach their target maturity level. Their underperformance is not due to outliers but a group-wise issue.

3. High-risk AIs are more mature than medium-risk AIs under many maturity domains, despite having better track records in terms of their cybersecurity threats and having an equal or lower risk in many inherent risk domains.

10

IV. AI’S CYBERSECURITY PRACTICE IN REAL LIFE

To understand how well the maturity assessment reflects the surveyed AIs’ actual security practices, we examined the AIs’ adoption of Secure Sockets Layer (SSL) and suboptimal certificates. SSL certificate is a digital certificate that authenticates a website identity.

Suboptimal certificates make AIs more vulnerable to cyberattacks.

Our observations: 1. The AIs’ overall maturity level and particularly the maturity level in the Protection domain appears to be significantly correlated to the adoption rate of SSL certificates.

2. AIs with high maturity attainment rate appear to show a higher adoption rate of security certifications and a low adoption rate of suboptimal certificates such as self-signed or weak hashing algorithm certificates.

3. The Maturity Assessment reflects the AIs’ actual cybersecurity measures. AIs that perform better in the Maturity Assessment in terms of their attainment percentage rate tend to take adequate cybersecurity measures to safeguard their assets.

AI’s Cyber Resilience Over Time We analysed the AIs’ adoption of SSL certifications to gain a better understanding on how the AIs’ cyber resilience has changed over time after the C-RAF survey and how the selfassessment exercise impacts the AIs’ intention in implementing cybersecurity measures.

Our observations: 1. We found a steady improvement in AIs’ SSL certification adoption rate. The improvement comes from those who have not attained all the required maturity measures.

2. There is a decrease in the adoption of weak hashing algorithm certificates among the AIs. For AIs whose attainment rate is 100%, the drop is even more than those AIs without full attainment.

3. In order to understand the impact of C-RAF in improving AIs’ cyber resilience over time, we suggest the C-RAF framework to further examine if AIs have taken additional cybersecurity measures or compare the number of cybersecurity incidents they face after the C-RAF selfassessment exercise.

11

CYBER RESILIENCE ASSESSMENT FRAMEWORK (C-RAF)

12

CYBER RESILIENCE ASSESSMENT FRAMEWORK (C-RAF)

INTRODUCTION

Cybersecurity is a growing concern worldwide across various sectors. According to a McAfee report, the global loss from cybercrime is estimated to be approximately US$945 billion in 2020.1 Meanwhile, the spending on cybersecurity is expected to exceed US$145 billion, putting the total cost related to cybercrime to more than US$1 trillion.

Global loss from cybercrime

US$945 billion US$523 billion

+80%

2018

McAfee estimates that since 2018, the cost of global cybercrime reached over

2020

$1 TRILLION

[ Exhibit 1 ] Global loss and cost from cybercrime

Nevertheless, the report above shows that only a handful of organizations have plans to prevent and respond to information technology security incidents. In the 2020 “SSH Hong Kong Enterprise Cyber Security Readiness Index” survey, Hong Kong companies’ overall cybersecurity readiness index fell by 2.4 points from the previous year to 46.9 out of 100. While financial services firms are the most vigilant performers, they have only scored a merely 62.9.

1.

McAfee (2020). The Hidden Cost of Cybercrime. Available at https://www.mcafee.com/enterprise/en-us/assets/ reports/rp-hidden-costs-of-cybercrime.pdf [accessed March 19, 2021].

2.

HKCERT and SSH.com (2020). SSH Hong Kong Enterprise Cyber Security Readiness Index 2020 Survey. Available at https://events.hkpc.org/1613633199/SSH-HKECSRI2020.pdf [accessed March 19, 2021].

13

SSH Hong Kong Enterprise Cyber Security Readiness Index

2019

46.9

2020

44.5

-2.4

points from 2019

[ Exhibit 2 ] SSH Hong Kong Enterprise Cyber Security Readiness Index

Given the growing cybersecurity risks, the Hong Kong Monetary Authority (HKMA) has been working with the banking industry to oversee and monitor their handling and management of cybersecurity risks.

In 2016, HKMA launched a campaign called cybersecurity fortification initiative (CFI), which comprises three components: (1) the Cyber Resilience Assessment Framework (C‐RAF); (2) the Professional Development Program (PDP); and (3) Cyber Intelligence Sharing Platform (CISP). C-RAF is a risk-based cybersecurity assessment framework for authorized institutions (AIs) to assess their risk profiles and the maturity of their cybersecurity measures. Through this process, AIs will be able to better understand and continuously improve their cyber resilience. C‐ RAF further comprises three stages: • Inherent Risk Assessment (IRA) • Maturity Assessment (MA) • Intelligence-led Cyber Attack Simulation Testing (iCAST) INHERENT RISK ASSESSMENT

MATURITY LEVEL ASSESSMENT

The Als perform an assessment across five domains to reflect their inherent risk level, which in turn determines the expected maturity level.

A self-assessment form across seven domains to determine the actual maturity level of the Als

INHERENT RISK LEVEL Low Medium High

MATURITY LEVEL Baseline Intermediate Advanced

[ Exhibit 3 ] Inherent risk assessment and maturity assessment introduction

This report focuses on the Inherent Risk Assessment (IRA) and Maturity Assessment (MA) under C-RAF, both of which are self-assessment exercises. IRA ensures the AIs in Hong Kong are mindful of their intrinsic risks related to their business size, scope, and variety of services. MA ensures the AIs have the corresponding security controls commensurate with their risk levels.

To provide some context, Exhibits 4 and 5 show a few examples of indicators (controls) for the five key domains for the Inherent Risk Assessment and the seven domains for the Maturity Assessment. We will elaborate on the two assessments in later sections.

14

CYBER RESILIENCE ASSESSMENT FRAMEWORK (C-RAF)

The Five Domains of the Inherent Risk Assessment Domain

Sample Indicators

Technologies

• The number of Internet service provider connections connected to the corporate network • Wholesale customers with dedicated connections

Delivery Channels

• Internet presence (customer) • Automated Teller Machines (ATM) (Operation)

Products and Technology Services

• Issue debit or credit cards • Global remittances

Organizational Characteristics

• Total number of branches • Changes in IT and cybersecurity staffing

Tracked Records on Cyber Threats

• Reported cyberattacks impacting the AI for Hong Kong businesses (last 12 months)

[ Exhibit 4 ] The five domains of inherent risk assessment and their sample indicators

The Seven Maturity Domains of the Maturity Assessment Domain

Sample Indicators

Governance

• Cyber resilience oversight • Strategy and policies

Identification

• IT asset identification • Cyber risk identification and assessment

Protection

• Infrastructure protection controls • Access control

Detection

• Vulnerability detection • Anomalies activity detection

Response and Recovery

• Response planning • Incident management

Situational Awareness

• Threat intelligence (information about emerging or existing threats) • Threat intelligence sharing

Third-party Risk Management

• External connections • Third-party management

[ Exhibit 5 ] The seven domains of maturity assessment and their sample indicators

15

Twenty-two AIs including licensed banks, restricted licensed banks and deposit-taking companies participated in a survey to capture the self-assessed classification and detailed responses of AIs. For example, for each of the IRA indicators, the AIs would note down one of the “Low”, “Medium” and “High” answers based on the threshold set in the survey, and to provide additional detailed information in both writing and numbers (when applicable). With the Maturity Assessment, the AIs would respond to whether the maturity controls are met while providing information on the implementation and any gaps noted. This allows us to conduct an investigation using the categorical responses and examine the detailed responses to gain a more in-depth understanding.

CYBERSECURITY ASSESSMENT

Before we analyse the C-RAF assessment survey data, we first review the motivation, purpose, and state of cybersecurity regulation and assessment development. We then report on other popular cybersecurity assessment frameworks and tools, and draw a comparison with C-RAF. Cybersecurity regulation can be broadly classified into three categories: (1) Self-regulation, (2) Co-regulation, and (3) Statutory regulation, depending on the balance between government intervention and voluntary industry participation. Unlike co-regulation, which includes state participation and enforcement, self-regulation is represented by non-hierarchical private actors, such as industry associations and organizations that implement different mechanisms to regulate the security practices within the industry. The involvement of these actors in fighting cybercrime and providing cybersecurity varies from ad hoc collaboration upon police request to sustainable self-regulation, for example, private hotlines for reporting illegal content (Tropina & Callanan, 2015).

Another form of self-regulation is alliances. Companies have started entering cybersecurity alliances, such as operational alliances and normative alliances. Built around small groups of companies, an operational alliance shares information about cyberattacks and threats to raise the collective level of cybersecurity (Dobrygowski, 2019). Examples of such operational alliances include the Cyber Threat Alliance, the Global Cyber Alliance, and the Trusted Computing Group. On the other hand, a normative alliance strives to proactively spur collective action in favour of digital peace and non-aggression. They explicitly call for government support and limit the use of private systems and networks against citizens (especially by a nation-state) (Dobrygowski, 2019). The Charter of Trust, initiated by Siemens, and the Cybersecurity Tech Accord, originated by Microsoft and other leading technology companies, are two examples of normative alliances.

16

CYBER RESILIENCE ASSESSMENT FRAMEWORK (C-RAF)

A security framework provides a common reference to measure cybersecurity capabilities within an organization (Le and Hoang, 2016). The framework may variously consider user, networking device, software, network, process, application, and information in storage or transit that can be directly or indirectly connected to the network (Srinivas et al., 2018). Multiple parties worldwide, including industry organizations, governments, and international organizations, have published cybersecurity standards, frameworks, or guidance. Below, we briefly introduce three internationally applied frameworks, compare them against C-RAF in terms of coverage and focus, and provide a comparison table with additional frameworks.

National Institute of Standards and Technology (NIST) cybersecurity framework

International Standards Organization (ISO)/IEC 27001

The Center for Internet Security Critical Security Controls (CIS CSC)

As defined on the official site, the NIST cybersecurity framework focuses on using business drivers to guide cybersecurity activities and considering cybersecurity risks as part of the organization’s risk management processes.

The framework consists of three parts: Framework Core, Framework Profile and Framework Implementation Tiers.

The ISO 27001 is a globally recognized standard for information security management system (ISMS), focusing on keeping the information assets secure. This ISO standard gives organizations guidance on establishing, implementing, maintaining, and continually improving an ISMS.

The CIS CSC is a set of prioritized and simplified best practice guidelines for computer security.

Also known as CIS Controls, the publication is owned by the Center for Internet Security, a communitydriven non-profit organization of IT professionals.

Notably, the Framework Core is a set of cybersecurity activities, desired outcomes and applicable references that are common across critical infrastructure sectors.

The core includes five high-level functions: Identify, Protect, Detect, Respond, and Recovery.

The standard covers ten clauses, including the organization’s context, leadership and commitment, planning, support, operation, performance evaluation and improvement. Organizations that meet the standard’s requirements can choose to be certified by an accredited certification body after completing an audit.

[ Exhibit 6 ] Examples of commonly used cybersecurity assessment frameworks

17

Aimed at protecting organizations and data from known cyberattack vectors, the controls consist of 20 key actions for three implementation groups. The actions cover basic, foundational and organizational measures ranging from asset management, data protection to incident response and penetration testing.

C-RAF AND OTHER CYBERSECURITY ASSESSMENT FRAMEWORKS

Recall that the C-RAF Maturity Assessment consists of seven domains: Governance, Identification, Protection, Detection, Response and Recovery, Situational Awareness, and Third-party Risk Management. These domains have in fact incorporated more aspects than their name implies. We have split and mapped the relevant areas in Exhibit 7.

C-RAF*

Domain of Analysis*

Governance (81)

Governance (59) Expertise and training (22)

Protection (106)

Data security (28) Security control and incident prevention (78)

Detection (60)

Detection (60)

Response and Recovery (51)

Incident management (51)

Situational Awareness (18)

Situational awareness (18)

Third-party Risk Management (27)

Oversight of interconnections (27)

Identification (23)

Risk analysis and assessment (23) Continuous learning/improvement

* Numbers in brackets denote the number of controls included in the C-RAF Maturity Assessment. [ Exhibit 7 ] Mapping of maturity assessment domains and the ten domains of analysis

With the mapping in mind, Exhibit 8 presents a table that compares C-RAF with a few additional reference frameworks in terms of 10 domains: governance, risk analysis and assessment, data security, security control and incident prevention, expertise and training, detection, incident management, situational awareness, oversight of interconnections, and continuous learning / improvement. Essentially, we found that the C-RAF has covered nine of the ten domains commonly included in other cybersecurity frameworks, but C-RAF does not measure continuous learning and improvement, similar to the NERC Cybersecurity standards and the BCBS Cyber-Resilience-Range of Practice.

18

CYBER RESILIENCE ASSESSMENT FRAMEWORK (C-RAF)

Gov1

Risk

Data

Sec

Exp

Det

Inc

Situa Over

Cont

Scope

Framework / Standards

C-RAF

HK authorized financial institutions

NIST

Int

UK NCSC2

UK

NERC CIP3

US Electricity

PCI DSS

Payment card

NZISM4

New Zealand

ISO

Int

CIS CSC

Int

COBIT

Int

National

5

ASD

Business Environment, Supply Chain Risk Management

Australia

CREST6 GoC

Unique

Int

7

Canada FRFI8

Organizations and Resources

International Guidance

FFIEC9

US Financial Institutions

ETSI10

Int

FINRA11 IIROC12

Small firms in US Finance Canada Investment Cyber insurance

BCBS13 CPMIIOSCO14

Banking Financial Markets Infrastructures

G7-CEG15

Financial Sector

16

OECD

Facilitation mechanisms

Testing as a separate function

National policy

[ Exhibit 8 ] Cybersecurity frameworks table of comparison

1.

Domain abbreviations: Gov for governance, Risk for risk analysis and assessment, Data for data security, Sec for security control and incident prevention, Exp for expertise and training, Det for detection, Inc for incident management, Situa for situational awareness, Over for oversight for interconnections, Cont for continuous learning/ improvement 2.

UK National Cyber Security Centre, Cyber Assessment Framework 3.

North American Electric Reliability Corporation - Critical Infrastructure Protection, Cyber Security Standards 4.

Government of New Zealand, NZISM Protective Security Requirements 5.

Australian Signals Directorate (ASD) Strategies to Mitigate Cyber Security Incidents 6.

CREST Maturity Assessment Tools 7.

Government of Canada, Cyber Security Self-Assessment Guidance 8.

Federally Regulated Financial Institutions

19

While the coverage of cybersecurity maturity of C-RAF is consistent with most internationally well-recognized frameworks, it also offers some unique advantages over its peers. As mentioned above, C-RAF consists of three parts: (1) Inherent Risk Assessment, (2) Maturity Assessment, and (3) Intelligence-led Cyber Attack Simulation Testing (iCAST). The three parts work cohesively to offer a more customisable measure to the organizations. First, IRA allows the entities to identify their own risk, then suggest the corresponding risk level and maturity requirement. Next, the AIs could evaluate their maturity level based on the requirement, identify any gaps for improvement and set the corresponding deadlines. Finally, the iCAST offers an opportunity for organizations to test their cybersecurity level beyond traditional penetration tests. Therefore, it forms a complete and tailored experience to the AIs that reflects their actual risk and maturity to help minimize potential cybersecurity induced losses.

9.

Federal Financial Institutions Examination Council's (FFIEC), Cyber Assessment Tool ETSI TR 103 305-1 V3.1.1 11.

Financial Industry Regulation Authority (FINRA), small firm cyber security checklist 12.

Investment Industry Regulatory Organization of Canada (IIROC), cybersecurity best practices guide & cyber governance guide 13.

Basel Committee on Banking Supervision (BCBS), Cyber-Resilience-Range of Practices 14.

The Committee on Payments and Market Infrastructures (CPMI) and the Board of the International Organization of Securities Commissions (IOSCO) , the Guidance on cyber resilience for financial market infrastructures 15.

G7 Cyber Expert Group (CEG) Fundamental Elements for Cybersecurity 16.

Organization for Economic Co-operation and Development (OECD), Recommendation of the Council on Digital Security of Critical Activities 10.

20

INHERENT RISK ASSESSMENT (IRA)

INHERENT RISK ASSESSMENT (IRA)

21

OVERVIEW

As the first part of the C-RAF framework, the Inherent Risk Assessment is designed to reflect AIs’ cybersecurity threat level, determine their cyber risk exposure, and decide which threshold they should be subject to in the Maturity Assessment. For each of the 51 indicators in the Inherent Risk Assessment, AIs would report their self-assessed risk levels, which are “Low”, “Medium”, “High”, or “Not Applicable”. By default, an AI’s overall risk level is determined by the most common risk level in the survey. An AI also can indicate an alternative classification by considering other relevant factors, including its size and business model.

We calculated a risk score for each AI by converting its response on an ordinal scale to quantify the results better, where we assigned a score of one to a low-risk response, two to a mediumrisk response, three to a high-risk response, and zero when a risk indicator is deemed “Not Applicable” by the AI. To extract any insights, we first grouped the AIs in terms of their overall risk level and broke the domains down by various indicator groupings. We then categorized the AIs by attributes such as business size and the type of services they provide. Lastly, we applied a clustering algorithm to identify hidden groupings based on their survey responses and utilized factor analysis to extract latent factors.

AIs’ Business Size

Average (HK$)

Risk Class Population High

Total Assets

Medium

Low

$227.3bn 14%

Deposits from Customers

$132.4bn

Total Equity

$29.2bn

Capital Ratio

26.7

45%

[ Exhibit 9 ] Key statistics of surveyed AIs

41%

[ Exhibit 10 ] Risk class population of surveyed AIs

22

INHERENT RISK ASSESSMENT (IRA)

In summary, we found that the overall risk increases as an AI provides a broader range of services under the IRA framework assessment criteria. Simultaneously, the expansion of an AI in size would accelerate the rise in cybersecurity risk. However, we also found that the riskiest and largest of AIs have made self-awareness about the issues they face and have taken actions to mitigate them better when efforts in their less risky counterparts are to be improved. Overall, the IRA framework risk class calculation methodologies effectively categorize the AIs, aligning with both classifications by size and service provision. Lastly, we have made a few suggestions for improving the risk calculation methodology according to our findings.

Technologies

The “Technologies” domain contains 17 indicators including aspects such as the number of network-connected devices and hardware related issues. This domain also assesses the condition of the software eco-system of an AI by taking the number of suboptimal software systems and the degree to which the AI rely on third parties into considerations.

Delivery Channels

There are 4 risk indicators under the “Delivery Channels” domain, which measure the AI’s exposure and presence in various channels to interact with their customers, including their internet, social media and mobile presence. Last but not least, this domain also assesses the risk induced by offering ATM services.

Products and Technology Services

The “Products and Technology Services” domain comprises 14 indicators, which assess the AI’s risk induced by banking-related services. The three main aspects are related to the provision of payment cards, fund transfer and other services, including treasury, trust and securities trading.

Organizational Characteristics

“Organizational Characteristics” consists of 9 indicators, assessing the AI’s risk in terms of their size and risk areas indirectly related to service provision, including the number of employees. Moreover, this domain also measures the risk related to the lack of cybersecurity staff and the turnover of cybersecurity staff.

Cyber Threat Tracked Records

As the name suggests, the “Cyber Threat Tracked Records” domain hosts a total of 7 risk indicators related to the AI’s history to cyber attack. There are broadly two categories of risk, including the frequency and variety of cyberattacks, recording both successful and failed attempts.

[ Exhibit 11 ] The five Inherent Risk Assessment domains

23

SURVEY RESULT

Let us first take a holistic view to understand the differences in risk levels among the three risk classes. Recall that the most common risk level determines the AIs’ overall risk. It is expected that higher-risk AIs would show elevated risk in each domain. The chart in Exhibit 12 helps us identify “Technologies” as the key differentiating domain between medium- and high-risk AIs, while “Products and Technology Services” presents the greatest gap between low-risk AIs and their peers. Lastly, we observed that “Tracked Records on Cyber Threats” is comparable across the three risk classes.

Risk Score Comparision Low

Medium

High

Biggest gap between

Largest gap between

Similar across all risk

medium- and high-

low- and medium-

classes

risk AIs

risk AIs

17 24 32

5 9 9

12 21 21

14 15 17

10 12 11

Technologies

Delivery Channels

Products and Technology Services

Organizational Characteristics

Tracked Records On Cyber Threats

[ Exhibit 12 ] Inherent Risk Assessment – risk score comparison

However, the risk score does not provide a fair comparison across the five domains as each has a different number of indicators. To provide a standardized view of risk propensity independent of the number of indicators, we computed the percentage risk by dividing the total scores by the maximum possible in each domain. Exhibit 13 shows how a percentage risk score is calculated.

Delivery Channels

For example – Under the “Delivery Channels” domain, which has a total of FOUR indicators

Cyber Threat Tracked Records

An AI responded with: 1x Low-risk 2x Medium-risk 1x High Risk

Therefore a risk score of: 1x1+2x2+1x3=8

But the maximum score is the number of indicators multiple by three, i.e. 3 x 4=12 Hence the percentage risk score of: 8/12 = 67% [ Exhibit 13] Inherent risk score and percentage risk score calculation

24

INHERENT RISK ASSESSMENT (IRA)

Percentage Risk Score Tracked Records On Cyber Threats

High

Organizational Characteristics Products and Technology Services Delivery Channels

Medium

Technologies

Low

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

[ Exhibit 14 ] Risk class percentage risk score

Based on the percentage risk score, we observed the following: 1. The Gap in “Technologies” Remains Salient The discrepancy in “Technologies” between medium- and high-risk AIs remain the most salient, with a 16 percentage points difference.

2. “Delivery Channels” Becomes Much Riskier High- and medium-risk AIs face the most potent threat in “Delivery Channels”, each with about a 15 percentage points lead compared to their second riskiest domain.

3. Low-risk AIs Show High Organizational Characteristics Risk On the other hand, low-risk AIs face the highest risk in “Organizational Characteristics” while facing lower risk in “Delivery Channels”, “Technologies”, and “Products and Technology Services”.

An absence of obvious risk pattern when comparing across risk classes has suggested that some risk class-dependent factors might lead to the discrepancy, signalling differences in the dynamic of cybersecurity risk across groups. There are serval possibilities to the overall lowrisk score to the low-risk group. For example, the observation above either indicates that they are competent in addressing technical and service induced risks; or have a low reliance on technologies and face fewer clients, but might have struggled in aspects such as IT staffing and cybersecurity staff retention. Meanwhile, medium- and high-risk AIs have experienced the highest risk in “Delivery Channels”, which could be driven by the active use of various external communication channels or a more comprehensive range of service provision. To determine the underlying reasons for the observation above, we have conducted a more thorough investigation in a later section.

25

SURVEY RESULT – BY SIZE

According to Fitch Rating’s Managing Director Christopher Wolfe,3 larger banks might face higher cybersecurity risk as they are more likely to have complicated or perhaps legacy IT infrastructure compared to smaller banks. Therefore, we sought to analyse the IRA results by grouping the AIs with equity instead of their risk level. As illustrated in the right-hand chart, when the log of equity increases, the overall score of the AIs rises. More specifically, we found that the log equity has a modest to a high positive relationship with its overall risk scores, and a 1% change in the equity level would, on average, increase the AI’s risk score by 0.11 percentage points.

Relationship between Equity and Overall Risk Score

Medium

Low

High

Overall Score

100 80 60 40 07

08

09

10

11

12

Log Equity [ Exhibit 15 ] Relationship between AIs’ equity and their overall risk score

To generalize our analysis, we have grouped the AIs into different equity bins. An AI is said to be small when it has an equity level of below HK$10 billion, and it is classified as large when it has an equity level of over HK$30 billion, with a medium anywhere in between.

Risk Class

Equity Size (HKD$ Billion) Small (<$10bn)

Medium ($10bn - $30bn)

Large (>$30bn)

Low

3

0

0

Medium

1

3

3

High

1

1

1

* With data of 13 respondents [ Exhibit 16 ] Survey risk class and equity size mapping

Interestingly, none of the low-risk AIs is of medium and large size when measured by their equity level, despite as seen in the section above, “Organizational Characteristics” is their worst performing domain percentage-wise. This provides evidence that the low-risk AIs have experienced risk from non-business-size-related risk under the “Organization Characteristics” domain.

Exhibit 17 indicates when we break the AIs down into three categories by size, we obtain a similar average score compared to when breaking down by risk class, despite a difference in sample size.

3.

https://www.fitchratings.com/research/banks/bigger-not-always-better-for-bank-cyber-risk-scores-13-04-2021

26

INHERENT RISK ASSESSMENT (IRA)

The bar chart in Exhibit 18 helps identify the best and worst performing domains for each group and allows us to compare the grouping with the overall risk class. Based on the percentage risk scores across equity categories, we observe the following:

Risk Score Comparison between Equity Size and Risk Class Size

Risk Class

100

1. Outperformance in “Technologies” “Technologies” is the least risky domain for the large AIs with 50% risk score, compared to the 63% for the high-risk AIs as seen in the previous section.

80

60

2. “Products and Technology Services” Becomes More Salient Large- and medium-sized AIs face higher risk in “Products and Technology Services” when mapped against large- and mediumrisk AIs, while AIs with lower than HK$10 billion equity have little changes compared to the low-risk AIs.

40

61

20

89

92

3. Narrow Gap in “Organizational Characteristics” Despite containing risk indicators that measure an AI’s size, in the “Organizational Characteristics” domain, we found the smallest gap between small-sized AIs and their peers.

0 <$10bn (Low)

$10bn - $30bn (Medium)

>$30bn (High)

[ Exhibit 17 ] Risk score comparison between equity size and risk class

Percentage Risk Score (By Size) Tracked Records On Cyber Threats

Large

Organizational Characteristics Products and Technology Services Delivery Channels

Medium

Technologies

Small

0%

10%

20%

30%

40%

50%

60%

70%

[ Exhibit 18 ] Percentage risk score across the three equity categories

27

80%

90%

Strikingly, technology risk is among the best performing risk areas for AIs across all three sizes, while medium-sized AIs face higher risk compared to their larger peers. This contradicts our beliefs that larger AIs might face higher cyber risk due to more complex and challenging IT infrastructure. Indeed, we found that larger AIs have lower suboptimal software systems and a lower number of applications and reliance on third-party service providers.

This suggests that largest banks might have already taken the initiative to move away from their legacy system, put substantial effort into digital transformation, and structurally organize their software systems to mitigate their technology risk.

Low “Technologies” risk score in large AIs Although large AIs have notably more network devices, including servers, routers and firewall (both physically and virtual) Compare to medium-sized AIs, large AIs have less: - End Of Life applications - Without commercial-support open-source software - Amount of third parties Compare to both medium- and low-risk AIs: - Less in-house applications [ Exhibit 19 ] Large AIs’ “Technologies” risk profile

The large gap in “Products and Technology Services” and “Delivery Channels” between a small AI and its larger counterpart suggests that smaller AIs offer a narrower range of services to their customers and hence face lower cybersecurity risks. Lastly, by examining the detailed responses, we believe that the lack of cybersecurity staff could partially explain the narrow gap in “Organizational Characteristics”. As most small AIs reported a high risk in the number of cybersecurity staff, where the answers are predominantly medium risk or below for their larger peers.

Overall, the results above suggest that while larger AIs face a higher risk for offering a more comprehensive range of services, they have also invested in mitigating technology-related / induced risks. Meanwhile, smaller AIs face lower risk partly due to their smaller exposure but are not equally invested in reducing the risk they face. While the similarity in risk levels between the two classification methods reveals little new insight about the AIs, it provides support to the effectiveness of the IRA framework’s risk classification methodology, given that business size is correlated to the firm’s overall risk score, as shown above.

28

INHERENT RISK ASSESSMENT (IRA)

INDICATOR AND SUB-DOMAIN ANALYSIS

To better understand which area AIs often struggle with, we have sorted the top and bottom five risk indicators out of the 51 IRA indicators in terms of total risk score, subject to the same score calculation rule above. The charts in Exhibits 20 and 21 reveal some interesting characteristics of the surveyed AIs. For example, Exhibit 20 helps identify that many of the respondents have a strong internet presence, indicating their use of the internet to provide online retail banking, wholesale banking and other banking services, instead of merely a channel that is just for providing information and not services. Again, the high risk in treasury services indicates that the surveyed AIs offer a large range of treasury services, including lockbox, currency services and online investing.

Most Risky Indicators by Risk Score

Least Risky Indicators by Risk Score

77%

83%

85%

88%

88%

3%

3%

6%

15%

15%

Number of Network Devices

Internet Presence

Treasury Services Risk

Asset Value Risk

Wire Transfer Channel Risk

Number of Clould Computing Services

Prepaid Cards

Host IT Service

Dedicated Connection

Merchant Acquirer Model Risk

[ Exhibit 20 ] Most and least risky indicators by risk score

Relevance of Risk Indicators to AIs Another observation we have made is how risky a risk indicator is depended partly on the applicability of the indicator to the AIs. For example, we observe an exceptionally low risk score for “Cloud Computing Services” and “Prepaid Cards” mainly because many AIs have indicated that these risk indicators are not applicable to them, hence a risk score of zero. Exhibit 21 shows the top five risk indicators by their number of “Not Applicable” response. The red dotted line is the maximum number possible, i.e. the number of surveyed AIs. The high “Not Applicable” rate, however, does not mean these risk indicators are redundant. Using cloud computing services risk as an example, with the increased progression of technology and increase in popularity of cloud computing services, AIs could in the future adopt the use of various cloud computing services, and the Inherent Risk Assessment would have acted as a benchmark to help AIs identify how much additional risk might be added and to incentivize them to implement the necessary measures.

Prepaid Card Risk Cloud Computing Services Risk Host IT Services Risk Merchant Acquier Model Risk Merchant Acquier Merchant Risk

20 20 19 16 16 0

[ Exhibit 21 ] Top “Not Applicable” risk indicators

29

5

10

15

20

25

Grouping of Related Risk Indicators into Sub-domains Next, we shall analyse the indicators and see how the risk varies between risk classes. To simplify the analytical process, we have grouped related indicators within a domain into various sub-domains. For example, for the “Technologies” risk domain, we have grouped the 17 indicators into three sub-domains: (1) Network risks, (2) Third-party risks and (3) Internal risks.

A complete mapping is provided in the appendix (appendix 1).

DOMAIN

SUB-DOMAIN

INDICATOR

Cyber Threat Frequency

1. Number of Attempted Cyber Attacks 2. Number of Successful Attacks 3. Number of Breaches

Cyber Threat Variety

1. Phishing Attempts 2. Denial-of-Service (DoS) Attacks 3. Social Engineering 4. Malware

Cyber Threat Tracked Records

The official catagory of the Inherent Risk Assessment, which takes into account various business and operational aspects of the AIs.

A sub-catagory grouping done within this report, to simplify the analytical process and reduce data dimension.

For each assessment indicators, the AIs select the most appropriate description under "low", "medium" or "high" inherent risk.

[ Exhibit 22 ] Inherent risk indicator grouping example

Percentage Point Difference Risk sub-domain

Medium - Low

High - Low

Network risks

8%

14%

Technologies

Third-party risks

7%

54%

Delivery Channels

Internal risks

26%

16%

Products and Technology Services

Internet presence

23%

30%

Mobile presence

56%

67%

Organizational Characteristics

Social media presence

16%

23%

ATM

38%

38%

Payment card risks

32%

36%

Fund transfer risks

11%

20%

Client services

28%

5%

Size risks

8%

11%

Service risks

12%

31%

Cybersecurity staffing risks

-5%

-20%

Frequency

-3%

9%

Variety

-11%

2%

Tracked Records On Cyber Threats

Top 3 in each column Bottom 3 in each column

* in percentage points [ Exhibit 23 ] Risk sub-domain percentage-point difference between low-risk AIs and its peers

30

INHERENT RISK ASSESSMENT (IRA)

Our Observations: 1. Delivery Channels Risk Driven by Mobile Presence Recall that high- and medium-risk AIs face the most potent risk in “Delivery Channels”. We see that it is mainly driven by their mobile presence risk, which are 56 and 67 percentage points higher than their low-risk counterparts.

2. Relatively Small Gap in Social Media Risk Under “Delivery Channels” domain, low-risk AIs have the smallest gap with their peers in social media presence. This is perhaps because low-risk AIs do not need to provide mobile banking services but would still, albeit to a smaller extent, use social media to interact with their customers.

3. Third-party Risk Caused Technology Risk Gap In previous sections, “Technologies” is the main distinguishing characteristic between medium- and high-risk AIs. Here we found that it is mainly prompted by third-party related risks, perhaps signalling firms with a high-risk classification use more third-party software / hardware or outsource various duties to vendors, making risk control more challenging.

4. Higher Risk AIs See Lower Risk In Cybersecurity Staffing Related Area Intriguingly, the higher the risk class, the lower the cybersecurity staffing risk appears to be.

Here we observe that high-risk AIs outperform both the low- and medium-risk peers by 20 and 15 percentage points respectively. This suggests higher risk AIs might be more aware of their own cybersecurity risk and have subsequently taken actions to hire the appropriate professionals to manage their cybersecurity risk.

The table in Exhibit 23 also shows that the extent of risks in terms of frequency and variety of threats from the past track records is similar across all AIs. Overall, only one out of the 22 AIs reported three breaches, while all others reported none. Collectively, the responding AIs signal that they experience the lowest risk in social engineering and Denial-Of-Service (DOS) attacks, both with 16 out of 22 AIs reporting low risk. Meanwhile, phishing and malware attacks are deemed riskier, each with over 15 AIs responding with a medium rating, though most supplemented that their anti-virus software could stop those attacks. In addition, some AIs reported other types of attacks that were not captured in the IRA questionnaire. More particularly, in the detailed responses one of the AIs reported a large number of SQL injection attempts while another one AI reported incidences of ransomware attacks. Amid the fast-changing cybersecurity landscape, we suggest adding an “Other Attacks” column in the IRA questionnaire for AIs to report other types of cyberattacks they have faced. This may give a fuller picture and help their peers and regulators to be aware of new forms of cybersecurity threats.

31

Overall, the observations above have helped us identify the key element of risk for the three groups. By looking at the detailed responses of the AIs, we could rationalize such differences and determine the cause of discrepancies. Therefore, based on the experience of conducting the analysis above, we believe the IRA exercise could allow regulators and AIs alike to easily identify their cybersecurity weaknesses and better establish plans to fill up any gaps.

ANALYSIS BY SERVICE PROVISION

As analyses in the previous section demonstrated, there is a large gap between the sub-domain risks for service provision, where high- and medium-risk AIs show more than 30 percentagepoint difference compared to low-risk AIs. To further investigate how they impact their overall cybersecurity risk level, we have conducted some analysis based on the types of services they provide.

Among the 22 surveyed AIs, 11 offer payment card services and 11 offer ATM services. (N.B. some AIs could offer both payment card and ATM services simultaneously.)

Risk Class Proportion By Service Provided 100%

9%

60%

Medium 64%

82%

64%

18%

18%

40% 20%

High

18%

80%

27%

0% Payment Card

Non-Payment Card

73%

Low

18% 9%

ATM

Non-ATM

[ Exhibit 24 ] Risk class proportion by service provided

As Exhibit 24 shows, service-providing AIs are predominantly high- and medium-risk, while most non-service providing AIs are low-risk. Evidently, AIs that offer a larger variety of consumer services would be more exposed to cybersecurity risk.

Since AIs often provide both payment card and ATM services, the two sub-domains are the most salient for both groups. In addition, both groups see a high mobile presence risk, which could be interpreted as they would also offer mobile banking to facilitate service provision. This would magnify their overall risk level.

32

INHERENT RISK ASSESSMENT (IRA)

Correction of Risk with AIs’ Service Offerings Percentage Point Difference Risk sub-domain

∆ ATM & Non-ATM

∆ Payment & Non-Payment

Network risks

12%

14%

Third-party risks

7%

17%

Internal risks

14%

21%

Internet presence

15%

21%

Mobile presence

45%

58%

Social media presence

15%

21%

ATM

55%

48%

Payment card risks

38%

41%

Fund transfer risks

18%

19%

Client services

17%

17%

Size risks

15%

15%

Service risks

7%

14%

Cybersecurity staffing risks -3%

-6%

Frequency

0%

2%

Variety

-2%

3%

Technologies Delivery Channels Products and Technology Services Organizational Characteristics Tracked Records On Cyber Threats

Top 3 in each column Bottom 3 in each column

[ Exhibit 25 ] Risk sub-domain percentage-point difference between service-providing and non-service-providing AIs

While service-providing AIs should show a higher risk under the sub-domain on the relevant services, the AIs often show high-risk scores in other possibly related sub-domains. The key takeaway is that under the design of the Inherent Risk Assessment, many areas of risk could be interrelated, which explains the large gap between service-providing and non-service providing AIs. Hence we utilize factor analysis in a later section to identify any unobserved factors that lead to the higher cybersecurity risk.

33

HIERARCHICAL CLUSTERING

While most AIs adhere to their default risk level, the rule does not categorize the AIs by their characteristics manifested in their survey responses. To identify any hidden groupings with the AIs, we utilized hierarchical clustering. We applied the clustering algorithm on the total scores over the five inherent risk domains for each AI. As seen in the heat map in the appendix (appendix 2), each AI has been sorted, and similar AIs are clustered closer together.

By examining the dendrogram in Exhibit 26, we have identified three clusters. Group A is composed of low- and medium-risk AIs, Group B contains five medium-risk and one high-risk AIs, and Group C consists of AIs from all three risk classes. While the risk classifications are mixed, we can see from the charts in Exhibit 28 that the domains’ inherent risk scores and percentage scores within a group are mostly similar.

The chart below shows that the surveyed AIs have been grouped into three clusters, where the x-axis is the AIs, and the y-axis represents how close the AIs are in terms of their survey response.

IRA Response Dendrogram 50 -

Dendrogram Group Low Medium High Colour Code

40 Closeness

Hierarchical clustering, or hierarchical cluster analysis (HCA), divides a dataset into clusters iteratively a n d c re a t e s a t re e - l i ke structured dendrogram. The dendrogram is then used to explain the relationship between all data points in the sample.

30 20 10 -

A

7

2

-

B

-

5

1

C

3

2

2

0AI [ Exhibit 26 ] IRA Response dendrogram and hierarchical cluster groupings

Hierarchical Clustering Group Attributes Indicators (on average) Equity Deposits from customers Overall Percentage Risk

Group A $3.9bn $48.8bn 54.4

Group B $51.6bn $301.2bn 90.3

Group C $14.5bn $83.2bn 76.5

# Third parties # in-house application # EOL System # OSS no commercial support # Network device # Branches # Employees Social Media Presence # Cards issued # P2P cosmotrons # Treasury clients # Correspondent banks # Entities the AI act as a merchant acquirer for

3.9 4.8 1.4 0.0 76.8 8.9 289.0 0.1 2,000 0.0 3,000 110 90

5.8 9.7 4.7 0.8 1311.5 43.7 2106.7 1.5 466,000 70,000 45,000 490 200

6.5 25.0 4.2 2.8 354.3 15.2 872.2 1.3 43,000 3,000 3,000 30 400

[ Exhibit 27 ] Hierarchical clustering group attributes

34

INHERENT RISK ASSESSMENT (IRA)

Technologies

Group A Average

Tracked Records on Cyber Threats

0.29

0.47

0.42

Delivery Channals

0.27

0.50 Organizational Characteristics

Products and Technology Services

Technologies

Group B Average

Tracked Records on Cyber Threats

0.51 0.79

0.54

0.63

Delivery Channals

0.63

Organizational Characteristics

Products and Technology Services

Technologies 0.54 Group C Average

Tracked Records on Cyber Threats

0.58

0.54

0.53 Organizational Characteristics

[ Exhibit 28 ] Hierarchical cluster grouping risk score

35

Delivery Channals

0.39

Products and Technology Services

Correlation of Risk with AIs’ Business Nature and Target Customers By examining the detailed responses in the survey, we have identified a few key statistics that might help us endow meaning to the three clusters. Exhibit 27 shows a full table of comparisons. An immediate observation we can make from the data is that, on average, Group B is the largest amongst the three groups, followed by Group C. The former is the largest in terms of multiple monetary measures, the number of branches, staff, and customers. Next, Group C is mostly defined by the high number of third-party, in-house applications, without commercial support Open-Source Software (OSS) and End-of-Life (EOL) systems, signalling that their risks are mainly from their weakness in managing technical risk. Intriguingly, Group A is the smallest on average by almost all measures. AIs in this group have only issued a small number of payment cards and have a weak presence in the P2P transaction space, yet they are almost as competitive in the treasury service space as Group C. The observation above indicates that Group A is mostly small-sized retail banks with a low number of branches and employees. They also have a limited outreach in terms of their digital presence and service provision. On the other hand, Group B appears to be large banks with mature internal technology systems and high outreach in their internet presence and P2P transaction. The high average deposit from customers indicates they might have a certain degree to support higher net worth customers or corporate clients. Meanwhile, Group C, being smaller than Group B but with a noticeably higher presence as a merchant acquirer, could be AIs that focus on the payment system side and support businesses on payment solutions.

Overall, the clustering result reveals that an AIs’ overall risk might be closely related to its business nature and target audience, which are not directly captured in the IRA framework.

While we cannot conclusively suggest how the type of AIs impact their risk, the analyses above have provided evidence for such possibilities.

36

INHERENT RISK ASSESSMENT (IRA)

FACTOR ANALYSIS

Exploratory Factor Analysis is a tool for investigating variable relationships.

The key concept is that multiple observed variables have similar patterns o f re s p o n s e s b e c a u s e t h ey a re a l l associated with a latent (i.e., not directly measured) variable.

We used a scree plot to help us determine how many factors we should use. The eigenvalue measures the explanatory power of the factor. Essentially, a factor with an eigenvalue greater than one explains more variance than an observed variable.

Scree Plot

Eignvalue

6 4 2 0

2

4

6

8 10 12 14 Factor

While the IRA process provides a comprehensive view of the AIs’ cybersecurity risk exposure, the number of indicators presents a challenge in generalising the statistical findings and insights. To better understand the root cause of the inherent risk and identify any underlying factors that drive an AI’s cybersecurity risk, we have utilized exploratory factor analysis (EFA).

Ideally, we would conduct EFA on all the 51 indicators and have the algorithm determine a set of latent risk factors. However, limited by the sample size, we could not simply feed all the indicators into the algorithm, as the “curse of dimensionality”4 impacts the validity of our EFA variables. Therefore, we substitute the 51 indicators with the 15 grouped sub-domains provided in the previous section.

The sub-domain score is calculated by summing up all the indicators scores. For example, there are three indicators under the sub-domain “Cyber Threat Tracked Records – Frequency”. If an AI reported medium risk for two of the indicators and high risk for the remaining one indicator, it would have a score of 2 × 2 + 3 × 1 = 7 for this sub-domain.

After obtaining all the sub-domain scores, we fitted a factor model with orthogonal rotation and drew a scree plot to determine the number of latent factors. As the correlation result below shows, we expect the EFA to help us identify and distil the sub-domain even further.

[ Exhibit 30 ] Factor analysis scree plot Below is an illustration of observed variable to factor mapping.

Observed Variables

Payment Card Risk

Factor 1

Factor 2

Factor 3

Factor 4

1

Payment Card Risks

Internet Presence

Cybersecurity Staffing Risks

Cyber Threat Variety

2

Mobile Presence Risks

Fund Transfer Risks

Cyber Threat Frequency

3

ATM Provision

Size Risks

4

Size Risks

Client Services Risks

5

Internal Risks

Other ServiceRelated Risks

Factors

Mobile Presence ATM Risk

Variable

Payment Provision Risk

Size Risk [ Exhibit 31 ] Factor analysis illustration

Factor 1 Factor 2 Factor 3 Factor 4

Payment Provision Risk Banking Services Risk Cyber Threat Risk Attack Variety Risk

[ Exhibit 29 ] Factor analysis variable groupings Note: See appendix 3 for the full factor loading score table.

4.

37

Since we have more features than observations (51 versus 22), we could be exposed to the risk of overfitting our model.

Underlying Factors of Inherent Risks To deduce what each factor represents, we examined the sub-domains with the highest correlation scores.

First, for the left-most factor, the most dominant risk indicator group is the Payment Card Risks, followed by ATM and mobile presence risk. This suggests that the first factor is highly likely to represent the payment provision risk of an AI. Moving on to factor 2, we can see that the firm’s fund transfer services, internet presence, and size are the top three sub-domains explaining this factor, indicating this factor is related to an AI’s fund transfer and banking service provision.

Next, factor 3 seems to be explained by the cyberattack frequency and the inverse of their cybersecurity staffing risk. Finally, the fourth factor appears to be largely about the cyberattack variety risk.

We then regenerate IRA plots using the extracted risk factors. The following chart shows that the risk score between high- and medium-risk AIs have been pulled much closer after the analysis.

Yet, the distribution of risk appears to align with the observations we made in distinguishing the AIs. More particularly, we see a strong presence in payment provision for both medium- and high-risk AIs, while high-risk AIs offer more other banking-related services.

Risk Score Comparison On The Four Factors Low

Medium

High

35 30 25 20 15 10 5 0

11 23 22

24 32 34

7 8 6

5 7 7

Payment Provision Risk

Banking Services Risk

Cyber Threat Risk

Attack Variety Risk

[ Exhibit 32 ] Inherent risk score comparison between the four latent factors

The grouping above forms a coherent picture of our findings in the sections above, where we found that type of services is particularly salient in explaining the overall risk of the AIs (factor 1) while confirming our observation that cybersecurity staffing risk is inversely related to cyberattack frequency risk. Moreover, it reveals that the banking and fund transfer services of the AIs, together with their internet presence, have helped contributed the second most of the risk level as a group. The latent factors above reveal the underlying cause of cyber risk and demonstrate that indicators across each domain can be highly correlated.

38

INHERENT RISK ASSESSMENT (IRA)

INHERENT RISK ANALYSIS: SUMMARY

As described in this section, we have conducted a thorough investigation on the Inherent Risk Assessment results of the 22 AIs. In summary, when comparing the AIs with the classification rule set out in the IRA framework, we found that “Technologies” is the main difference between high- and medium-risk AIs, which is mainly driven by third-party risk, with a difference of 47 percentage points, and the largest gap in any sub-domains. We also found that low-risk AIs differ from their riskier peers due to low risk in “Products and Technology Services”, driven by the narrower range of services the low-risk AIs provide.

Comparing the AIs by their equity size, we found a similar risk score as the one under risk classification rule. Nevertheless, there are a few interesting observations. More specifically, we found that the “Technologies” risk score for large AIs is lower compared to medium-sized AIs, suggesting that largest AIs have put in some effort to reduce their technology-related risk. The similarity in risk profile suggests that the IRA classification rule is effective in classifying the AIs, based on our finding that the size of an AI correlates with their risk level.

Next, the key takeaway of the service provision analysis is that many of the indicators are interconnected despite being in different domains. For example, mobile risk presence is highly related to payment card services, which means AIs who issue payment cards often have an accompanying mobile application for their clients. This has led us to investigate the unobserved latent factors on the survey responses using Exploratory Factor Analysis. Using EFA, we found that the risk specific to our 22 AIs is mainly caused by banking and payment service provision, which is correlated to internal and size risk areas.

When we clustered our AIs together by using the hierarchical clustering algorithm, we found three distinct groups. Essentially, the findings form a coherent picture with the factor analysis results. The three groups show differences in characteristics in their internal technology risk, size risk, and service provision risk.

Overall, we believe that the IRA has helped the AIs identify their risk. The classification rule provides a clear guideline to the AIs while also revealing some interesting characteristics about the surveyed AIs. As an extension of our study, we also investigated the classification rule of the C-RAF 2.0.

39

C-RAF 2.0 CLASSIFICATION

HKMA released a revised version of the Cyber Resilience Assessment Framework (C-RAF 2.0) in November 2020. Here we would like to take a quick look at how our findings are impacted if we use the updated classification rule: “If the number of Low risk assessment criteria is less than or equal to the total number of Medium and High risk level, the inherent risk level should be adjusted to Medium.”

AIs' Risk Class Population

Low

Medium

High

Low

Medium

High

14%

36%

50%

[ Exhibit 33 ] AIs’ risk class population under the updated classification rule

Overall Risk Score by Banks and Risk Classification C-RAF

|

C-RAF 2.0

|

Overall Risk Score |

40

60

80

100

[ Exhibit 34 ] AIs’ risk score under the C-RAF 2.0 classification rule

After applying the additional IRA classification rule, two out of ten low-risk AIs are upwardly adjusted to medium risk. As the scatter chart above indicates, the two AIs score the highest within the low-risk AIs in terms of overall risk score. This indicates that the additional rule is able to better reflect the overall risk score in this scenario.

40

MATURITY ASSESSMENT (MA)

MATURITY ASSESSMENT (MA)

41

OVERVIEW

The Maturity Assessment (MA) covers seven key domains, with over 360 controls across three maturity levels, which are: “Baseline”, “Intermediate”, and “Advanced”. The MA is designed to provide a comprehensive review of the operating environment and emphasizes a sound governance framework.

Surveyed AIs' Maturity Assessment Population Advanced 14%

Intermediate 41%

Baseline 45%

[ Exhibit 35 ] Maturity Assessment population of surveyed AIs

As introduced in HKMA’s C-RAF consultation paper, the seven domains are split into three levels, depicted in Exhibit 36. The centre represents the governance aspect; the inner circle consists of areas of controls related to the internal environment; and lastly, the outer circle shows the controls related to the external environment.

The level of maturity required for each AI depends on the risk classification of the Inherent Risk Assessment. For low-risk AIs, they need to attain the “Baseline” maturity level as a minimum requirement. Correspondingly, medium-risk AIs need to attain an “Intermediate” maturity level, and the “Advanced” level is required for high-risk AIs. In general, the risk controls adopted by all AIs are consistent with their inherent risk level, where higher-risk AIs adopt more controls.

S it

ua

n

Pr

ot

Percentage Attainment

Low

Medium

High

Baseline

88.0%

90.7%

99.6%

82.8%

100.0%

Intermediate

n

Ide

ti o

nt

if

io at

ess

ec

ic

Awaren ti o n a l

Advanced

nd Re cove ry

99.1% Not Required

te cti on

a nse Respo

Governance

Partially Attained

De

t Th en ird em Party Risk Manag [ Exhibit 36 ] The seven domains of the maturity assessment5

All Requirements Met [ Exhibit 37 ] Percentage maturity control attainment by risk class

While the AIs can respond to parts of the survey they are not required to, we only conducted analyses on those assessments

the AIs are subjected to. Therefore, this section will only report the “Baseline” control responses for low-risk AIs, both “Baseline” and “Intermediate” controls for medium-risk AIs, and finally, all three “Baseline”, “Intermediate”, and “Advanced” controls for high-risk AIs.

5.

HKMA (2016). Cyber Resilience Assessment Framework Consultation Draft.

42

MATURITY ASSESSMENT (MA)

An AI’s level of attainment of the maturity level is calculated by the number of “Yes”, “Alternative Control”, “Risk Accepted”, and “Not Applicable” responses as the percentage of total numbers of control. As evidenced by Exhibit 37, most AIs have identified gaps in their cybersecurity measures.

For risk controls that are not accomplished, surveyed AIs responded with the answer “No”. The average number of “No” under the “Baseline” control given by low-risk AIs is 17.6. Only one lowrisk AI out of ten successfully fulfilled all the “Baseline” controls. On the other hand, mediumrisk AIs noted an average of 17.1 “No” responses under the “Baseline” control and 15.4 at the “Intermediate” level. Of the nine medium-risk AIs, only one has met all the requirements. Highrisk AIs have performed the best under the “Maturity” Assessment. Of the three high-risk AIs, two of them have completed all the required controls, while the remaining one has failed in three “Baseline” controls and two “Advanced” controls.

Risk Profile in Different Environments A quick examination of the AIs’ risk profile in terms of their Governance, Internal Environment, and External Environment would reveal that low-risk AIs struggle the most under the Governance aspect. This implies that the low-risk AIs commonly fail to take suitable measures in the oversight, strategy, and policy aspects. Meanwhile, medium-risk AIs have struggled the most under the external environment category, implying these AIs perhaps lack good threat intelligence or have issues managing third-party risk. Lastly, high-risk AIs are mostly on target under all three categories, meaning these AIs are rather balanced and well-rounded in preventing losses from cyber threats.

LOW RISK CHART LEGEND External Environment Internal Environment Governance 83% 89% 90%

MEDIUM RISK RISK

HIGH RISK

88% 88% 78% [ Exhibit 38 ] Risk profile in different environments by risk class

43

99% 100% 98%

Seven Domains of Maturity Assessment Each domain comprises a number of components. Each component has a set of control principles (controls) for AIs to determine if they achieve the required level or extent of implementation for attaining a particular maturity level of that component.

Governance

The“Governance" domain contains 81 controls across 5 components as defined by the HKMA. This domain aims to encompass measures related to cyber resilience oversight, cyber risk management, as well as audit and aspects related to staffing, training and policies.

Identification

“Identification" is one of the four internal environments related domain, alongside "Protection, "Detection", "Response and Recovery". In particular, the "Identification" domain consists of 23 controls across 2 components related to aspects such as IT assets identification and cyber risk identification and assessment.

Protection

There are 6 components covering 106 controls under the "Protection" domain, the highest number of all seven domains. These controls are related to aspects such as patch management, data security, infrastructure protection and access control.

Detection

The "Detection" domain encompasses 4 components which cover a total of 60 controls. The four components are vulnerability detection, anomalies activity detection, cyber incident detection and finally, threat monitoring and analysis.

Response and Recovery

"Response and Recovery" is a maturity domain composed of 51 controls across 3 components. The maturity controls in this domain are related to response planning, incident management and lastly, escalation and reporting.

Situational Awareness

"Situational Awareness" is one of the two domains under the External environment umbrella, alongside "Third-party Risk Managament".

Under the "Situational Awareness" domain, there are only 18 controls, belonging to either the threat intelligence or the threat intelligence sharing component.

Third-party Risk Management

As the name implies, the "Third-party Risk Managament" domain consists of 27 controls across 3 components related to various aspects of third-party risk related measures. These include the ongoing monitoring of third parties, external connections and other third party related measures.

[ Exhibit 39 ] Introduction to the seven maturity assessment domains

44

MATURITY ASSESSMENT (MA)

In the following section, we will examine the Maturity Assessment responses, assess the landscape of cyber risk control maturity level of the AIs, and investigate the common weakness for improvement. Similar to our work in the Inherent Risk Assessment analysis section, we will examine the domain level, the component (i.e. subcategory) level and the individual control level in an attempt to identify the cause of gaps in their maturity level.

In short, the results align with the statistics above, where we found high-risk AIs perform the best while medium-risk AIs are the worst. The underperformance is partly driven by their low attainment rate under the “Third-party Risk Management” domain, which could be due to the relatively low Third-party Risk as shown in the IRA analysis. Nonetheless, when we used the MA results to check against the AIs’ actual security measures, we found that the results are somewhat effective in understanding the real-life implementation of SSL certifications of the AIs.

Domain The scope of the maturity assessment covers seven key domains. These domains are categorized in three levels: governance, internal environment, and external environment.

Component Each domain comprises a number of "components", the relationship is defined by the official Maturity Assessment framework.

Control The framework has also set out a number of "control principles" which are divided in to different maturity levels.

Baseline: 1. IT Asset Management 2. IT Configuration Management IT Asset Identification

Intermediate: 1. IT Asset Management 2. IT Configuration Management Advanced: 1. IT Configuration Management

Identification Baseline: 1. Cyber Risk Identification 2. Assessment Scope Cyber Risk Identification And Assessment

Intermediate: 1. Cyber Risk Identification 2. Assessment Scope Advanced: 1. Cyber Risk Identification 2. Assessment Scope

[ Exhibit 40 ] Maturity control grouping example

45

SURVEY RESULT

With the components and controls of the maturity assessment in mind, we began to analyse the performance of different risk classes. Exhibit 41 presents a chart that allows us to examine if there are any areas of improvement for AIs in various risk classes. Note the percentage is calculated in terms of the total number of controls on each maturity level for every domain. For example, under the “Governance” maturity assessment domain, there are a total of 30 baseline controls. If an AI attained 27 of which, it would achieve a score of 90%.

Again, we only show the results for the required maturity level for each group of AIs; hence there are only high-risk AIs in the “Advanced” section, and only medium and high risks in the “Intermediate” section.

Maturity Assessment Domain Results Governance Situation Awareness

Identification

Protection

Detection

Response and Recovery

Third-party Risk Management

% 100 90 80 70 60 50 40 30 20 10 0

Low

Medium

High

Baseline

Medium

HIgh

Intermediate

HIgh Advanced

[ Exhibit 41 ] Maturity assessment domain result comparisons

Key findings: 1. High-Risk AIs Outperform The previous section shows that high-risk AIs have adopted almost all controls across the three maturity levels. More specifically, only one high-risk AI does not meet all the requirements. Yet, the AI only has a slight shortfall in “Governance”, “Protection”, and “Situation Awareness” and has stated reasonable timelines to fill in the gaps to mitigate those risks. This suggests that high-risk AIs have proactively taken actions to minimize the cyber risk induced by their day-today operations and characteristics and are fast-acting in closing any gaps. This echoes our finding in the Inherent Risk Assessment, where high-risk AIs face lower cyber staffing risk.

2. Most Medium-Risk AIs Fail to Meet the Target By contrast, medium-risk AIs perform less well, with an overall attainment rate of 87% across the Baseline and Intermediate maturity levels. This group of AIs has underperformance in “Governance”, “Protection”, and “Third-party Risk Management”. On average, they have failed to meet six or more controls in the domain above across the two required maturity levels. In particular, the lack of third-party controls might be caused by their low exposure to third-party

46

MATURITY ASSESSMENT (MA)

risk, as seen in the Inherent Risk Assessment section, where the average medium-risk AIs are only seven percentage points riskier than their low-risk peers. Although low-risk exposure in a certain area is no excuse for omitting adequate measures, it also highlights that AIs might prioritize areas where they face a higher risk when resources are scarce. Regulators could perhaps keep this in mind when adjusting a later iteration of the survey assessment or provide incentives for the AIs to bridge those gaps as a precautionary measure.

3. Low-Risk AIs See Mixed Result Across Domains For low-risk AIs, they are mostly on target in terms of “Protection”, “Situation Awareness”, “Response and Recovery”, and “Third-party Risk Management” but have failed to meet expectations under all three other domains. This again signifies that many low-risk AIs have not met the minimum standard and underperformed in terms of internal environment and governance control.

CONTROLS AND SUB-DOMAIN ANALYSIS

As the maturity assessment covers a wide range of cybersecurity measures, it allows us to identify the common weaknesses and strengths of the responding AIs. Below are some of the best- and worst-performing maturity components out of the total 25 components. The percentage score is calculated by the average attainment rate across the maturity controls inside the components, and only the required maturity controls according to the AIs risk class are considered.

Data Security Incident Management Esclation And Reporting Access Control Threat Monitoring And Analysis Response Planning %0

99.1% 97.3% 96.7% 96.5% 96.3% 95.8% 94

95

96

97

98

99

100

[ Exhibit 42 ] Best performing maturity components

Key findings: 1. Near Full Attainment in Data Security The data security control has an attainment rate of over 99%, indicating that all surveyed AIs have taken as much control as possible to safeguard their sensitive information and data.

Controls related to end point data security, data protection, and data disposal are of high importance to the AIs, allowing them to avoid data breaches. In fact, CybelAngel, a digital risk protection organization, has found that over 90% of data breaches are due to negligence, and by enforcing such data protection measures, the AIs could greatly reduce potential losses related to data breaches.

2. The AIs Have Performed Particularly Well in Components Under “Response and Recovery” We notice that half of the top six performing maturity components are related to the “Response and Recovery” domain. The components are Incident Management, Escalation and Reporting, and Response Planning. This shows that most of the surveyed AIs have generally performed

47

well and are quite all-rounded in implementing follow-up measures in case of a cybersecurity incident. This is a somewhat surprising finding as we see in the previous section, medium-risk AIs struggle with controls under this domain.

Upon inspecting individual maturity control results, we found that the “Incident response to widely reported events”6 control under the “Response and Recovery – Response Planning” component has the single most negative answers out of all 365 controls. The main reason AIs have underperformed in this domain perhaps has to do with the relatively strict condition of this control, where an AI has to meet three different categories to warrant a “Yes” response.

Audit Anomalies Activity Detection Cyber Risk Identification and Assessment External Connections Ongoing Monitoring On Third-party Risk Third-party Management %0

90.8% 90.8% 89.1% 88.2% 84.9% 82.9% 78

80

82

84

86

88

90

92

[ Exhibit 43 ] Worst-performing maturity components

Key findings: 1. Over 80% Attainment Rate for all 25 Maturity Components We notice at least 80% attainment rate for all 25 maturity components, which is encouraging.

21 of the 25 components achieve over 90% average attainment rate, and only four components’ with an attainment rate below 90%.

2. Components Under The “Third-party Risk Management” Maturity Domain Performed the Worst Components under the “Third-party Risk Management” domain appear to be the worst of the underperformers. The three components in this domain (External Connections, Ongoing Monitoring on Third-party Risk, and Third-party Risk Management) scored the lowest in the level of attainment by the surveyed AIs. 3. Underperformance in “Third-party Risk Management” Due to Low Attainment in Individual Controls We plotted the attainment rate for the worst-performing controls out of the total 365 maturity controls and have found that the underperformance in “Third-party Risk Management” appears to be due to a wide base underperformance where many individual risk controls show a low attainment rate. This is particularly significant for the “Third-party Risk Management” as it only has 27 controls, some of the lowest in the seven domains.

6.

Definition of “Incident response to widely reported events”: If widely reported events such as massive destruction or alternation of data are used to improve incident detection and response.

48

MATURITY ASSESSMENT (MA)

Average Attainment Percentage (Maturity Control) 55% 55% 64% 67% 68% 68% 68% 70% 71% 71% 71%

Incident response to widely reported events DNSSEC Deployment Security Information Thresholds On-going Third-party Risk Assessment Process to Correlate Event Information Anomalies Activity Detection Thresholds Additional Training for Privileged Users Third-party Due Diligence Before Contract Third-party Responsibility for Notification On-going Third-party Review Security Incidents Third-party Responsibility Establishment %0

10

20

30

40

50

60

70

80

[ Exhibit 44 ] Average maturity control percentage attainment

If we examine the detailed responses, where AIs who have failed to meet the requirement for a control would note down the gap they noticed, we have found that 8 out of 10 AIs responded with a negative answer have indeed partially adopted relevant measure. This is a similar case to many other controls in the Maturity Assessment, where we have found that of the top five controls with the most “No” answers, four of them have been partially adopted by at least half of the AIs. We believe the Maturity Assessment could benefit from a similar three-level system to the Inherent Risk Assessment. Instead of allowing only binary “Yes” and “No” answers, the AIs could indicate a partial adoption of the measures, which could better reflect the implementation status of the vast maturity controls.

Top 5 controls by non-adoption rate Control

Total No(s)

Partially Adopted

Not Adopted/ No Formal Procedure

Widely Reported Incident Response

10

80%

20%

DNSSEC Deployment

9

11%

89%

Security Information Thresholds

8

75%

25%

Ongoing Third-party Risk Assessment

7

57%

43%

Process to Correlate Event Information*

7

43%

43%

* One response was missing [ Exhibit 45 ] Partial adoption of maturity controls

Despite the observation above, we believe the Maturity Assessment not only allows HKMA to understand the gaps in the AIs’ practice better, but it also acts as an opportunity for the AIs to conduct a thorough check on their cyber risk profile. Over 90% of participating AIs found the C-RAF useful, especially in identifying previously unrecognized gaps.7

7.

https://www.moodysanalytics.com/regulatory-news/Nov-03-20-HKMA-Enhances-Cybersecurity-Fortification-Initiative

49

RESPONSES IRA: LOW CYBERSECURITY STAFFING RISK FOR HIGH-RISK AIs

In the IRA section, we found that high-risk AIs face a lower risk related to cybersecurity staffing, here we investigate the detailed responses between high-risk AIs and their peers.

More particularly, we examine controls under the Staffing and Training component in the larger umbrella of the “Governance” domain.

Risk Class

Staffing and Training Component Average Attainment Rate

Low

87%

Medium

89%

High

98%

[ Exhibit 46 ] Staffing and training maturity component attainment rate

A quick inspection would reveal that the highrisk AIs have notably higher attainment under the staffing and training component. This confirms our observation in the IRA section that high-risk AIs have put more effort into investing in their cybersecurity and related staff. More specifically, the high-risk AIs have excelled in controls that are still struggled with by many other lower-risk AIs. For example, in acquiring professionals with adequate qualifications and providing annual training to staff with privileged access.

The similar findings in both IRA and MA sections have demonstrated that the two selfassessment exercises work coherently together, and the AI’s answers are consistent across both sections. AIs and regulators could then utilize the survey to effectively identify gaps between associated risk and maturity control areas, therefore presenting a tool that allows AIs to target areas in which they notice the biggest gap.

MATURITY ASSESSMENT: SUMMARY

In this section, we have investigated the Maturity Assessment survey responses using a topdown approach. We first investigated the overall results across the three risk classes of AIs, based on the maturity requirements they are subjected to and then down to the domain component and control levels.

We notice that high-risk AIs have performed particularly well in the Maturity Assessment despite being subjected to the highest number of maturity controls. On average, this AI group achieves over 99% of controls at all the three maturity levels and even completely meet all the requirements at the Intermediate level. On the other hand, the medium-risk AIs have underperformed, achieving only 91% and 83% of controls at the Baseline and Intermediate level respectively. Meanwhile, low-risk AIs have also underperformed at the Baseline level, and on average they achieve less than 90% of their required controls.

An examination of AIs’ risk profile in terms of Governance, Internal Environment, and External Environment reveals that low-risk AIs struggle the most under the Governance aspect, implying that they commonly fail to take the required measures in aspects related to governance and oversight. Meanwhile, medium-risk AIs have underperformed in the external environment category, indicating these AIs perhaps lack good threat intelligence or have issues managing third-party risk. Finally, high-risk AIs are mostly on target as expected. This reveals that high-risk AIs are rather well-rounded in preventing losses from cyber threats.

50

MATURITY ASSESSMENT (MA)

At a domain level, we found that the slight shortfall in the high-risk AIs comes from a single AI, who falls short slightly in the “Governance”, “Protection”, and “Situation Awareness” domains, but has established a reasonable timeline to implement the missing measures. By contrast, medium-risk AIs perform worse in “Governance”, “Protection”, and “Third-party Risk Management”. For low-risk AIs, they are mostly on target in terms of “Protection”, “Situation Awareness”, “Response and Recovery”, and “Third-party Risk Management” but have failed to meet expectations under the other three domains. This again signifies that many low-risk AIs have not met the minimum standard and underperformed in the internal environment and governance controls.

Next, we seek to understand the best and worst components out of the total 25 components.

We found that none of the components is fully adopted by all the AIs. Still, the Data Security component is by far the best performing with an attainment rate of over 99%, suggesting the AIs have taken as much control as possible to safeguard their sensitive information and data.

All the components reached at least 80% attainment rate, with the three worst components being from the “Third-party Risk Management” domain. Upon further investigation, we found it is mainly driven by the fact that many of the infidel controls in the “Third-party Risk Management” domain have a low attainment score. Almost half of the twelve worst performing controls are in this domain.

Finally, we cross-checked a finding from the IRA section to draw links between the Inherent Risk Assessment and the Maturity Assessment. In particular, in the IRA section we found that AIs with a “High” overall risk experience the lowest cybersecurity staffing risk. We believe they have conducted the adequate measure in staffing and training their staff. To check if the findings in the maturity assessment align with the abovementioned findings, we investigated a particular maturity component specific to cybersecurity staffing and training. As we expected, we found that the high-risk AIs excel in that component, which is consistent with our IRA finding. The two self-assessment exercises work coherently together, and the AI’s answers are consistent across both sections.

51

INHERENT RISK AND MATURITY

52

OVERVIEW

While the rule for minimum maturity attainment gives a clear and concise guideline to AIs, we could expand the idea further by examining the relationship between risk and maturity. As more cybersecurity efforts are required for the riskier entities, we hope to see a positive relationship between an AI’s risk and its maturity, or better yet, AIs with low risk would still attain a high maturity level to minimize chances of cybersecurity-related losses. Here, we have calculated the percentage maturity attainment for the AIs and plotted it against the overall risk score, reflecting each AI’s level of maturity under their required measures.

AI Inherent Risk and Maturity Score

Low

Medium

High

400 Maximum Maturity Score

350

Maturity Score

300

250

200

150

100

40

60

80

100

Inherent Risk Score [ Exhibit 47 ] Relationship between inherent risk and maturity score

As shown in the figure above, there exists a positive relationship between the AIs’ inherent risk score and their maturity, which is a reassuring sign. To further investigate how each maturity domain varies with respect to the AIs’ inherent risk, we have plotted the seven domains (y-axis) against the overall risk scores of the AIs (x-axis). In Exhibit 48, we have also added the risk class average as a reference to allow a clear comparison between the risk class. In addition, we have drawn dotted lines to indicate the maximum number of control principles for each maturity level. The blue line represents the Baseline level, the yellow line for Baseline and Intermediate level, and the red dotted line acts as the maximum possible value, i.e. the number of indicators in a domain. The AI should reside on its corresponding dotted line and in fact above its corresponding dotted line in the ideal scenario.

53

INHERENT RISK AND MATURITY

Low

Governance

High

Benchmark

Identification

100

High Risk Average Maximum Maturity Score

20 Identification

Governance

Medium

Baseline + Intermediate

50 Baseline

10

Medium Risk Average Low Risk Average

0

50

0

100

50

Inherent Risk Score

100

Inherent Risk Score

Protection

Detection 60 50 Detection

Protection

100

80

40 30

60 20 40

50

100

Inherent Risk Score

50 Inherent Risk Score

[ Exhibit 48 ] Relationship between the seven maturity domain and inherent risk score

54

100

Low

Response and Recovery

Situational Awareness

Response and Recovery

Benchmark

20

40

20

15

10

5 50

100

Inherent Risk Score

30

20

10

50

100

Inherent Risk Score

(cont.) [ Exhibit 48 ] Relationship between the seven maturity domain and inherent risk score

55

50

100

Inherent Risk Score

Third-party Risk Management

Third-party Risk Management

High

Situational Awareness

60

0

Medium

Key findings: 1. Align with General Trend - Higher Risk Score Often Accompanies with Higher Maturity Score Similar to the main scatter plot at the beginning of this section, a high overall risk tends to mean a higher maturity score in each domain. They indicate to us that for all the Maturity Assessment domains, riskier AI would have taken the corresponding actions in each domain.

2. Most Low- and Medium-Risk AIs Fail to Reach Target on a Domain Level As seen in Exhibit 48, only about half of non-high-risk AIs manage to reach or get close to their corresponding target for each domain. This suggests that the underperformance of the groups is often caused by a group of AIs failing to reach target instead of a few anomalies. More worryingly, some medium-risk AIs perform worse than low-risk AIs despite having a higher number of maximum controls, particularly in “Third-party Risk Management” and “Identification”.

Fortunately, the MA exercise includes fields for AIs to set deadlines to implement these measures. Regulators could utilize this information and investigate the underlying reason for such underperformance and provide support and guidance to these AIs.

INHERENT RISK AND MATURITY

DOMAIN COMPARISON

To better understand if the AIs have put sufficient resources into addressing various risks they face, we created various scatterplots based on the Inherent Risk and Maturity Assessment Domains. We first examine the relationship between the AIs’ “Technologies” and “Cyber Threat Track Records” risk against the “Identification”, “Protection”, “Detection” and “Thirdparty Management” maturity domains. We focus on these four domains in particular as they are deemed the traditional or typical steps advocated for cybersecurity risk management, as depicted in cybersecurity risk management frameworks such as the US National Institute of Standards and Technology (NIST) Security Framework.8

DIFFERENT PERSPECTIVES IN MEASURING RISK VS REQUIRED MATURITY Group A (1) - Technologies Risk Score versus Four Traditional Maturity Domains Maximum Maturity Score

100 High Risk Average

Baseline + Intermediate

10

0

Baseline

Medium Risk Average

Protection

Identification

20

60

Low Risk Average

10

20

80

40

30

10

Technologies Risk Score

30

Technologies Risk Score

60 Response and Recovery

60

Detection

20

40

20 10

20 Technologies Risk Score

30

40

20 10

20

30

Technologies Risk Score

[ Exhibit 49 ] Group A scatterplots – “Technologies” risk score versus four traditional maturity domains 8.

There are five domains under the US National Institute of Standards and Technology (NIST) Security Framework, which are “Identify”, “Protect”, “Detect”, “Respond”, and “Recover”. The five domains above corresponds to four of our Maturity Assessment domains, namely “Identification”, “Protection”, “Detection” and “Respond and Recovery”.

56

Group A (2) - Tracked Records on Cyber Threats Risk Score versus Four Traditional Maturity Domains Maximum Maturity Score

Baseline + Intermediate

10

Medium Risk Average

Baseline

Protection

High Risk Average

20 Identification

100

60

Low Risk Average

0

40

15 10 Technologies Risk Score

15 10 Technologies Risk Score

60 Response and Recovery

60

Detection

80

40

20 15

10

Technologies Risk Score

40

20 10

15

Technologies Risk Score

[ Exhibit 50 ] Group A scatterplots – “Tracked Records on Cyber Threats” risk score versus four traditional maturity domains

By examining Exhibits 49 and 50, we see that the overall trend persists, where high-risk AIs often perform better in the various maturity domains. Interestingly, we also observe that, on average high-risk AIs are more mature compared to medium-risk AIs under the four traditional maturity control domains, despite having better track records in terms of their cybersecurity threats. The above finding reinforces that AIs facing low risk in certain areas are not excused from implementing the appropriate measures and likely indicate that AIs could see lower cyber threats levels as they implement tougher security measurements.

Next, we take an organization and governance perspective by plotting the AIs’ “Product and Technology Services Risk” and “Organizational Characteristics” against their maturity level in “Governance” and “Situation Awareness.” This grouping allows us to identify gaps between the AIs’ risk induced from organizational planning and the measures taken at an organizational management level.

57

INHERENT RISK AND MATURITY

Group B (1) - Products and Technology Services versus Governance and Situational Awareness 20

100 Situational Awareness

Governance

Maximum Maturity Score High Risk Average Baseline + Intermediate

50 Medium Risk Average

Baseline

Low Risk Average

0

20

10

15 10 5 0

30

20

10

Products and Technology Services

30

Products and Technology Services

Group B (2) - Organizational Characteristics versus Governance and Situational Awareness 20 Situational Awareness

Governance

100

50

0 10

15 Organizational Characteristics

30

15 10 5 0

15

10

30

Organizational Characteristics

[ Exhibit 51 ] Group B scatterplots – “Products and Technology Services” and “Organizational Characteristics” versus “Governance” and “Situational Awareness”

By examining the Group B plots, we noticed while high- and medium-risk AIs have the same average score under the “Products and Technology Services” risk domain, they have adopted more controls in the “Governance” and “Situational Awareness” maturity domain, both in terms of the total number of required controls and percentage of controls. Meanwhile, results in the two bottom charts with “Organizational Characteristics” are mostly in line with our expectations.

The number of controls adopted was mostly consistent within each group, with a few mediumand low-risk AIs failing to meet their targets. Nevertheless, it also allows us to identify that some medium- and low-risk AIs have adopted a high number of required controls despite a low “Organizational Characteristics” risk score.

We have plotted a final set of charts on all the five domains of the AIs and “Response and Recovery” maturity domain. This grouping shows the performance of the AIs in terms of their

58

risk against how well they can minimize or mitigate losses arising from potential cybersecurityrelated incidences. For riskier AIs, we wish to see a higher score in “Response and Recovery”, which signals that they could better respond to their cybersecurity flaws or vulnerabilities.

Otherwise, inappropriately reacting to cybersecurity incidents could deepen losses.

Group C - All Five Inherent Risk Domains versus Response and Recovery Maturity Domain High Risk Average Maximum Maturity Score

40

Baseline + Intermediate

Medium Risk Average

Low Risk Average

20

Baseline

10

60 Response and Recovery

Response and Recovery

60

40

20

30

20 Technologies

Delivery Channels 60 Response and Recovery

60 Response and Recovery

10

5

40

20

0

10

30

20

Products and Technology Services

40

20

10

15 Organizational Characteristics

Response and Recovery

60

40

20

15

10

Tracked Records on Cyber Threats [ Exhibit 52 ] Group C scatterplots – All inherent risk domains versus the “Response and Recovery” maturity domain

59

20

INHERENT RISK AND MATURITY

The Group C plots (Exhibit 52) have revealed that high-risk AIs have met all controls on postevent risk mitigation while most other AIs have failed to do so. This is an interesting observation as high-risk AIs experience same or lower risk in “Delivery Channel”, “Products and Technology Services”, and “Tracked Records on Cyber Threats” compared to medium-risk AIs. It appears that the main motivation for such increased controls could be largely driven by their higher technology and organizational characteristics related risks.

INHERENT RISK AND MATURITY ANALYSIS SUMMARY

As mentioned at the beginning of this report, the C-RAF framework is unique because it offers an assessment framework tailored to their cybersecurity risk level. In this section, we have investigated the connection and correlation between the two self-assessment exercises. One of the main findings in this section is a positive relationship between the AIs’ inherent risk score and their maturity. We also found that the underperformance of low and medium-risk AIs appears to be a group-wide phenomenon where most low- and medium-risk AIs have failed to meet the required maturity level in each of the seven maturity domains.

To better understand if the AIs have put sufficient resources into addressing various risks they faced, we then took three different perspectives in examining the AIs’ risk and maturity in more detail. The three groups of additional scatterplots are able to demonstrate the relationship in (1) The traditional cybersecurity measurement perspective, (2) An organizational planning and characteristics view, and (3) A response and recovery perspective. Essentially, under the three groupings, we found that high-risk AIs are more mature than medium-risk AIs under many of the maturity control domains, despite having better track records regarding their cybersecurity threats and having an equal or lower risk in many inherent risk domains.

60

C-RAF AND REAL-LIFE MEASURES

61

C-RAF AND REAL-LIFE MEASURES

SSL CERTIFICATION ADOPTION

SSL CERTIFICATES Secure Sockets Layers (SSL) certificate is a global standard security digital certificate that enables encrypted communication between a web browser and server.

Companies need SSL certificates for their websites to keep user data secure and to verify the validity on the website, preventing malicious attackers from creating a counterfeit version of the site.

When a website is secured by an SSL certificate, HTTPs will appear in the start of the URL

As set out at the beginning of the report, one of our aims is to identify if C-RAF and hence the Maturity Assessment are able to reflect the actual cybersecurity practice of the AIs. To achieve this, we have gathered data on the SSL certification adoption data for the AIs’ official websites across 17 six-month periods, which allow us to identify the AIs’ actual level of maturity practice beyond the self-assessment exercise. Later in this section, we have also conducted analyses on the time trend and adoption rate of suboptimal SSL certifications, including the self-sign certificates and weak hashing algorithm certifications.

While the adoption of SSL certificates is only a small part of the vast measure an AI ought to take, we believe it reflects the AI’s awareness and willingness to implement the fundamental measures in cybersecurity.

We looked at the longitudinal average adoption rate of SSL certificates for each AI over the 17 periods and have calculated the average SSL certificate adoption rate. The idea behind is that it allows us to see the full picture of the AI’s implementation instead of merely a snapshot. More specifically, if an AI adopts no SSL certificates at a given time, it will have a score of zero at that period and one otherwise. For example, an AI who started using SSL certificates to secure their websites at the final ten periods would have an adoption rate of 10 17 = 59%

SSL

SSL CERTIFICATION ADOPTION Rate Calculation ExampleAn AI started using SSL certificates to secure their websites in the last ten of the 17 six-month periods. Hence a percentage adoption rate of: 10/17 = 59%

The number of SSL certificates is at least one

0

7

17

The number of SSL certificates is less than one [ Exhibit 54 ] SSL certification adoption rate calculation

62

By applying least square regression, we found that both the overall maturity assessment score and the “Protection” maturity domain score are significantly correlated with the average adoption rate of SSL certificates over the 17 periods, despite having a modest level of explanatory power (~0.4 R2 ). To better understand the relationship between the two areas and the SSL cert adoption, we plot the data as a scatterplot.

Average Certification Adoption Rate and Maturity Assessment Score

Low

Medium

High

Average Number of Certification

100 80 60 40 20 0 50

100

150

200

250

300

350

400

Maturity

Average Certification Adoption Rate and Maturity Assessment Score

Low

Medium

High

Average Certification Adoption Rate

1.0 0.8 0.6 0.4 0.2 0.0 50

100

150

200

250

300

Maturity [ Exhibit 55 ] Number of SSL certification and adoption rate versus Maturity Assessment score

63

350

400

C-RAF AND REAL-LIFE MEASURES

By examining the scatterplots, we can better understand how the maturity score correlated with the adoption of SSL certificates of each AI. While the number of certificates has little to no correlation with the AI’s maturity score, the average adoption rate over the 17 periods shows a positive relationship with the maturity score. This can be explained by the nature of SSL certificates, where a company could opt to use one SSL certificate to cover all its website subdomains instead of using one certificate for each sub-domain. It is also reassuring to see that, as shown in Exhibit 55, several low- and medium-risk AIs had a full SSL certification adoption rate over the period, which is essential to safeguard consumer data and the security of the AI’s official site.

SUBOPTIMAL CERTIFICATION ADOPTION In addition to the above analysis, we have also studied the use of the various types of SSL certificates of the AIs, mainly those that might prove to be vulnerable to cyberattacks due to problems such as the lack of sophisticated encryption algorithm. Although in most cases the number of SSL certificates does not reflect how secure the AI is, the number of suboptimal certification types reflects if the problem is an organization-wide problem or specific to a part of the organization.

In the ideal scenario, an SSL certificate should be verified by a trusted third party, known as a Certificate Authority (CA), trusted by the users’ browsers, providing an extra layer of security.

Alternatively, an AI could issue a self-signed certificate for its sites, which provides basic security and still enables data encryption between the user and the application. However, since the users cannot authenticate these certificates by a known root CA, users are warned about the unknown certificate and must accept it to proceed.

[ Exhibit 56 ] Self-signed SSL certification warning message

Another crucial aspect that dictates if an SSL certificate effectively provides the desired security is the type of hashing algorithm used. If an SSL certificate is signed using a weak hashing algorithm (such as MD5 and SHA1), it will likely be vulnerable to collision attacks. The attacker could then exploit this to create a certificate with the same signature and thus allowing them to disguise as the affected service.

64

For the purpose of our analysis, we have obtained the number of self-signed and weak hashing algorithm SSL certificates of the 20 AIs which adopted any SSL certificate during the investigation period. If AIs with higher maturity does take more adequate measures to ensure cybersecurity, we should see the percentage of suboptimal SSL certification used decrease with a higher maturity score.

Self-signed Certificates and Maturity Assessment Score

Low

Medium

High

Percentage of Self-signed Certificates

0.20 0.15 0.10 0.05 0.00 100

150

200

250 Maturity

Percentage of Weak Hashing Algorithm Certificates

Weak Hashing Algorithm Certificates and Maturity Assessment Score

300

Low

350

Medium

400

High

0.4 0.3 0.2 0.1 0.0 100

150

200

250 Maturity

300

350

400

[ Exhibit 57 ] Percentage suboptimal SSL certifications versus maturity assessment scores

As the charts in Exhibit 57 indicate, while a few low-risk AIs have a relatively high adoption rate of weak certifications, the relationship is unclear. By conducting a regression, we found that neither the inherent risk of AI nor the maturity scores have significant statistical power in explaining the use of such certifications. Upon closer examination of the detailed Maturity Assessment response, we have found that while many have stated the use of SSL certifications,

65

C-RAF AND REAL-LIFE MEASURES

there was no mention of the type of hashing algorithm used or whether a CA signed the certifications. This has shown that while the Maturity Assessment is able to reflect holistically if the AIs have implemented the appropriate measures, the quality and to what extent it is implemented remain unclear. Nevertheless, we believe the level of details set out in the survey is adequate and is filled with insightful information about the AIs’ cybersecurity measures.

Maturity Level Attainment vs Security Certification Adoption To better understand whether AIs with higher cybersecurity maturity controls performs better, we checked the certification adoption status for AIs with a full attainment rate in their required controls and compared the result against those who have failed to meet their requirements.

Here we report on the average adoption rate of these AIs and check for the average number of suboptimal certificates and the adoption rate of such certificates. Again, the attainment percentage is calculated concerning maturity controls that the AIs are subjected to, based on their inherent risk level.

Full attainment (100%)

High attainment (90%+)

Yes

No

Yes

No

Number of AIs

4

18

12

10

Certification Adoption Rate*

0.79

0.67

0.72

0.66

Adoption Rate of Self-signed Certificates**

0.00

0.11

0.08

0.11

Adoption Rate of Weak Hashing Algorithm Certificates**

0.29

0.31

0.30

0.31

Number of Self-signed Certificates**

0.00

0.17

0.11

0.16

Number of Weak Hashing Algorithm Certificates**

0.93

1.60

2.08

0.76

Average over the 17 six-month periods * Higher the better; ** Lower the better [ Exhibit 58 ] Certification adoption in full attainment AIs and AIs with high attainment rate (90%+)

Exhibit 58 shows that AIs with full attainment have a higher average adoption rate of certificates over the 17 six-month periods than other AIs. In addition, they have not adopted any self-signed certifications. These AIs have also adopted less weak hashing algorithm certificates compared to their peers. These findings are similar for the high attainment grouping, though to a small extent, where the group of AIs with higher maturity control attainment often performs better. This has demonstrated the groupings and classification of the maturity assessment is able to explain to some extent the real-life implementation of cybersecurity measures of the AIs and shows AIs with higher attainment percentage have indeed handled their cybersecurity more appropriately.

66

TIME-SERIES ANALYSIS Apart from allowing the AIs to identify their inherent risk and gaps in their cybersecurity measures, the implementation of C-RAF could also help raise the AIs’ awareness in cybersecurity. Therefore, we analysed if their adoption of SSL certifications changed after participating in the selfassessment exercise. Exhibit 59 shows the plotted average adoption rate of certification, selfsigned certificate, and weak hashing algorithm certificates based on the AIs’ responses. The postC-RAF period was defined from the first half of 2017 to the first half of 2020.

SSL Certification Adoption Rate Change Over Time - All AIs After C-RAF

Certification

Self-signed

Weak Hashing Algoritnm

1.0 0.8 0.6 0.4 0.2 0.0

H1 H2 H1 H2 H1 H2

H1 H2

H1 H2

H1 H2

H1 H2

H1 H2 H1

2012

2015

2016

2017

2018

2019

2013

2014

2020

[ Exhibit 59a ] Change in SSL certification adoption rate overall time – All AIs

SSL Certification Adoption Rate Change Over Time - AIs With Full Attainment After C-RAF

Certification

Self-Signed

Weak Hashing Algoritnm

1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0

H1 H2 H1 H2 H1 H2

H1 H2

H1 H2

H1 H2

H1 H2

H1 H2 H1

2012

2015

2016

2017

2018

2019

2013

2014

[ Exhibit 59b ] Change in SSL certification adoption rate overall time – AIs with full attainment

67

2020

C-RAF AND REAL-LIFE MEASURES

SSL Certification Adoption Rate Change Over Time - AIs Without Full Attainment After C-RAF

Certification

Self-signed

Weak Hashing Algoritnm

1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0

H1 H2 H1 H2 H1 H2

H1 H2

H1 H2

H1 H2

H1 H2

H1 H2 H1

2012

2015

2016

2017

2018

2019

2013

2014

2020

[ Exhibit 59c ] Change in SSL certification adoption rate overall time – AIs without full attainment

Key findings: 1. Strong Trend Only for The Decrease of Weak Hashing Algorithm From our time-series analysis (Exhibit 59a), we see a weak yet increasing trend for the use of certifications and self-signed certifications while finding a strong decreasing trend in using weak hashing algorithms in their SSL certifications. This suggests that the AIs have improved the security of their websites by increasing the use of certifications, eliminating the use of weak hashing algorithms in their certifications, but have adopted slightly more self-signed certificates as a result of the higher overall adoption rate.

2. Increased Adoption of Certifications and Self-signed Certificates from AIs Without Full Attainment Most of the increased adoption of certificates (and self-signed certificates) come from the AIs who have failed to attain the required maturity controls. In comparison, the improvement in the elimination of weak hashing algorithm certificates is from both groups of the AIs (with or without full attainment).

3. No Sudden Changes After C-RAF The AIs’ adoption of SSL certifications has not experienced a sharp change after the C-RAF survey. However, the AIs have seen a steady improvement in their certification implementation, signalling that the AIs are aware of the importance of certifications and have adopted suitable measures. This should be encouraging for the regulator since the inherent risk analysis and the maturity analysis have helped AIs identify the areas to improve. The lack of significant change could be caused by a time lag in the impact or reflected in measures higher up in the organizational perspective, which is not reflected here. Better gauges to understand if C-RAF is

68

effective in improving the AI’s cybersecurity measures would be to (1) identify how much of the improvement the AI claimed to make in security has been implemented, or (2) to compare the cybersecurity incidents before and after participating in the C-RAF exercise.

C-RAF AND REAL-LIFE MEASURES: SUMMARY To verify if the Maturity Assessment reflects the AI’s actual maturity practice, we have conducted some studies on the relationship between the maturity assessment results and the AI’s SSL certification implementation status. While the use of SSL certifications is a small part of the cybersecurity measures the AIs ought to take, we found that the Maturity Assessment risk scores and the “Protection” domain show a mild but statistically significant positive relationship with the SSL certificates adoption rate. Lastly, when we checked if the maturity results are related to the use of suboptimal SSL certification, we found that the Maturity Assessment results are not able to reflect them properly. However, we believe the Maturity Assessment survey has acted well as a checklist and tool to help AIs identify their security measure gap.

Lastly, we have conducted a time-series analysis on the AI’s change in the adoption of SSL certificates. We hypothesize that the C-RAF exercise is able to impact the AIs in adopting more cybersecurity measures positively. While we did not find evidence supporting our hypothesize, we found that the AIs have progressively improved their adoption rate before the implementation of C-RAF, which might have limited the impact the C-RAF could bring. Indeed, the impact of the C-RAF exercise could have been more towards the bigger picture and have driven the AI to implement more rigorous and robust policies and measures. A better gauge to measure its impact would be to examine the additional measures the AIs take or the change in their cybersecurity incidents after their participation.

CONCLUSION Self-assessment is commonly adopted in cybersecurity risk management. The C-RAF is a thoughtful self-assessment framework that combines risk with control measure assessments to help AIs in the financial industry evaluate their risks and address the gaps in protection. This research synthesizes the survey findings from 22 AIs to provide a holistic view of the cybersecurity status of the Hong Kong financial industry. It provides an informative benchmark for AIs and organizations in other industries about the state of cybersecurity, and raises public awareness about the importance of addressing cybersecurity gaps.

69

APPENDIX

APPENDIX 1 INHERENT RISK SUB-DOMAIN MAPPING Technologies risk indicators Network Risks

Third-party risks

Internal risks

ISP connections

Non-corporate devices (2)

Internally hosted and in-house developed applications

Unsecured external connections

Third-party access to internal systems (2)

Internally hosted vendordeveloped applications

Wireless network access

Wholesale devices

User developed technologies

Network devices (e.g., routers, Individuals and/or third-party and firewalls) service providers supporting critical activities

End-of-life systems

Cloud computing services hosted externally to support critical activities (2)

Open-source software

Delivery Channels risk indicators Internet presence

Mobile presence

Social media presence

ATM

Internet presence

Mobile presence

Social media presence

ATM

Products and Technology Services risk indicators Payment card risks

Fund transfer risks

Client services

Issue debit or credit cards

P2P

Treasury services and clients (2)

Prepaid cards

Wire transfers (2)

Trust services

Merchant acquirer (2)

Global remittance

Securities trading

Act as a correspondent bank

70

Organizational Characteristics risk indicators Size risks

Service risks

Cybersecurity staffing risks

Number of branches

Host IT services for other organizations

Changes in IT and cybersecurity staffing

Revenue

Direct employees

Number of cybersecurity staff supporting HK business

Asset value

Privileged access (2)

Tracked Records on Cyber Threat risk indicators Frequency

Variety

Number of attempted cyber attacks

Phishing

Number of successful attacks

DoS

Number of breaches

Social engineering Malware

71

APPENDIX

2 HIERARCHICAL CLUSTERING HEATMAP ILLUSTRATION Technologies

Delivery Channels

Products and Technology Services

Organizational Characteristics

Tracked Records On Cyber Threats

72

3 FACTOR ANALYSIS LOADING SCORE TABLE Factor 1

Factor 2

Factor 3

Factor 4

Network Risks

0.51

0.49

-0.03

0.11

Third-party risks

0.13

0.09

0.17

0.15

Internal risks

0.55

0.13

0.23

0.51

Internet presence

0.21

0.72

-0.03

0.26

Mobile presence

0.69

0.41

0.25

0.25

Social media presence

0.18

0.23

0.60

0.15

ATM

0.69

0.43

0.23

-0.09

Payment card risks

0.94

0.24

-0.09

0.10

Fund transfer risks

0.69

0.81

0.06

-0.05

Client services

0.23

0.55

0.25

0.29

Size risks

0.59

0.56

0.02

-0.13

Service risks

0.32

0.52

0.10

0.12

Cybersecurity staffing risks

-0.09

0.11

-0.74

0.19

Frequency

-0.01

0.10

0.83

0.27

Variety

0.02

0.13

0.09

0.95

73

APPENDIX

74

HTTPS://WWW.BM.UST.HK