Validation of the s-‐peek ’s MORE Rating
As you well know s-‐peek uses the MORE (Multi Objective Rating Evaluation) rating model which is developed and owned by modeFinance (www.modefinance.com). One of the main elements when using a rating model is to understand if the rating model is predictive of default of companies. It is basically studying how good the model has been in anticipating the bankruptcy of the company; in fact, what you would expect from a credit rating is that it anticipates the awareness of the bankruptcy well in advance. There are countless scientific studies that you can follow to validate a credit rating model; and to validate the discriminant power of the MORE rating, a well-‐known statistical technique ROC (Relative Operating Characteristic) has been adopted. When used for the evaluation of rating models, ROC curves are constructed by ordering on the x axis, the firms which did not bankrupt, based on their score, from the worst to the best. The y-‐axis shows the cumulative percentage of bankrupt firms (compared to the total of the same) with a score of less than or equal to that is corresponding to the x-‐axis. In other words, the y-‐axis quantifies the percentage of the bankrupt excluded companies, depending on the number of the “not bankrupt” excluded companies.
An appropriate measure to summarize the performance expressed by the ROC curve is so-‐called "Area Under the Curve": AUC. It is expressed as the ratio between the area under the ROC curve, respect to the area of the unit square. An AUC value of 0.5 indicates a random model, a value equal to 1 indicates a model with perfect discrimination. modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 1
A basic classification of the performance of the models, based on AUC is provided below: • 1.00 -‐ 0.90: excellent • 0.90 -‐ 0.80: good • 0.80 -‐ 0.70: adequate • 0.70 -‐ 0.60: poor • 0.60 -‐ 0.50: inefficient Below, we show the statistics obtained with the ROC method applied to the MORE ratings in the following countries: -‐ Italy; -‐ The UK; -‐ Germany; -‐ France; -‐ The Netherlands; -‐ Denmark; -‐ Finland; -‐ Sweden.
Italy For Italy, there is a large database of bankrupt companies. In order to evaluate the performance of the MORE model in a precautionary manner, we consider the last financial statements filed before the bankruptcy as unreliable. Therefore, we indicate with "1 year before" the performance of the model implemented on the basis of financial statements filed between 12 and 24 months before the bankruptcy. "2 years before", the performance between 24 and 36 months before, etc.
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 2
4 years before bankruptcy
3 years before bankruptcy
2 years before bankruptcy 1 year before bankruptcy
As you can see, the performance of the MORE model is very high already 4 years before the bankruptcy with an AUC of 83. In particular, one year before the bankruptcy, the AUC reaches the value of 91, indicating an excellent discriminatory ability of the MORE model.
The United Kingdom Even for the UK, a large amount of data for companies which went bankrupt is available, so you can make an accurate statistical analysis of predictability. The first analysis that we do is to see the evolution of the MORE rating of the companies which went bankrupt (the values are -‐1 -‐2 -‐3 years prior to the last financial statements filed before the modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 3
bankruptcy). From the following figure, it is easily understood as approaching the date of the bankruptcy, the rating of the companies decreases severely, reaching the values close to the CCC.
United Kingdom
B5
CCC 4
CC 3 -‐3
-‐2
-‐1
If we study the distribution of the ratings and not just the average value, we realize that there is as much as 74% of the companies are classified in a negative way two years before the bankruptcy. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
-‐3
-‐2
-‐1
Risky
26%
30%
37%
Vulnerable
42%
44%
41%
Balanced
28%
21%
20%
Healthy
4%
5%
3%
Below, the graphs with AUC and ROC values for the UK, in the years prior to bankruptcy can be seen:
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 4
4 years before bankruptcy
3 years before bankruptcy
2 years before bankruptcy 1 year before bankruptcy
From these graphs, we can understand that the MORE Rating model has been very accurate and predictive of bankruptcy in the UK.
Germany The same studies for the UK have been replicated in Germany, in order to understand the ability of the MORE rating to predict the failure of the companies in this country:
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 5
Germany B5
CCC4
CC3 -‐3
-‐2
-‐1
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
-‐3
-‐2
-‐1
Risky
20%
29%
31%
Vulnerable
48%
46%
51%
Balanced
28%
23%
17%
Healthy
4%
3%
2%
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 6
4 years before bankruptcy
2 years before bankruptcy
3 years before bankruptcy
1 year before bankruptcy
As it can be seen both from the behavior of the bankrupt companies in previous years and what ROC models give, also in Germany the MORE model is extremely precise in bankruptcy evaluation.
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 7
France
France 5B
4CCC
3CC
-‐3
-‐2
-‐1
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
-‐3
-‐2
-‐1
Risky
39%
42%
50%
Vulnerable
36%
36%
34%
Balanced
20%
18%
13%
Healthy
5%
4%
3%
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 8
5 years before bankruptcy
3 years before bankruptcy
4 years before bankruptcy
2 years before bankruptcy
Also in France, the results are excellently confirmed.
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 9
The Netherlands
Netherlands B4
CCC 3
CC2
C1
-‐3
-‐2
-‐1
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
-‐3
-‐2
-‐1
Risky
40%
50%
60%
Vulnerable
23%
22%
19%
Balanced
36%
28%
20%
Healthy
1%
1%
1%
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 10
3 years before bankruptcy
1 year before bankruptcy
2 years before bankruptcy
Also in the Netherlands, the results are confirmed (the discontinuity of the curves is given by the MORE rating model that applies constraints on the income statement of the companies in case of lack of data, this is a situation frequently met in the Netherlands).
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 11
Denmark B CCC CC
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 12
5 years before bankruptcy
3 years before bankruptcy
1 year before bankruptcy
4 years before bankruptcy
2 years before bankruptcy
The results have been confirmed also in Denmark: the MORE rating model is successful in predicting the bankruptcy also in this country.
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 13
Finland
Finland B5
CCC 4
CC3
C2
-‐3
-‐2
-‐1
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
-‐3
-‐2
-‐1
Risky
44%
54%
69%
Vulnerable
35%
31%
22%
Balanced
16%
12%
6%
Healthy
5%
3%
3%
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 14
4 years before bankruptcy
2 years before bankruptcy
3 years before bankruptcy
1 year before bankruptcy
Same results in Finland: the MORE rating has a high performance also in this country.
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 15
Sweden
B CCC CC
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 16
3 years before bankruptcy
1 year before bankruptcy
2 years before bankruptcy
And also in Sweden the results have been confirmed.
Conclusions In this short document, the efficiency of the MORE rating evaluation model used by s-‐peek for the analysis of the financial risk of all European companies has been demonstrated. To validate the model, we used the AUC model that allows us to understand if a model discriminates efficiently the healthy companies from the bankrupt ones. The model was tested on European companies, particularly in the following countries: Italy, the UK, Germany, France, the Netherlands, Denmark, Finland and Sweden. The tests showed an excellent predictability power of the MORE model across Europe, showing how the rating model used by s-‐peek is highly accurate in assessing the economic-‐financial quality of the companies in Europe.
modeFinance srl
Iscritta al Registro delle Imprese di Trieste PI/CF: 01168840328 E-‐mail: info@modefinance.com
Area Science Park, Padriciano 99 34149 Trieste, Italy Tel: +39 040 3755337 Fax: +39 040 3756741
Pagina | 17