Is competence enough?

Karen Collins, ROSEN USA and Ralf von Rahden, CERTivation GmbH, Germany, emphasise the importance of globally-accepted processes of assessment, especially when becoming certified in pipeline management.

ipeline standards and regulations specifically require staff to be both competent and qualified in the tasks they perform. ASME B31.8S 2004 states, “The personnel involved in the integrity management programme shall be competent, aware of the programme and all of its activities, and be qualified to execute the activities within the programme.” Why do they need to be qualified? Is being competent not enough?

With the globalisation of our workforce, the ability to verify the competence of a person becomes more important. Individuals can claim they have the knowledge, skills and experience necessary, but how can one be sure? Organisations have no way of knowing whether educational programmes or training from different countries are equal. Does the same job title mean people have the same experience?

Most likely not – and this is where ‘qualified’ comes in. ‘Qualified’ is commonly defined as individuals that have been evaluated and can perform assigned tasks.

Often times, organisations have ‘qualified’ their workforce by only assessing their personnel’s level of education and/or experience against their own processes and criteria, omitting the formal link between their education/experience and the knowledge and skills needed to perform a job successfully.

Competence = skills + experience + knowledge The US National Transportation Safety Board (NTSB) concluded in their safety study ‘Integrity Management of Gas Transmission Pipelines in High Consequence Areas’ that “the professional qualification criteria for pipeline operator personnel performing IM functions was inadequate” and stated, “Only one operator listed any required training beyond a basic familiarity with the company’s IM programme and the federal IM regulations.”

Important to note here is that how a person is qualified is as important as the qualification criteria themselves. These ‘employer-based’ qualifications can lead to a variety of criteria, inconsistencies and maybe even a lack of credibility. Additionally, the integrity of these assessments is jeopardised by threats to impartiality caused by: ) Self-interest threats that arise from a person or body acting in it’s own interest to benefit itself.

) Subjectivity threats that result from personal bias overruling the objective evidence.

) Familiarity threats that occur when the assessor is familiar with the person being assessed, affecting their objectivity.

Although these processes, criteria and assessment results are documented, they are not usually ‘certified’ (a verification that the product, service or system in question meets specific requirements).

The terms ‘competent’, ‘qualified’ and ‘certified’ are often confused with each other; while not the same, they are interrelated. You can be competent but not qualified or certified. Similarly, you can be qualified and competent but not certified.

Certification can only be achieved through independent third-party assessments that provide a neutral setting to assess personnel competence, adding credibility to the results. Additionally, certification of persons can only occur when there is a certification scheme.

ANAB Accreditation Just as individuals are certified, organisations are accredited. Accreditation is the formal recognition by an independent body, generally known as an accreditation body, that a certification body operates according to international standards.

The International Organisation for Standardisation (ISO) and the International Electrotechnical Commission (IEC) developed ISO/IEC 17024:2012 Conformity assessment – General requirements for bodies operating certification of persons “with the objective of achieving and promoting a globally accepted benchmark for organisations operating certification of persons. Certification for persons is one means of providing assurance that the certified person meets the requirements of the certification scheme. Confidence in the respective certification schemes for persons is achieved by means of a globally accepted process of assessment and periodic re-assessments of the competence of certified persons.” 1

Certification bodies must demonstrate that their assessment process is fair, valid and reliable, giving all eligible applicants an equitable opportunity to participate in order to achieve accreditation. The 2014 Standards for Educational and Psychological Testing manual states, “The central idea of fairness in testing is to identify and remove construct-irrelevant barriers to maximal performance for any examinee.”

Validity is an indicator to establish that the process measures what it is intended to measure, and reliability shows that the test measures a person’s abilities in a consistent manner.

As of 22 December 2021, CERTivation GmbH, a wholly owned subsidiary of the ROSEN Group, received accreditation by the ANSI National Accreditation Board (ANAB), for fulfilling the requirements of ISO/IEC 17024:2012 Conformity assessment – General requirements for bodies operating certification of persons within the certification schemes Certified in Pipeline Integrity Management: CS_014F and Certified in Pipeline Defect Assessment: CS_020F.

A certification scheme details the competence (knowledge, skills and abilities) and other requirements, including such prerequisites as experience, education and mentoring. Qualification criteria, by contrast, may only define education and experience requirements. The certification scheme must be verified through a job task analysis (JTA) to ensure the job tasks reflect current industry needs.

There are several methods for conducting a job analysis, including interviews, focus groups, surveys and job observation. CERTivation chose to use a survey to conduct the JTA. Approximately 1500 pipeline integrity engineers, operators, industry practitioners, regulators and industry consultants in 46 countries were invited to participate in the survey. Respondents could select from four different languages in which to take the survey: English, Spanish, Russian and Portuguese. The JTA is only the first of many required steps necessary to meet the requirements laid out in ISO/IEC 17024:2014. Ensuring test fairness is a fundamental part of validity, starting with a clear definition of the construct – the knowledge, skills and abilities – intended to be

Figure 1. The ROSEN Group now offers ANAB accredited certification in Pipeline Integrity Management and Pipeline Defect Assessment.

measured, and the development of assessment questions (items) that are explicitly designed to assess the construct that is the target of measurement.

Each exam question was reviewed to identify and remove invalid aspects of test questions that might hinder people in various groups from performing at levels that allow appropriate inferences about their relevant knowledge and skills. The review of items was performed by individuals who are familiar with the specifications and purpose of the test, the subject matter of the test as necessary, and the characteristics of the test’s intended population. Subjectmatter experts reviewed items to identify any that were likely to be inappropriate, confusing or offensive for groups in the test-taking population. Important aspects of the review included: ) Content accuracy.

) Suitability of language.

) Match of items or tasks to specifications.

) Completeness and appropriateness of scoring rubrics.

) Appropriateness of presentation and response formats.

) Appropriateness of difficulty.

Fairness was also addressed using empirical methods; each item was beta-tested through psychometric analysis and calibrated based on the fit statistics – p-values, pt-biserials, etc. – reducing each exam to a 40 question multiple-choice exam.

Certified in Pipeline Integrity Management: CS_014F Individuals certified in Pipeline Integrity Management can define, and distinguish between, different integrity management methods/techniques – particularly pipeline integrity management and systems – and can list the threats to pipeline safety and the consequences of pipeline failure.

The programme was designed for individuals working in pipeline integrity who either have a university degree from a recognised or accredited university, 4 years of relevant work experience in the pipeline integrity field, an engineering degree from a recognised or accredited university, or a professional engineering qualification such as PEng or CEng.

Applicants must have 100 hours of experience, 18 hours of training and 36 hours of mentoring to be eligible. Applicants must pass an online, computer-based, 40-question, multiple-choice Pipeline Integrity Management: CS_014F certification examination under the supervision of a live invigilator to be certified in the competency. Topics covered on the exam include: ) Various approaches to pipeline integrity management.

) Standards requirements (including ASME B31.8S, API 1160, API 1173, CSA Z662, DNV RP F116, BSI 8010-4, EN 16348).

) Interpreting inspection and survey reports and inspection, testing, maintenance and surveillance options (all methods).

) Risk-reduction options, preventive measures and mitigations. ) Establishment of baseline inspection and testing intervals.

) Defect assessment, including fatigue assessment.

) Pipeline repair and rehabilitation; repair-programme design.

) Threats to pipeline integrity and consequences of pipeline failure.

) Critical data, missing data, treatment of uncertainty.

) Emergency preparedness, emergency response, site investigation.

Certified in Pipeline Defect Assessment: CS_020F Individuals certified in Pipeline Defect Assessment can describe pipeline integrity and pipeline defect assessments (for all types of defects found in pipelines) and can summarise and give examples of fatigue assessments. The programme was designed for individuals working in pipeline integrity who either have a university degree from a recognised or accredited university, four years of relevant work experience in the pipeline integrity field, an engineering degree from a recognised or accredited university, or a professional engineering qualification such as PEng or CEng.

Applicants must have 100 hours of experience, 18 hours of training and 36 hours of mentoring to be eligible.

Applicants must pass an online, computer-based, 40 question, multiple-choice Pipeline Integrity Management: CS_020F certification examination under the supervision of a live invigilator to be certified in the competency. Topics covered on the exam include: ) Material properties (with a detailed focus on fracture toughness).

) Fracture mechanisms (brittle, transitional, ductile) and fracture mechanics approaches (including J integral and crack-tip opening displacement).

) Assessment of corrosion using ASME B31G, assessment of cracks using API 579/BS 7910 and assessment of mechanical damage.

) Pipeline failure causes (including external interference, external forces, corrosion, fatigue ground movement, sabotage, theft, human error [including over-pressure, over-temperature] spanning, hydrodynamic loads, buckling and thermal stressing).

) Effect of hydro-testing on defect behaviour.

) Commercial, corporate, political and sabotage threats to pipelines and facilities.

) Case studies in pipeline failures, modes and mechanisms.

) Corrosion growth and fatigue-crack growth models.

References

1. www.iso.org/obp/ui/#iso:std:iso-iec:17024:ed-2:v1:en