Evaluating international instruments - Organisation for Economic Co-operation and Development

Illustrative examples of IO Practices: Evaluation

A number of specific practices undertaken by IOs provide illustrative examples of the trends and experiences described in this Compendium. The table below provides a selection of practices related to evaluation of international instruments, which are under collection and will feature more in detail in the final Compendium.

Ex ante impact assessment

IO Name & Practice	Brief Description
ILAC, Structure of the ILAC Mutual Recognition Arrangement and Procedure for Expansion of the Scope of the ILAC Arrangement (ILAC - R6:05/2019)	ILAC R6:05/2019 Structure of the ILAC Mutual Recognition Arrangement and Procedure for Expansion of the Scope of the ILAC Arrangement requires the review and consideration of criteria and impacts prior to extending the ILAC MRA.
IMO, Ex Ante Impact Analysis	The International Maritime Organisation (IMO) conducts ex ante impact assessments with respect to certain proposed regulations, in order to gauge their probable effects in advance of adoption.
European Commission, Inception Impact Assessment (IIA) and Regulatory Impact Assessment (RIA)	The EU routinely carries out impact assessment for new EU initiatives (including Regulations and Directives) expected to have significant economic, social or environmental impacts.
OIML, Scrutiny of critical factors	The OIML scrutinises a number of critical factors before any new project is submitted for approval, including the development of new international standards. Factors reviewed include: • why there is a need for an OIML publication on the proposed subject • the scope of the proposed publication • the reasons for regulating this category of instrument • details of the of countries which already regulate this category of instrument or which intend to regulate it • a description of similar work underway in other organisations (if any); and • details of proposed liaison organisations. These factors and others are reviewed prior to submitting the project for approval by Member States in accordance with voting rules contained in the Convention establishing an International Organisation of Legal Metrology.

Ex post evaluation

IO Practice	Brief Description
ASTM, Review of Standards Procedure	Under the ASTM Review of Standards Procedure, all standards are entirely reviewed by the responsible subcommittee and balloted for re-approval, revision, or withdrawal within five years of its last approval date. If a standard fails to receive a new approval date by December 31 of the eighth year since the last approval date, the standard is withdrawn.
BIPM, Review of CIPM MRA	After more than 15 years of operation, a Review of the CIPM MRA, which is one of the core instruments of the BIPM, was set out to consider the benefits of the instrument and to establish views on what was working well and what needed to be improved regarding its implementation.
IEC, Maintenance of Deliverables	The Maintenance of Deliverables procedures enable the review of all IEC international standards for their acceptable usage, and determines whether revisions or amendments are required: ISO/IEC Directives Part 1 + IEC Supplement.
ILAC, ILAC MRA Periodical Evaluation of Full Members by Recognised Regional Cooperation Bodies	ILAC R1:03/2018 Management of ILAC Documents establishes the criteria for the review of all ILAC publications. The ILAC MRA evaluation processes are reviewed as part of the feedback system in place. There are studies available from http://www.publicsectorassurance.org/ on the impact of the reliability of and acceptance of results/data/certificates from accredited conformity assessment bodies.
ISO, Systematic Review (SR)	The Systematic Review process supports the review of individual standards every five years (max) by experts at the national level, to assess whether they are still valid, require updating, or should be withdrawn. ISO has issued guidance to support this systematic review process.
OECD, Legal Provisions for Monitoring and Evaluating Instruments	Most OECD legal instruments include a provision by which the Council instructs the responsible committee(s) to monitor the implementation and, to report back to Council within a given timeline, usually five years from the date of adoption and regularly thereafter. The reporting is intended to evaluate the usefulness of the instrument rather than compliance by individual adherents.
OIML, Recommendation Review process and periodic review of publications	Under the Recommendation Review process, OIML recommendations and documents are reviewed every five years after their publication, to decide whether it should be confirmed, revised or withdrawn. The priority for the periodic review of OIML publications is defined by the Presidential Council and the OIML secretariat in consultation with the OIML-CS Management Committee, and approved by the International Committee of Legal Metrology (CIML). High priority publications shall be subject to a periodic review every two years.

Stock review

IO Practice	Brief Description
ILO, Standards Review Mechanism	The Standards Review Mechanism was established in 2011 to ensure that the organization has a clear, robust and up-to-date body of international labour standards. The Standards Review Mechanism Tripartite Working Group (SRM TWG) assesses and issues recommendations on a) the status of the standards examined, including up-to-date standards, standards in need of revision, outdated standards, and possible other classifications; (b) the identification of gaps in coverage, including those requiring new standards; (c) practical and time-bound follow-up action, as appropriate.
IMO, Ad Hoc Steering Group for Reducing Administrative Requirements (SG-RAR)	The IMO Ad Hoc Steering Group for Reducing Administrative Requirements (SG-RAR) was established in 2011 to conduct a comprehensive review of the administrative requirements contained in the mandatory instruments of the organisation, with a view to issuing recommendations on how the burdens imposed by these requirements could be reduced. This involved the creation of an Inventory of Administrative Requirements, which identified 560 administrative requirements, and the conduct of an extensive public consultation process.
OECD, Standard-Setting Review (SSR)	In 2016, the OECD Secretary-General launched an organisation-wide Standard-Setting Review (SSR) C/MIN(2019)13. During its first phase, the SSR involved a comprehensive assessment by OECD substantive committees of the overall stock of OECD legal instruments and led to the abrogation of a first set of 32 outdated legal instruments. Now in the second phase of the SSR, substantive committees are implementing their respective standard-setting Action Plans (i.e. the review, revision or monitoring of implementation of 134 legal instruments), and considering proposals for additional abrogations or the development of new instruments in some areas.
WCO, Performance Measurement Mechanism (PMM) (under development)	The WCO Performance Measurement Mechanism (PMM) which is under development aims to incorporate all the Customs competences (revenue collection, trade facilitation and economic competitiveness, enforcement, security and protection of society, etc.) for assessing and improving organisational efficiency and effectiveness, but also to reflect the extent to which the relevant WCO tools and Conventions are applied in support of revenue mobilisation and safe and smooth movement of people and goods. Taking advantage of the WCO participation in the work of the IO Partnership, the design of the prospect WCO PMM adopts the evaluation principles of progression and proportion presented in this Compendium and prospectively aims to build up an evaluation framework that connects Members’ implementation, awareness and use of the main WCO tools to the relevant organisational performance dimensions of the WCO constituency.

Review of a sub-set of instruments

IO Practice	Brief Description
IUCN, Impact of IUCN Resolutions on International Conservation Efforts	The Review of Impact of IUCN Resolutions on International Conservation Efforts highlights some of the major impacts and influences that IUCN Resolutions have had on conservation.
UNESCO, Evaluation of UNESCO Standard-Setting Work in the Culture Sector	The Evaluation of UNESCO’s Standard-Setting Work of the Culture Sector in the Culture Sector examines the impact of four culture conventions on the legislation, policies, and strategies of the Parties to them, as well as their implementation at the national level. Each evaluation is supported by the issuance of a series of recommendations, designed to increase the relevance and effectiveness of these normative instruments.