Assessing the Effectiveness of Compliance and Ethics Programs


My notes, live, from Jack Holleran of Ernst & Young  and Patricia Prince-Taggart of CA on ways to measure program effectiveness, with an analysis of qualitative and quantitative measures.

Jack put forth three primary objectives to compliance programs:

  • To prevent non-compliance
  • To detect non-compliance
  • To enhance business processes and decision-making

He offered the following as the benefits of measurement:

  • Enables you to “know where you are”
  • Enables you to demonstrate effectiveness to Executive management
  • Enables you to demonstrate effectiveness to Audit committee
  • Enables you to demonstrate effectiveness to Regulators
  • Enables you to identify and prioritize opportunities for improvement in ethics and compliance program (design and execution)
  • Enables you to demonstrate business case, or value, that ethics and compliance program provides to the business

He offered this as his illegible diagram of a compliance program:


Qualitative measures

  • Provide some indication of awareness of ethics and compliance program
  • Tend to be subjective in nature
  • Useful in identifying trends

Quantitative measures

  • Provide objective insights into program effectiveness
  • Tend to be hard data
  • Useful for benchmarking your company to other organizations or within industry

Measuring effectiveness – the role of auditing and monitoring:

Evaluate each control for adequacy:

  • As designed, will it prevent / detect? Alone, or with other controls?
  • If design is adequate, test to verify control is operating as designed

Testing examples:

  • Field work: policy application within business units
  • Continuous testing: review of helpline calls, customer complaints
  • Transaction reviews for red flags
  • Risk-based reviews (e.g., FCPA, environmental)
  • Surveys/focus groups to measure awareness, attitudes, knowledge

Establish procedures for conducting investigations:

  • Confidentiality
  • Case resolution procedures
  • Post-resolution surveys of callers
  • Checking for possible retaliation

You want to determine the “Effectiveness Gap”: the difference between the inherent risk and management’s effectiveness.

Ethics and compliance, like any business function, faces the internal challenge of demonstrating return on investment (ROI). Measuring effectiveness can enhance your ability to demonstrate ROI. Trending over time can produce insights.

Starting is the most important part of effectiveness. Doing nothing is not effective. You can’t be afraid to find out information.

Here are some other resources they recommended:

Metrics Qualification Tool

The Elephant in the Room: Program Evaluation & Performance Measurement

Measurement & Metrics Guide: Performance Measurement Approach and Metrics for a Compliance & Ethics Program

Metrics Full Listing

Metrics & Measurement Guide Presentation: Beyond Effectiveness

(These notes are taken live, so I apologize if I left out anything or misquoted someone. Please forgive any typos or grammatical errors.)