Audit / Assessment

End-to-end evaluation of an AI system's ethical risks and controls.

Gradient Institute conducts comprehensive system assessments to detect failures of intent, specification or implementation of an AI system and its ethical controls at any stage of the system lifecycle. We also examine an organisation's governance around such systems.

Our assessments verify organisations' AI systems operate safely in accordance with their intent, ethics and values.

System Risk Snapshot

A light-touch examination of the system from an owner's perspective to elicit ethical risks and advise on appropriate controls.

Duration: 1-2 weeks

Format: Interviews with key stakeholders and system documentation review.

Examination of:

  • Sources of ethical risk
  • System intent and specification

System Governance Snapshot

A targeted case study that examines the human oversight, accountability and governance structures around an existing AI system within the organisation.

Duration: 1-2 weeks

Format: Interviews with key stakeholders and policy documentation review.

Examination of:

  • Limited view of RAI governance (from a system perspective)
  • Accountability for aspects of the system's performance

System Audit

A detailed technical examination of a system's design, implementation and operation to identify risks, examine controls and evaluate evidence that the system is operating in alignment with the organisation's values and intent.

Duration: 4-6 weeks

Format: Technical deep-dive examining design documentation and performance measures.

Examination of:

  • Sources of ethical risk
  • System intent and specification
  • Evidence of ethical operation
  • System design and implementation
  • Accountability for aspects of the system's performance

AI Governance Review

A detailed examination of an organisation's AI policy and governance from a responsible AI perspective.

Duration: 4-6 weeks

Format: Extensive review of existing policy and governance through workshops and documentation.

Examination of:

  • Scope and proportionality of governance
  • Control and oversight gaps
  • AI lifecycle stopgates
  • Transparency
  • Accountability

Recent engagements

Triangular bullet

Audit of core AI systems at a major recruiting platform

Triangular bullet

Snapshot of several key systems developed by a major Australian data analytics company

Triangular bullet

Governance and system snapshot at a major Australian telecommunications company

Triangular bullet

Fairness audit at a major international bank

Triangular bullet

Governance and system audit at a major Philippines bank

Triangular bullet

Audit of an AI system used by NSW Government

If you're interested in assessing the ethical risks and controls of an AI system at any stage of its lifecycle, please contact us.