The What, How, and Why of AI Governance
Demystifying AI governance. Check out organizational use cases and model levels.
As AI technology continues to be implemented by businesses around the globe, a massive need for AI governance has emerged. Andrew Gamino-Cheong, co-founder & CTO of Trustible, explains the three layers of AI governance and how to implement them successfully.
Since its inception, AI has been a powerful tool to help improve internal operations by automating tasks and streamlining operations, building better technologies, and making end-user experiences easier and more personalized. While AI’s impact on society isn’t new, the rise in more advanced AI solutions has created cause for concern as to how these technologies will be leveraged moving forward.
The AI boom has created a massive need for strong AI governance. In the grand scheme, AI governance is a holistic term that can mean many different things to many different groups, likely due to the many layers of governance that can (and should) be implemented. Specifically, AI governance can be broken down into three levels: organizational governance, use case governance, and model governance. Since each layer of governance holds a specific purpose for different members of any organization, it’s essential to examine the three levels of governance in more detail.
Organizational Level of Governance
The first level in any multi-layered AI governance program is the organizational level. This level acts as a north star, helping all practitioners adhere to specific, necessary levels of ethics and responsibility. Each company may have its own proprietary AI code of conduct, but it typically centers around several key tenets. Mastercard, for example, has built an AI code of conduct centered on three core principles: inclusivity, explainability, and responsibility. Core principles may vary, but this is a good reference for companies building similar values-based mission statements.
While mission statements are essential starting points, organizational levels of governance are about more than just words; they are crucial in helping organizations prepare for impending AI regulations before it’s too late. At this level, defining responsibilities for each member involved in AI development is crucial to deploying successful implementation procedures. Every effective AI governance program clearly defines internal policies for AI ethics, accountability, and safety and develops processes enforcing these policies continuously.
Fortunately, businesses don’t have to start from square one when implementing these policies. Existing policies and frameworks, such as the NIST AI Risk Management Framework and the EU AI Act, offer clear directives on how businesses can build a successful organizational level of governance. For example, the NIST Framework’s “Govern” pillar lays out the various organizational governance policies and processes, while the EU AI Act contains several articles outlining organizational requirements for AI governance.
See More: EU Artificial Intelligence Act: Will the AI Act Fuel
Use Case Level of Governance
The second level of AI governance focuses on the specific use cases for AI within any business. Use case levels of governance are mainly focused on ensuring that any application of AI and its usage for specific tasks meets all necessary governance standards. This is because the risks stemming from improper AI usage are closely correlated with how it will be applied to any business’ operations.
AI models can be used for many different things, and these use cases can be bucketed into two separate categories: low-risk and high-risk. For example, a low-risk use case might be mundane tasks like summarizing meeting notes. In contrast, high-risk use cases, such as summarizing healthcare patient records, involve more sensitive information and thus require more scrutiny. At this stage, risk mapping and mitigation are also key to success, which usually relies on legal and compliance teams to play a major part, especially regarding reviewing and scrutinizing AI use cases and intended goals.
This level of governance also maps back to the overarching organizational level of governance, as these legal and compliance teams will need to affirm that each AI use case falls in line with regulatory requirements and parameters. As these teams’ involvement might suggest, governance at the use case level means that organizations must carefully and diligently document several things for both low and high-risk use cases, including intended goals of using AI for a specific task, justifications for why AI is appropriate, context-specific risks, and both technical and non-technical mitigation strategies to reduce risk across the organization.
See More: How AI Is Refactoring Finance, Manufacturing & Healthcare
Model Level Governance
The final and most granular level is the model level of governance. AI practitioners will have primary responsibilities at this stage, including model evaluation, testing for data accuracy, and conducting model bias and fairness assessments.
As the name might suggest, model-level governance caters to the actual technical functions of any AI systems and ensures they meet the expected standards of fairness, accuracy, and security. More specifically, practitioners tasked with managing model levels of governance must take steps to account for protecting private information while affirming that no biases exist that could affect protected or marginalized groups.
At the technical level, model levels of governance also must test the model continually to prevent model drift, which happens when a model’s prediction power is degraded due to shifts in external environments. Model drift may occur for several reasons, such as demographic changes that models have not had the time to “catch up” to. Fortunately, while model drift and bias may happen despite human oversight, the model level of governance can be supported by technology solutions that help accurately train and evaluate datasets.
Due to the hyper-focused nature of this type of governance, it might only sometimes be possible for businesses to implement specific model-level strategies, especially when accessing publicly available large language models (LLMs) such as ChatGPT. If companies find themselves in this position, they will need to double down on managing what they can control, such as the use case and organizational levels of governance.
AI governance can no longer solely focus on evaluating ML models and datasets; a combination of organizational, use-case, and model-level governance will create the holistic governance model needed for businesses to use AI moving forward responsibly. As regulatory guidance continues to take shape, organizations can act to build their governance policies, prepare for any sweeping changes that legislation may cause, and feel more confident about building compliant, ethical, and accurate AI moving forward.
What steps have you followed to implement AI governance successfully? Let us know on Facebook, X, and LinkedIn. We’d love to hear from you!
Image Source: Shutterstcok
MORE ON AI GOVERNANCE
