AI Assessment | Survey

Employee Information

Full Name

Sector

Project Name

Project Phase

Email Address

1. Risk Evaluation

This assessment will evaluate the potential risks associated with the AI solution. It will consider factors such as the severity of potential consequences, the likelihood of those consequences occurring, and the number of people who could be affected. By combining these factors, the assessment will have a comprehensive understanding of the overall risk level associated with the AI solution. The objective is to evaluate overall risk level for a specific use-case, classifying it as high, minimal or limited. Please, also bear in mind that use cases with unacceptable risk levels are prohibited.

Risk Evaluation

1. How severe could the potential negative effects be?

Very Mild
Mild
Moderate
Severe
Highly severe

2. How likely is it for the AI system to cause a negative impact?

Very unlikely
Unlikely
Possible
Likely
Very likely

3. How many people are/could be affected by the AI system?

Very few people
Few people
Some people
Many people
Everyone

Risk Evaluation

2. Ethical Evaluation

The assessment will evaluate the ethical implications of the AI solution based on 26 questions related to the six RAI principles. These questions will cover various aspects of ethical AI development and deployment, including fairness, transparency, accountability, privacy & security, robustness, and human centricity. By answering these questions, the assessment will calculate an ethical score that reflects how well the AI solution aligns with ethical principles. The assessment takes you through each of the ethical principles, with the specific questions differentiated depending on the risk level. Depending on your answers to the questions, you may receive suggestions for further actions.

Ethical Principles Evaluation : Transparency & Explainability

1. Do users know that they interact with an AI system and that a significant decision has been made or assisted by AI?

Yes

No

2. Are you transparent to the user with respect to the data the system uses and for what purpose?

Yes

No

3. Did you establish mechanisms to inform users about the implications, purpose, criteria, and limitations of the decision generated by the AI system?

Yes

No

4. Would users understand how the algorithm has reached its conclusions? For example, what variables have influenced the result?

Yes

No

5. Are your end-users likely to understand a technical explanation?

Yes

No

Ethical Principles Evaluation : Human Oversight

1. Have you determined the appropriate level of human involvement? (HITL, HOTL, HOOTL).

Yes

No

2. Have those responsible for oversight been given specific training?

Yes

No

3. If the system is human-out-of-the-loop (HOOTL), do you offer the possibility to intervene/ do you give users the option to request a human in the loop (to manually test the outcome)?

Yes

No

Ethical Principles Evaluation : Fairness

1. In the input data, did you consider diversity and representativeness of users with respect to the target audience?

Yes

No

2. Have you removed sensitive variables from your training set, and checked for correlations with the remaining variables?

Yes

No

3. Do you assess the performance of the algorithm in respect to demographic subgroups (e.g., gender, race, age)?

Yes

No

4. Does the AI system cater for the variety of preferences and abilities in society?

Yes

No

5. Did you assess whether the AI system's user interface is usable by those with special needs or disabilities or those at risk of exclusion?

Yes

No

Ethical Principles Evaluation : Privacy & Security

1. Are you following stc Cybersecurity AI policy and internal privacy policy?

Yes

No

2. Have you ensured that the AI system is complying with all contractual and regulatory security obligations?

Yes

No

3. Have You established strict access control to allow only authorized access to ML models and training data?

Yes

No

4. Have you implemented measures to ensure the system's integrity against attacks throughout its operating cycle?

Yes

No

Ethical Principles Evaluation : Safety & Robustness

1. Do you monitor the AI system performance to know when recalibration is required?

Yes

No

2. Do you perform quality assessment of the input data and the results?

Yes

No

3. Is the accuracy level high enough for the use case?

Yes

No

4. Do you carry out validation beyond standard training and testing procedures to ensure reliability and reproducibility?

Yes

No

5. Have you included a 'red button' and designed contingency plans to address possible errors in the AI system?

Yes

No

Ethical Principles Evaluation : Accountability

1. Have you established mechanisms (e.g., documentation) to facilitate the AI system's auditability?

Yes

No

2. Have you established a comprehensive mechanism to track the accountability? For example, RACI (Responsible, Accountable, Consulted, Inform)matrix ?

Yes

No

3. Have you established detection and reporting mechanisms for undesirable adverse effects of the AI system on the user?

Yes

No

4. Have those responsible for supervising the activity of the system received specific training on how to exercise supervision?

Yes

No

Survey Completed

Thank you very much for your time. Please press submit to see full report.

Responsible AI Assessment

Introduction to AI Assesment

Overview

Objectives

Key Considerations

Employee Information

1. Risk Evaluation

Risk Evaluation

1. How severe could the potential negative effects be?

2. How likely is it for the AI system to cause a negative impact?

3. How many people are/could be affected by the AI system?

Risk Evaluation

Your risk evaluation score is

Your risk evaluation score is

Your risk evaluation score is

2. Ethical Evaluation

Ethical Principles Evaluation : Transparency & Explainability

1. Do users know that they interact with an AI system and that a significant decision has been made or assisted by AI?

2. Are you transparent to the user with respect to the data the system uses and for what purpose?

3. Did you establish mechanisms to inform users about the implications, purpose, criteria, and limitations of the decision generated by the AI system?

4. Would users understand how the algorithm has reached its conclusions? For example, what variables have influenced the result?

5. Are your end-users likely to understand a technical explanation?

Ethical Principles Evaluation : Human Oversight

1. Have you determined the appropriate level of human involvement? (HITL, HOTL, HOOTL).

2. Have those responsible for oversight been given specific training?

3. If the system is human-out-of-the-loop (HOOTL), do you offer the possibility to intervene/ do you give users the option to request a human in the loop (to manually test the outcome)?

Ethical Principles Evaluation : Fairness

1. In the input data, did you consider diversity and representativeness of users with respect to the target audience?

2. Have you removed sensitive variables from your training set, and checked for correlations with the remaining variables?

3. Do you assess the performance of the algorithm in respect to demographic subgroups (e.g., gender, race, age)?

4. Does the AI system cater for the variety of preferences and abilities in society?

5. Did you assess whether the AI system's user interface is usable by those with special needs or disabilities or those at risk of exclusion?

Ethical Principles Evaluation : Privacy & Security

1. Are you following stc Cybersecurity AI policy and internal privacy policy?

2. Have you ensured that the AI system is complying with all contractual and regulatory security obligations?

3. Have You established strict access control to allow only authorized access to ML models and training data?

4. Have you implemented measures to ensure the system's integrity against attacks throughout its operating cycle?

Ethical Principles Evaluation : Safety & Robustness

1. Do you monitor the AI system performance to know when recalibration is required?

2. Do you perform quality assessment of the input data and the results?

3. Is the accuracy level high enough for the use case?

4. Do you carry out validation beyond standard training and testing procedures to ensure reliability and reproducibility?

5. Have you included a 'red button' and designed contingency plans to address possible errors in the AI system?

Ethical Principles Evaluation : Accountability

1. Have you established mechanisms (e.g., documentation) to facilitate the AI system's auditability?

2. Have you established a comprehensive mechanism to track the accountability? For example, RACI (Responsible, Accountable, Consulted, Inform)matrix ?

3. Have you established detection and reporting mechanisms for undesirable adverse effects of the AI system on the user?

4. Have those responsible for supervising the activity of the system received specific training on how to exercise supervision?

Survey Completed