ISTQB Certified Tester: AI Testing

Prerequisites

Attendees intending to take the ISTQB Certified Tester AI Testing examination must hold the ISTQB Certified Tester Foundation Level certificate. To effectively participate in the technical exercises, delegates should set themselves up in advance of the course, on Google Colab or a similar platform as this will enhance the learning experience and provide the necessary tools to complete hands-on exercises.

Course objectives

By the end of this course, participants will be able to: Understand the fundamental concepts of AI and machine learning. Identify and explain quality characteristics specific to AI-based systems. Apply various testing techniques and methods to AI-based systems. Evaluate and improve the performance of AI models. Address challenges in testing AI-based systems, including bias, transparency, and explainability.

Who should attend?

This course is designed for: Testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers involved in testing AI-based systems and/or AI for testing. Project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants seeking a basic understanding of testing AI-based systems and/or AI for testing.

Introduction to AI

Definition of AI and AI Effect Narrow, General, and Super AI AI-Based and Conventional Systems AI Technologies AI Development Frameworks Hardware for AI-Based Systems AI as a Service (AIaaS) Pre-Trained Models Standards, Regulations, and AI

Quality Characteristics for AI-Based Systems

Flexibility and Adaptability Autonomy Evolution Bias & Ethics Side Effects and Reward Hacking Transparency, Interpretability, and Explainability Safety and AI

Machine Learning (ML) Overview

Forms of ML ML Workflow Selecting a Form of ML Factors Involved in ML Algorithm Selection Overfitting and Underfitting

ML Data

Data Preparation as Part of the ML Workflow Training, Validation, and Test Datasets in the ML Workflow Dataset Quality Issues Data Quality and its Effect on the ML Model Data Labelling for Supervised Learning

ML Functional Performance Metrics

Confusion Matrix Additional ML Functional Performance Metrics for Classification, Regression, and Clustering Limitations of ML Functional Performance Metrics Selecting ML Functional Performance Metrics Benchmark Suites for ML

ML Neural Networks and Testing

Neural Networks Coverage Measures for Neural Networks

Testing AI-Based Systems Overview

Specification of AI-Based Systems Test Levels for AI-Based Systems Test Data for Testing AI-Based Systems Testing for Automation Bias in AI-Based Systems Documenting an AI Component Testing for Concept Drift Selecting a Test Approach for an ML System

Testing AI-Specific Quality Characteristics

Challenges Testing Self-Learning Systems Testing Autonomous AI-Based Systems Testing for Algorithmic, Sample, and Inappropriate Bias Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems Challenges Testing Complex AI-Based Systems Testing the Transparency, Interpretability, and Explainability of AI-Based Systems Test Oracles for AI-Based Systems Test Objectives and Acceptance Criteria

Methods and Techniques for the Testing of AI-Based Systems

Adversarial Attacks and Data Poisoning Pairwise Testing Back-to-Back Testing A/B Testing Metamorphic Testing Experience-Based Testing of AI-Based Systems Selecting Test Techniques for AI-Based Systems

Test Environments for AI-Based Systems

Test Environments for AI-Based Systems Virtual Test Environments for Testing AI-Based Systems

Using AI for Testing

AI Technologies for Testing Using AI to Analyse Reported Defects Using AI for Test Case Generation Using AI for the Optimization of Regression Test Suites Using AI for Defect Prediction Using AI for Testing User Interfaces

How certification is earned

The course and syllabus include a one-hour multiple-choice exam. To earn certification, participants must achieve a score of 65% or higher. This e-proctored exam is scheduled on a separate day, not within the 4-day course. Additional exam time may be granted to eligible individuals under certain conditions to be agreed with iSQI.

Related Certifications

After completing this course attendees may consider: ARTiBA Certified AI Engineering for Business and Management ISTQB Certified Tester Advanced Level Test Analyst ISTQB Certified Tester Advanced Level Technical Test Analyst