US20250028977
2025-01-23
Physics
G06N5/022
The patent application describes systems and methods for using artificial intelligence (AI) to audit other AI systems. The auditor AI operates on hardware processors, continuously scanning various AI systems to assess their productivity and operability. This involves identifying data sources powering these systems, measuring output discrepancies, and analyzing trends in quality. The auditor AI labels each system as positive, neutral, or negative based on its performance.
As AI becomes more autonomous and integrated into commercial industries, traditional supervision methods are less effective. This lack of oversight can lead to decreased performance and efficiency. The application highlights the need for AI auditors to monitor these systems, ensuring they maintain optimal functionality without human intervention.
The auditor AI systems evaluate operating AI systems by analyzing data sources, which can be open-source or closed-source. They assess the AI's decision-making quality, identifying biases and manipulations in outputs. By comparing multiple AI systems within the same environment, auditors can detect inconsistencies and classify outcomes.
Auditor AIs may work in teams, using various techniques to analyze the operating AIs. They assign labels—positive, neutral, or negative—based on performance metrics. These labels facilitate communication management with operating AIs, allowing entities to prioritize interactions with positively labeled systems while monitoring or restricting others.
The auditing process involves multiple levels of AI auditors, each independently assessing productivity and operability scores. These scores are grouped, with outliers removed to produce a final syndicated score. This score determines the label assigned to the operating AI system. The auditing process includes continuous scanning of data sources and output analysis to identify anomalies.