the standards of check AI

The standards for check AI mainly include the following aspects:
Accuracy standard: In information retrieval tasks, the recall rate should not be less than 80%. For example, in a database with 10000 news data, the system should recall at least 8000 news data related to keywords.
Reliability standards:
System stability: The system should be able to operate continuously for 7 × 24 hours with a fault free time of no less than 99.9%, that is, the cumulative downtime of system failures per year should not exceed 8.76 hours.
Data consistency: The error rate of data consistency should be less than 0.01%. For example, in a database containing 1 million user information records, the number of inconsistent data records should not exceed 100 after each data update operation.
Safety standards:
Data privacy protection: The encryption transmission rate must reach 100%, and all data involving sensitive user information must be encrypted using industry standard encryption algorithms, without plaintext transmission. In terms of data storage, the probability of data leakage risk should be less than 0.001%.
Algorithm robustness: Under adversarial sample attacks, the decrease in model accuracy does not exceed 10%. For example, for a trained image classification model, generating 100 adversarial samples for attack testing, the model's accuracy on normal samples is 90%, and under adversarial sample attacks, the accuracy should not be lower than 81%.
Interpretability criteria:
Model interpretability index: For complex deep learning models, at least 80% of decision results should provide reasonable explanations. For example, in a deep learning based loan approval model, analyzing 100 approval decision results, at least 80 results can be reasonably explained through methods such as feature importance analysis and locally interpretable model independent interpretation (LIME).
Explanatory readability: The explanatory content should be understandable by non professional technical personnel. Through a survey of 100 ordinary users, at least 70% of users stated that they can understand the explanatory content provided by the model.
Other standards:
Knowledge graph standards: including knowledge exchange protocols, etc.
Evaluation criteria for deep learning algorithms: Evaluating the performance of deep learning algorithms.
Assessment criteria for service capabilities of large models: Evaluate the maturity of service capabilities of large models.