Assign-ClassificationError.pdf

Assignment: Understanding Classification Error Goal In this activity you will practice calculating the ROC curve and computing the confusion matrix. I. Evaluate an AI-based COVID-19 Diagnosis System Note: The information provided here is based on real research, however, given the seriousness of COVID19, please assume the information here is hypothetical and may contain errors and should not be used for any purpose beyond this assignment. The current gold-standard for diagnosis of COVID-19 is real-time polymerase chain reaction (RT-PCR) lab test [Bai 2020]. However, lab resources are expensive, limited and time consuming. A quick, cheaper and non-invasive alternative may be to perform CT imaging and use the features such as peripheral distribution, ground-glass opacity and vascular thickening of the CT images for diagnosis [Bai 2020]. Assume the scientists designed an alternative AI system, which takes in a CT image, recognizes the ground-glass opacity (GGO) feature, and performs the diagnosis in a few seconds. However, there is a trade-off between efficiency and accuracy, so we have to evaluate how much we can trust the system. (Simulated) Dataset : 100 patients were both tested by RT-PCR and the CT-based AI system: 51 patients were diagnosed by RT-PCR (the gold-standard) as positive (True) while 49 tested negative (False). The raw GGO values were collected from the AI system before making any thresholding. The data is saved in data/GGO_value.mat and data/diagnosis.mat respectively. Question 1 Assume the probability of positive and negative patients follow Gaussian distributions (see the two schematic plots below). Notice there is overlap between the two distributions (which means if we take different thresholds, we’ll obtain different prediction results). 1

(a) Using MATLAB, load the data and find the mean and standard deviation (std) of the Gaussian that models the positive distribution for 51 subjects. (b) Find the mean and std of the Gaussian that models the negative distribution for 49 subjects by MATLAB. (c) Plot of the two distributions in MATLAB (using the mean and std values found in parts (a) and (b). Label your axes to obtain a figure similar to the schematic plot shown above. Hint: use MATLAB’s normpdf function. Question 1. Your Answers: a) Mean of positive subjects: 72.57 Std of positive subjects: 12.57 b) Mean of negative subjects: 29.80 Std of negative subjects: 10.43 2

c) Paste plot here: Paste Code Here: Loading data: % Load the data from the provided MATLAB files load('data/GGO_value.mat'); % Assuming GGO_value.mat contains a variable named 'GGO_values' load('data/diagnosis.mat'); % Assuming diagnosis.mat contains a variable named 'diagnosis' % Separate GGO values for positive and negative subjects ggo_positive = GGO_values(diagnosis == 1); ggo_negative = GGO_values(diagnosis == 0); 3

Calculating mean and std: % Calculate mean and standard deviation for positive and negative distributions mean_positive = mean(ggo_positive); std_positive = std(ggo_positive); mean_negative = mean(ggo_negative); std_negative = std(ggo_negative); Choose the threshold range and step % Choose the threshold range and step (customize as needed) threshold_range = linspace(min(GGO_values), max(GGO_values), 1000); Build the distribution function % Build the distribution function (PDF) for positive and negative subjects pdf_positive = normpdf(threshold_range, mean_positive, std_positive); pdf_negative = normpdf(threshold_range, mean_negative, std_negative); Plot your figures % Plot the distributions figure; plot(threshold_range, pdf_positive, 'b', 'LineWidth', 2, 'DisplayName', 'Positive Distribution'); hold on; plot(threshold_range, pdf_negative, 'r', 'LineWidth', 2, 'DisplayName', 'Negative Distribution'); xlabel('GGO Values'); ylabel('Probability Density'); title('Gaussian Distributions for Positive and Negative Subjects'); legend('Location', 'best'); grid on; hold off; 4

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help