"You still need to cough and get confirm whether the diagnosis...

"You still need to cough and get confirm whether the diagnosis are correct or not"

Yes, absolutely. Say you have data for 50 patients, all diagnosed with croup. You use 49 of those to train the algorithm. Then you get the algorithm to predict the condition of the 50th patient. You measure success or fail for the prediction. Then you take a different 49, train with those, then test the different 50th patient. Another success or fail. Once you've given each of the 50 patients a chance at prediction you can say "we got it right 84% of the time". This is the "leave-one-out" test methodology.

You'll note that the numbers for each condition are pretty low in the Australian study, which makes it hard to do larger test groups (e.g. train with 70, test with 30).

In the double-blind trial you need to have a pre-trained algorithm because you aren't given the diagnosis (so you can't train during the trial). You run the algorithm for each patient and provide the predicted condition back to the test team. The test team then get the diagnoses back from the lab. They then match your predictions to the diagnoses and let you know how you went.