Thanks for the clarification. From my understanding...

Thanks for the clarification. From my understanding leave-one-out cross validation both trains the algorithm and tests it at the same time. This method calculates the "apparent error", or how we expect it to be in real world. But of course your right, what the FDA/TGA/CE want is the "true error"- which is calculated by double blinding. When sample sizes are small, the apparent error could be quite different to the true error - which is what concerns FDA etc and you are probably right as to say they are not clinically valid tests. However with sample sizes as large as the Australian data, the apparent error should very closely reflect the true error and in fact this particular type of testing is slightly pessimistic - meaning the true error is going to be better (given its a large enough sample).

Happy to hear other opinions on this.