I find this argument really weird. Are you assuming that if someone held USD in 1900 that the only comparison to now would be if they held their cash underneath their bed. The only reason gold price has risen in value since 1900 is due to inflation as well.
