A few central bankers can't take over the world. The people would rise up instantly. And why would the central bankers want to take such a huge risk - they are already super-rich.
