australian medical records are not private, page-7

This brings up an interesting website I came across a few years ago.

It's amazing what can now be done once you have a bunch of metadata files. Crucial ones are bank transactions and phone records. These files are usually millions of lines per day, no names, just account numbers, transaction dates and times, transaction amounts, phone numbers at each end, etc. Far too much data for anyone to sort out in any meaningful way.

That's where this company comes in.

http://www.palantir.com/

They make software which can rapidly sort through these files and display it graphically so an operator can interact with it. You only need to start with one piece of information that you know for sure, for example you might know a person's bank acct number. From there you can bring up a timeline showing graphically all deposits and withdrawals from that account.
You might click on one and it brings up where the money came from or went to. Same with phone calls.
In the center of the screen you can build up what looks like a group of interconnected circles, each one representing the name of a person or organisation who interacted with the account. There will be lines between the circles which might represent a phone call, or money flow, whatever you told it to do.
The software will go out on the net and search for names, addresses, photos etc that matches the relevant info, usually from social networking sites, so each "circle" ends up being a description of that person. To cut a long story short you can end up with a complete description of a persons's life, everything they have done, and same for anyone or anything they have interacted with.

A few years ago they had some nice tutorials on the site which showed various user interface examples but I can't find them on the site now.

I first got interested when I heard they had helped to go after Julian Assange and Wikileaks to see how the organisation was structured and where the weaknesses were.

Palantir had a nice presentation on their site which I downloaded in pdf format. Is anyone interested? Can I upload a pdf to HotCopper?

Palantir got a lot of bad publicity over that and apologised shortly afterwards.

Here is another example in a presentation that was on the site. The US was wondering where all it's aid money to Afghanistan was going so they engaged a security company that subcontracted to Palantir. Palantir ended up knowing everything. In particular, a lot of the money ended up buying a villa in the middle east for each of the government ministers, around 12 or 14 from memory. Unfortuately I didn't download that one. I bet it has been taken down now.

The point is that once you have the data files the sky is the limit, and it's fast. I'm amazed how little is publicised about what this software is capable of. The Palantir website looks fairly bland now. A few years ago it was all about software engineering and advertising for the smartest software engineers to join. The present site tends to mask the true capability unless you know what you are looking for.

Of course, the critical bit is getting the metadata files. You need to be in a position of power to get them.