Hey brother, thanks for the cudos but in honesty I am nothing...

Hey brother, thanks for the cudos but in honesty I am nothing more than an old seabird...

I just spend hours on the wing scooping krill and then regurgitate when I arrive home.... And for tonights sea oil surprise a tasty morsel from a new fishing ground... Enjoy

http://www.storagesearch.com/

	are we ready for infinitely faster RAM?
1
2	(and what would it be worth)
3
4	by Zsolt Kerekes, editor - May 14, 2018
5	I f someone could offer you a memory system which had the same storage density (bits per chip / module / box) as mainstream RAM - but which had latency and bandwidth (as measured by what the application sees) which was infinitely faster - could we use it? - how much would that be worth? and how would it change markets? For the past 25 years the computer market has voted with its spending for bigger rather than faster memory - but is the market now receptive to a disruptive change in its ideas about the user value proposition of memory performance?
6
7	what's infinitely faster RAM?
8
9	I know that some of you who are reading this (and maybe it's You) are the kind of people who found companies or fund them (thanks for staying with me on this ) and when you noticed the words "infinitely faster" in my title above you wondered if it was some kind of late April Fool article. (No - I wrote something else.)
10
11	Infinitely? Really? - I know you can't put the value of infinity into a business plan (although it does come in useful sometimes for testing boundary assumptions about how markets will react to disruptive change). So let me explain my use of the term "infinitely faster RAM" in this article to mean "RAM that's maybe 20x or 100x faster than what you can get today. As measured by critical bottlenecks in applications. For my purposes here I'm saying that latency is the most critical fastness factor - and while acknowledging that there aren't generally accepted methods of defining what "faster memory" means - I think it's good enough for my argument below to assume that if the way the black box works behaves consistently as if the memory was X-times faster (or X-times sooner) than before - then that's a good enough understanding.
12
13	This also assumes we're on the same page - when it comes to agreement on - what is RAM? - which is a shifting subject I have written about before. For my purposes - if it behaves like RAM - and can transparently replace conventional RAM (chips, modules or boxes or markets) then that's good enough for now - without worrying about implementation details. I'm not going to speculate on the technology of the infinitely faster RAM - I'll leave that problem for someone else - (maybe You). In this article I'm posing the same kind of philosophical and business what-ifs which I did in earlier phases of the SSD and memoryfication markets - which asked - if we could get this new stuff - what new products and markets would we get? - and how would that change pre-existing markets.
14
15	No! to the infinitely fast one transistor memory cell. I'd like to make it clear I'm not interested here in the idea of so-called "ultra fast" transistors, memory cells and that ilk of research. As far as I'm concerned if you can't put many megabytes and preferably gigabytes of raw capacity into the infinitely faster RAM (at chip / module level) then it's not the kind of animal I'd talking about here.
16
17	some lessons from history - applications create markets and define acceptable latency
18
19	Upto the early 2000s the value propositions for different implementations of semiconductor RAM were graded by latency and power - and the order of precedence (DRAM, SRAM, SoC memory on chip - from slowest to fastest) hadn't changed since the dawning of their mutual market coexistence in the 1970s.
20
21	If you wanted bigger capacity - you chose DRAM. If you wanted faster latency at a board level of integration you chose SRAM which ran hotter and was smaller in capacity.If you wanted faster than that - there was no contest. It had to be SoC (usually in the form of RAM on a true ASIC or gate array but also latterly on FPGA).
22
23	At a board level - and system level - DRAM and SRAM reached their latency limits in about 1999 and haven't get any faster since.
24
25	It didn't matter so much in the early 2000s because enterprise processors weren't getting any faster either. And the shape of applications (users doing simple stuff on the internet ) meant that datacenters could get by with affordable technologies which offered higher densities and lower power (more users satisfied per box or watt or dollar) rather than users getting speed they didn't need and couldn't use. The computer industry didn't need faster memory. And when demand for more applications performance did grow - particularly in the early days of the cell phone market (and social networks) the enterprise SSD market took up the slack adequately - as there had been plenty of latency bottlenecks built around earlier generations of (rotating) storage.
26
27	Nowadays cell phones are coinage, spies and slot machines. And they've been joined by IoT. There's so much intelligence which can be gathered about the meaning of it all. But no memory or computing platforms fast enough to resolve everything which can imagined by the next master plan in a timely fashion.
28
29	memory world war 1 - flash versus DRAM - in enterprise storage
30
31	I guess the first time there was a serious challenge to the role of enterprise DRAM from another memory type in the acceleration space was in the early years of enterprise flash adoption (from around 2004). Which was fought out and soon won by flash arrays supplanting RAM SSDs.
32
33	If you'd asked most SSD people even as late as 2007 whether they really expected DRAM to be replaced by flash as the mainstream enterprise SSD based acceleration technology there were arguments which could be made for either. But (as we now know) by 2012 the RAM SSD market was effectively extinct. The principal reasons that a slower latency memory (flash) could and did replace a faster latency memory (DRAM) in an acceleration role were:-
34
35	Typical user installations needed more memory capacity than could be integrated by DRAM in a single box. The latency fabric from interfaces wrapped around these SSD assets negated the latency advantage of DRAM chips compared to flash chips. (The flash chips had much higher storage capacity per chip and required much less electrical power.)
36
37	Most of the easy acceleration advantages of enterprise SSDs came from read requests rather than writes. That's just the way that the legacy installed software base worked. That bias in the profile of memory R/W meant that the asymmetic R/W latency of flash chips - with reads being orders of magnitude faster than writes - was not a serious obstacle in adoption.
38
39	Those acceleration lessons - initially duelled out in Fibre Channel SAN rackmount boxes - were won by the time the PCIe SSD market got started. It showed that a faster memory could lose out to a slower memory in an acceleration focused role.
40
41	But that was storage... What does that tell us - if anything about different speeds of memory used as memory?
42
43	The early experiences (2014 to 2017) of tiered memory from the DIMM wars market - in which flash can be tiered with DRAM (using form factors as diverse as DIMMs, PCIe modules and even SATA arrays) is that there can be trade offs in big data applications whereby trading size of memory in the box can offset the native speed of raw memory (when the fastest memory is DRAM) for exactly the same reasons which pertained in flash storage accelerators. And the benefits of doing so have been mostly related to improvements in cost rather than any risk free overwhelming advantages in application latency. That's because the low hanging fruit of tiered flash speedup was mostly already harvested by the bottlenecks uncovered and bypassed by server based PCIe SSD adoption.
44
45	Here's one last look backwards at the lessons from history - before going on to speculate about the value of infinitely faster memory in the futre.
46
47	I think that if you could go back in time and take with you a warehouse of today's fastest and highest capacity DRAM chips - along with plug compatible adapters to retrofit them into past server and storage systems - then you wouldn't change the world of applications because most performance constrained servers in the past - already had the maximum amount of DRAM installed. And if they didn't - then there were so many bottlenecks built into interfaces and software that - even with an ideally configured modern memory array taken back in time and fitted as an upgrade - you would at best typically get a 2x or 3x speedup or - very often - get no speed up at all. That's why the early adoption of enterprise SSD accelerators was slow and problematic. There were too many problems baked into the ecosystem for any one new product to make enough of a change.
48
49	today's world of enteprise memoryfication and memory defined software
50
51	Today's computing market - where SSDs are everywhere and storage latency bands have a precise value and can be controlled - is better placed to consider and use faster memory systems. It's the only direction of travel to enable faster software.
52
53	The business needs are well understood. It's easy nowadays to see a direct link between faster decisions (based on diverse internet signals picked up and analyzed by real-time AI) and measurable economic outcomes. Also new business models and markets are being created by the application of heavy duty machine learning.
54
55	The software industry has had nearly a decade to become accustomed to thinking about having meaningful choices in latency - which are determined by different types and tiers of semiconductor devices. Most of the benefits of SSDs came when enough software changed to fit in with an SSD world. Those changes were hard to make because of decades of architectural lethargy in the leadup to the modern SSD era. The next stage of software change - towards more memoryfication - has already been underway - due to momentum and despite the lack of revolutionaty memory technologies to take hardware to the next level.
56
57	So if you could offer a GB scale memory chip with 20x or 100x lower latency than existing mainstream DRAM - there are companies who could do useful things with it.
58
59	simplistic valuation of much faster memory accelerator systems
60
61	The most obvious market sizing example for infinitely faster memory accelerator systems is their application in dealing with temporary data.
62
63	A simplistic way of looking at this is that overwhelmingly most of the data which enters processor space is temporary data. So if you have a memory accelerator which is 100x faster than the original memory which you used before to solve this type of problem - then provided that you don't run out of new data to feed into the machine and provided that the usable memory size for any single instance of the computation can fit into the memory space provided - then you need approximately 100x less machines to provide the same services.
64
65	The equivalence of speed and machines (one machine which is 100x faster being equivalent in capability to 100 slower machines) is similar to the SSD-CPU eqyuivalency model which helped to cost justify SSD accelerators in the early 2000s as server replacements rather than expensive $/TB storage.
66
67	However, the memory accelerator has a greater utility than this comparison might suggest on its own because the ability to solve problems sooner, and the ability to solve more complex problems for the first time within a shorter real-time period create new value propositions by making new algorithmic engines viable for new markets and applications.
68
69	This ability to create new markets is like the dynamic energy seen in the computer market in the 1980s and 1990s which began with microprocessors making computation cheaper but ended (around 1999) when limits were reached in how fast the GHz clock rate of any particular core could run. And the collective decision of server companies that fater was not necessary is partly to blame for the dumbing down of processor design architecture for nearly 20 years thereafter.
70
71	The good thing which emerged from that lack of investment in making commercial processors faster was that it created the fertile soil for the SSD acceleration market - which became the only game in town.
72
73	And now that there is a greater understanding of memory (and the interplay of roles and values between raw memory capacity and raw latency) and helped by an SSD rich ecosystem in which larger portions of any computing problem can be economically gathered into systems where the random access time can be arranged to be a handful of microseconds rather than tens of milliseconds - the creative juices of computing architecture have been turning to the much needed creation of new memoryfication compatible computing engines and new processor architectures.
74
75	That's what has been giving rise to the proliferation in recent years of commercial in-situ SSDs, processing in / near memory FPGA arrays and dedicated memory accelerators for machine learning and similar neural algorithms.
76
77	The early implementations which you can read about in the SSD news archives demonstrate 2 things.
78
79	The value of an Nx faster memory-compute accelerator can indeed be measured by at least the cost of all the previous traditional hardware which was needed to solve similar problems before. So 1 new PU (TPU etc) is indeed worth Nx conventional server CPU / GPU / when there is sufficient work to be within a particular problem shape universe. Their value is application specific. (Some workloads are accelerated better than others.)
80
81	Modern memory accelerators don't have to resemble either dumb memory or dumb processors. Provided that they can interoperate with conventional servers and infrastructure the new memory accelerators are best viewed as black boxes whose internal details may change and adapt (just like search engine algorithms) when more data suggests future areas of improvement.
82
83	This will creates an existential problem for makers of memory testers - because the future of high performance memory systems (where the money used to be) will become increasingly proprietary.
84
85	And as you can realize yourself this will create a cut-off point for manufacturers of high end server memory. Because high end memory systems will inevitably look more like a custom processor market.
86
87	And as for traditional processor makers the new memory accelerator systems don't care about and don't need their "instruction set" based backwards compatibilities and roadmaps - because the new ML / NN engine roadmaps (if they need any compatibility at all) will be "application" and "algorithm" based.... More like the kind of compatibilties you witness in successive generations of cloud APIs.
88
89	An interesting question is how many different types of memory accelerators the world needs and can support?
90
91	One view might be that the world doesn't need more than a handful (one for Google, Amazon, Apple, Baidu etc) because if the biggest benefit and visibility into design optimization only occurs at massive scale then those companies will each drive their own designs.
92
93	On the other hand if this is indeed the start of a new rennaissance in computing architecture - then you could argue that there will be the usual explosion of startups hoping to serve new markets created by the new ideas in architecture. (And there may be benefits of such new ideas which occur without being colocated in the cloud.
94
95	conclusion
96
97	Going back to my questions in the title...
98
99	Are we ready for infinitely faster RAM?
100
101	I think I've made a case for the answer being - Yes. More ready than we've ever been before.
102
103	And as to - what would it be worth?
104
105	My advice to founders of startups in infinitely faster memory accelerators is - don't sprinkle the number "infinity" about too much in your spreadsheets when guessing the market size or attached to the price which you think ideal customers would be prepared to pay. There are plenty of big numbers you can choose which are smaller and will still sound impressive without straining credulity.

$1-$3B 8tey's guess....