The ‘R1’ and it’s ram-ifications.

[Jimmyjames]

Apologies for the pun and if this topic has been discussed already.

Today I saw a link to an article Source, and the translation on Reddit concerning the R1 chip and its need for very low latency ram. Apparently it’s only 1GB, and bandwidth is 2Gbps vs LPDDR5X 8Gbps. However it has a total of 512 pins vs LPDDR5X 64 pins, meaning it has total bandwidth of 128GB/s or double that of LPDDR5X.

Do we think this has any implications for other products, or will it be limited to the Vision Pro?

Article translation below.

Hello to our readers who have a keen interest in the Information Technology (IT) market. Recently, we reported that SK Hynix plans to supply a new type of D-RAM for Apple's mixed reality (MR) device, 'Vision Pro'. Specifically, this D-RAM is designed to interact with Apple's new MR chip, 'R1'. Although SK Hynix has been reticent, stating "we cannot confirm client information," various media have highlighted this development.

In this series, we aim to delve deeper into several aspects of our report that were omitted. If I were to summarize this lengthy article in just one word, it would be "latency." It's fascinating to imagine the challenges Apple faced in developing a chip to reduce "operation time" and the technological choices SK Hynix made to meet Apple's demands for D-RAM. There's another point to note.




The D-RAM that SK Hynix will introduce today for Apple's Vision Pro is a custom product specifically for 'near memory' and 'edge devices'.

SK Hynix's High Bandwidth Memory (HBM), which we have covered in previous series, is a type of near memory closely attached to processors for server use. This new custom D-RAM represents another form of near memory, focused on consumer IT devices, also known as edge devices. This suggests a paradigm shift spreading across the memory hierarchy. Now, let's get to the main content.




Apple has established a 'dual chip' system within Vision Pro, incorporating its existing self-developed processor, M2, along with the MR device processor, R1. This R1 is equipped with SK Hynix's custom D-RAM. (Photo provided by Apple)

◇ Imagining the Challenges Faced by R1 Chip Developers

Firstly, let's discuss the concept of Vision Pro's ambitious R1 chip. Have you ever experienced Virtual Reality (VR) or Augmented Reality (AR) devices? If you have, you might have felt a bit dizzy, a phenomenon known as 'digital motion sickness'. Apple's R1 chip is their solution to this issue.

The main reason for digital motion sickness is the slower processing of images by MR devices compared to human senses and movements. Apple wanted to solve this speed issue with their chip. The Apple Vision Pro, with its R1 chip, processes incoming external information from various cameras and sensors, creating a new screen in just 0.012 seconds - eight times faster than the blink of an eye. This rapid data processing significantly reduces latency.

For the R1 to effectively reduce information delay, the accompanying D-RAM memory also needs to be different, aligning in step. Apple engineers likely pondered the following:

1. If the D-RAM itself could transmit information faster, the R1 would perform its inherent functions more effectively.
2. For quicker information exchange between the processor and D-RAM, the D-RAM and R1 must be positioned very closely.

Let's pause here. We are aware of SRAM (Static RAM), which temporarily stores data inside the processor, acting as a 'cache memory'. SRAM is known to be tens of times faster in processing information than D-RAM and is embedded right within the CPU, making the distance to the CPU negligible. Why not just increase the SRAM space within the R1 to reduce latency?




The issue is capacity. SRAM has a more complex design structure compared to D-RAM, requiring more transistors per bit. Consequently, it takes up more space and is more expensive to produce. This is why it's jokingly referred to as 'cash memory' in the industry.

The application scope of SRAM is limited. For example, a typical CPU might have 128 kilobytes (KB) of SRAM built next to it. In SRAM, 1,024,000 bits of data can be stored (1 byte = 8 bits, kilo = 1,000). Even with extensive usage, it's only a few megabytes (MB) in size – just enough for a song.

When compared to general-purpose D-RAM, the difference is stark. Modern DDR5 D-RAM has a capacity of 16 billion bits (16 gigabits), which is incomparable. Moreover, Apple's Vision Pro primarily processes high-volume 3D images – would MB-level capacity suffice?

While modern techniques exist to create SRAM chips that attach like a sticker onto processors, the inherent capacity limitations of SRAM would necessitate stacking dozens of layers, pushing the final market price and size of Vision Pro beyond reasonable limits. Thus, Apple likely approached memory companies for a custom D-RAM solution for their 'R1'.

◇ A Closer Look at SK Hynix's Custom D-RAM for R1

Now, let's examine the specifications of the custom D-RAM that SK Hynix supplied for the R1. Its capacity is understood to be 1Gb, meaning it can store approximately 1 billion bits of data.

This capacity might seem small compared to the latest mainstream D-RAM used in smartphones, LPDDR5X, which has a capacity of 16Gb per chip. However, it's deemed sufficient for assisting the R1 located right next to it. It provides a much larger storage space than typical SRAM.

The most crucial aspect is how SK Hynix increased the speed compared to existing D-RAM. This D-RAM has 512 input/output (I/O) pins, which is significant when compared to the 64 I/O pins on a single LPDDR5X D-RAM chip. It's like expanding a 64-lane highway by eight times, a concept referred to in the industry as 'wide I/O'.

However, the speed per I/O pin of this custom D-RAM is slower compared to smartphone D-RAM. The custom D-RAM has a speed of 2Gb/s per pin, while each pin of LPDDR5X D-RAM operates at 8.5Gb/s. But, the sheer number of pins compensates for this. With 512 pins at 2Gbps each, the total data transfer speed (bandwidth) reaches 1024Gb/s or 128GB/s. This is double the speed of 68GB/s bandwidth achieved by LPDDR5X D-RAM with 64 pins. Additionally, by using 'fan-out' packaging, which allows the D-RAM to attach closely to the R1 and operate as a single chip, the distance for data transfer is minimized.

In conclusion, SK Hynix's D-RAM for Apple's Vision Pro meets the requirements of having sufficient capacity, significantly faster speeds than existing mobile D-RAMs, and a reduced distance for data transfer. From Apple's perspective, this D-RAM would be highly appealing, providing the low-latency, high-speed D-RAM they sought to eliminate the sensation of motion sickness.

◇ Will 'Foundryization of Memory' Become a Trend?

This supply by SK Hynix is unlikely to significantly impact the company's production or revenue, as the industry consensus suggests. The annual production volume of the Vision Pro, set to launch next year, is estimated at a maximum of 1 million units. If one D-RAM chip is supplied per R1, it amounts to a total of 1 million chips, a relatively small scale considering the annual global smartphone shipment is around 1 to 1.2 billion units.

However, the message this project sends to the industry is quite intriguing. If the launch of Apple's MR device leads to an unexpectedly large application D-RAM market, the pioneers will have the advantage.

Another interesting aspect is the 'foundryization of memory'. Traditionally, the memory industry's hallmark has been 'produce first, sell later', akin to manufacturing consumer goods with standardized specifications for mass production and subsequent sales.

However, SK Hynix's approach with this custom D-RAM closely resembles that of a foundry business. They took pre-orders for a client-specific D-RAM and then proceeded with production – a concept referred to as 'foundryization of memory'.

This unique project form is expected to increase in the future for two reasons. Firstly, the semiconductor industry is paying significant attention to advanced processor-memory combinations and packaging technologies. For example, AMD's 'V-Cache', which combines SRAM chips to create up to 64MB and then integrates them with processors, is a notable example. As such cases increase, the likelihood of D-RAM and system semiconductor combinations also grows.

Secondly, there's the evolution of form factors. Moving beyond smartphones and laptops shaped like boxes or bricks, we now see devices that fold, roll, or assume entirely new forms like Vision Pro. The transformation of end-point, or edge devices, is boundless, suggesting that memory will not be confined to 'off-the-shelf' products.

Thank you for navigating through this lengthy and complex tech journey. Take care in the hot summer and have a happy weekend.

 

[dada_dave]

That’s really interesting, probably limited in its utility to the R1 … for now. But who knows in the future what other custom processors might need a small amount of very high bandwidth DRAM that is not HBM?

 

[leman]

What I find very interesting is that the R1 package looks like a tile solution and the DRAM is very closely placed to the main chip. This is different from the current M-series where things are still more separate.

I think it's a cool thing too. I mean, imagine similar technology on upcoming M-series. Still, probably not a replacement for SLC given the power requirements.

 

[dada_dave]

What I find very interesting is that the R1 package looks like a tile solution and the DRAM is very closely placed to the main chip. This is different from the current M-series where things are still more separate.

View attachment 28291


I think it's a cool thing too. I mean, imagine similar technology on upcoming M-series. Still, probably not a replacement for SLC given the power requirements.

I would think the SLC would still be faster too, no? How I read it is that compared to RAM and standard cache, this is something in between but still closer to the former. It’s still only got twice the bandwidth of standard LPDDR5X though closer and presumably less power hungry and maybe a touch lower latency (although maybe not since individual pins are slower)? Perhaps the M-series could benefit: another cache level or if it scales economically even a better replacement for DRAM.

 

[thenewperson]

Is this feasible as a replacement for LPDDR5? @leman I remember you mentioned in another thread about their need for more RAM bandwidth. Could this be a solution they're looking into? Or is that just wishful?

(Sorry I know I'm asking a lot of questions about something we don't know much about )

 

[leman]

Is this feasible as a replacement for LPDDR5? @leman I remember you mentioned in another thread about their need for more RAM bandwidth. Could this be a solution they're looking into? Or is that just wishful?

(Sorry I know I'm asking a lot of questions about something we don't know much about )

I don't think that anyone can say, really. I saw some speculations that these dies might not even be DRAM, but SRAM instead (using a cheaper more mature node). Apple does have some patents in this direction. Could also be something they are exploring for M-series. Putting SLC and memory controllers to a separate die could save costs, free up the valuable die space for more compute, but also increase the cache size. The latter in turn would allow Apple to scale performance without significantly increasing the DRAM bandwidth. I mean, imagine an M-series chip with 1-2GB low-power SRAM cache, that would be quite a device

 

[Jimmyjames]

I wonder if the function of the R1 could be integrated into future ‘M’ chips? Would it potentially help with graphics related tasks?