Complete memory hierarchy of modern pc. Hierarchical structure of memory. Register and register memory view

Memory hierarchy (performance and size)

Building a disk array

When designing high-performance computers and systems, there are many tradeoffs to be made, such as size and technology for each level of the hierarchy. You can consider a set of different memories (m 1, m 2, ..., m n) located in the hierarchy, that is, each m i level is subordinate to the m i-1 level of the hierarchy. To reduce latency at higher layers, lower layers can prepare data in large buffered chunks and, when the buffer is full, signal the upper layer about the possibility of receiving data.

Often there are 4 main (enlarged) levels of the hierarchy:

Most modern PCs use the following memory hierarchy:

Most programmers usually assume that memory is divided into two levels, RAM and disk drives, although in assembly languages ​​and assembly-compatible (such as) there is a possibility of direct work with registers. Taking advantage of the memory hierarchy requires synergy between the programmer, hardware, and compilers (and basic support in operating system):

Many programmers do not take memory layering into account when programming. This approach works until the application experiences a performance degradation due to a lack of memory subsystem performance. When correcting the code (refactoring), it is necessary to take into account the presence and peculiarity of the operation of the upper levels of the memory hierarchy in order to achieve the highest performance.

Decentralization of governance implies hierarchical organ computer structure . The control unit of the main, or central, processor determines only the sequence of work of the slave modules and their initialization, after which they continue to work according to their own control programs. The results of performing the required operations are presented by them "up the hierarchy" for the correct coordination of all work.

Slave modules (controllers, adapters, KVV) can, in turn, use special tires or highways to exchange information. Standardization and unification have led to the emergence of a hierarchy of tires and their specialization. Due to differences in the operating speeds of individual devices and PC structures, the following appeared:

system bus - for the interaction of main devices;

local bus - to speed up the exchange of video data;

peripheral bus - for connecting "slow" peripherals.

Hierarchical principle of construction and management characteristic not only for the structure of the computer as a whole, but also for its individual subsystems. For example, a computer memory system is built on the same principle.

The principle of hierarchical memory

It is desirable for the user to have a computer memory of large information capacity and high speed. However, single-level memory design does not allow meeting these two conflicting requirements at the same time. Therefore, the memory of modern computers is built on a multilevel, pyramidal principle.

Processors may include super fast memory small capacity, formed by several tens or several hundred registers with a fast access time of one processor cycle (nanoseconds, ns). Data that is directly used in processing is usually stored here.

The next level forms cache memory , or notebook type memory , which is a buffer memory for storing active pages of tens and hundreds of Kbytes. In modern PCs, it, in turn, is divided: into the L1 cache (E n = 16-32 Kbytes with an access time of 1-2 processor cycles); on the L2 cache (E n = 128-512 KB with an access time of 3-5 cycles) and even on the L3 cache (E n = 2-4 MB with an access time of 8-10 cycles). The cache memory, being faster, is intended to speed up the retrieval of program instructions and processed data. Associative data retrieval is possible here. The main volume of user programs and data to them is located in the random access memory (capacity - millions of machine words, sampling time - 10-20 processor cycles).

Some of the data-constants required by the operating system to control calculations and are used most often can be located in read-only storage(ROM). At the lower levels of the hierarchy are external storage devices on magnetic media. They can be implemented on hard and floppy magnetic disks, magnetic tapes, magneto-optical disks, etc. They are distinguished by low speed and very high capacity.

The organization of the advance exchange of information flows between memory of different levels with decentralized management of them allows us to consider the memory hierarchy as an abstract single apparent (virtual) memory. The coordinated work of all levels is ensured under the control of the operating system programs. The amount of memory used, due to this, significantly exceeds the RAM.

Memory in modern computers is built on a hierarchical basis. One of the phenomena characteristic of von Neumann computers is the principle of locality. This means that for a limited period of time, each executable program does not make calls to all of its data and commands evenly, but tends to access a limited part of its address space. For example, such a situation arises in scientific and engineering calculations when solving equations, when small sections of code are executed containing a large number of nested loops and subroutines and with their help more and more relatively small portions of data are processed with frequent access to intermediate results. It follows from this that in order to increase performance without significantly increasing the cost, memory can be built according to a hierarchical principle. At the same time, there are several levels of the memory hierarchy with different volumes and access times. Information is distributed among them for storage in accordance with its importance, frequency of access and "urgency" of service. Each level of the hierarchy is characterized by a certain cost of storing one byte (word) and the rate at which a byte (word) or block of several bytes (words) is retrieved.

Interaction within a memory hierarchy between two adjacent levels is usually considered.

The smallest unit of information that can either be present or absent at one of the two interacting levels in the hierarchy will be called a block. The block size can be either fixed or variable. If this size is fixed, then the memory size is a multiple of the block size.

When referring to each level of the hierarchy, two outcomes are possible. Either the desired block is at the requested level (hit) or it is absent (miss) and you have to turn to the next level of the hierarchy, which has a lower sampling rate. The efficiency of the storage engine can be described by the hit rate. The mechanism for organizing each level of the hierarchy should be able to accommodate any block from a lower level of the hierarchy. Hence, a certain associative mechanism for mapping a block from the lower level of the hierarchy to the higher one is needed.

The memory hierarchy in a computer is shown in Fig. 17. Memory capacity grows from top to bottom in the figure, cost of storing a byte (word) and byte (word) retrieval rate grows from bottom to top.

The topmost level of the hierarchy is register memory. The capacity of register memory usually does not exceed several tens (sometimes hundreds) of bytes or words. Physically, the register memory is located directly in the central processor, so the memory access time is minimal and does not exceed the duration of 1 processor cycle. In the register memory, as already noted, the most operative data are stored, such as the addresses of the memory cells being processed at the moment, the cycle counters, the operands of the arithmetic operations performed at the current moment.

Massive RAM has a volume of several tens of kilobytes to several hundred megabytes. Due to a number of technological features, the speed of access to RAM grows much slower than the speed of central processors, and usually the access time is from 5 to 15 processor cycles. To speed up the retrieval of information between the main memory and central processing unit another level of hierarchy is introduced - intermediate, cache memory or cache memory. The word cache in translation from English means hiding place, secret warehouse, hidden stock. The meaning of this term is that the cache memory is invisible, transparent to the central processor. The cache memory controller manages the operation of the cache memory. It looks at the central processor's accesses to the main memory and determines whether the required data is stored in the cache (cache hit) and, therefore, can be read from it or not (cache miss), and therefore, it will be necessary to access the slower main memory. The access speed to the cache memory cell can be from 3 to 7 processor cycles. The cache memory, in turn, can itself be implemented according to the hierarchical principle, i.e. can be divided into several levels:

· Cache of the first level, several kilobytes in size with an access time of 2-3 cycles, built directly into the processor;

· Cache of the second level with an access time of 3-5 cycles and a volume of several tens of kilobytes, located on the same board with the central processor;

Cache of the third level with an access time of 5-7 cycles and a volume of several hundred kilobytes per motherboard etc.

Later in this section, we will return to the discussion of cache memory and take a closer look at its functioning.

The lowest level of the memory hierarchy is large external memory. Its volume is several orders of magnitude larger than the capacity of the main RAM. The processor does not have the ability to quickly access data located in external memory, and, for processing, they must be transferred to the main memory. RAM and external memory together form what is called virtual memory.

A short history of computing and creation software allowed computer developers to draw one important conclusion: the physical memory available on the computer is always not enough to meet the needs of the programmer ... It is unprofitable for a computer to supply a computer with an enormous random access memory for economic reasons. This realization led to the idea of ​​organizing virtual memory . Virtual is called an abstract resource that does not actually exist physically, but is modeled using the available technical means.

In a system with virtual memory, programs "consider" that they have been provided with some rather large address space (the space provided to a program or a group of programs can significantly exceed the amount of physical memory available in the computer). In this case, the workspace is divided into blocks - pages (typical page size is 2-4 Kbytes). Pages related to the task being performed are located partly in high-speed memory, partly on some slower and cheaper storage device. The virtual memory manager monitors memory accesses and an interrupt occurs if the requested page is not in physical memory. The interrupt handler makes a call to an external device to read the missing page, after which the execution of the command that caused the interrupt is repeated again. Thus, application program"Does not notice" that her page was missing from memory. Naturally, due to the need for periodic calls to external memory for missing pages, the program execution speed decreases. This slowdown is a "necessary evil" - the price to pay for the large size of the available address space. Designers of virtual addressable processors provide various means to minimize the slowdown effect.

Virtual memory systems can be divided into two classes: systems with fixed block sizes, called pages, and systems with variable block sizes, called segments. Both types of virtual memory organization are discussed below.

), various technical solutions are used that have excellent performance, both technical and price and weight-dimensional. Long-term storage in expensive super-operative and even random access memory is usually not profitable, so this kind of data is stored on drives - disk, tape, flash, etc.

When designing high-performance computers and systems, there are many tradeoffs to be made, such as size and technology for each level of the hierarchy. You can consider a set of different memories (m 1, m 2, ..., m n) located in the hierarchy, that is, each m i level is subordinate to the m i-1 level of the hierarchy. To reduce latency at higher layers, lower layers can prepare data in large buffered chunks and, when the buffer is full, signal the upper layer about the possibility of receiving data.

Often there are 4 main (enlarged) levels of the hierarchy:

1. Internal processor memory (registers organized into a register file and processor cache).
2. System RAM (RAM) and auxiliary memory cards.
3. Hot-access drives (On-line mass storage) - or secondary computer memory. Hard drives and solid-state drives that do not require long (seconds or more) actions to start receiving data.
4. Drives requiring media switching (Off-line bulk storage) - or tertiary memory. This includes magnetic tapes, tape and disk libraries that require lengthy rewinding or mechanical (or manual) media switching.

Most modern PCs use the following memory hierarchy:

1. Processor registers organized in a register file are the most fast access(about 1 clock cycle), but only a few hundred or, rarely, thousands of bytes in size.
2. Level 1 processor cache (L1) - access time of the order of several cycles, tens of kilobytes in size
3. L2 processor cache - longer access time (2 to 10 times slower than L1), about half a megabyte or more
4. Processor cache level 3 (L3) - access time of about a hundred cycles, several megabytes in size (used in mainstream processors recently)
5. System RAM - access times range from hundreds to perhaps thousands of clock cycles, but huge sizes of several gigabytes, up to hundreds. The access time to RAM can vary for different parts of it in the case of complexes of the NUMA class (with non-uniform memory access)
6. Disk storage - many millions of cycles, if the data was not cached or buffered in advance, sizes up to several terabytes
7. Tertiary memory - delays of up to several seconds or minutes, but practically unlimited volumes (tape libraries).

Most programmers usually assume that memory is divided into two levels, random access memory and disk drives, although in assembly languages ​​and assembly-compatible (such as) it is possible to work directly with registers. Taking advantage of the memory hierarchy requires a collaborative effort from the programmer, hardware, and compilers (as well as basic support in the operating system):

• Programmers are responsible for organizing the transfer of data between disks and memory (RAM) using file I / O; Modern operating systems also implement this as paging.
• The hardware is responsible for organizing the transfer of data between memory and caches.
• Optimizing compilers are responsible for generating code that, when executed, the hardware efficiently uses the processor's registers and cache.

Many programmers do not take memory layering into account when programming. This approach works until the application experiences a performance degradation due to a lack of memory subsystem performance. When correcting the code (refactoring), it is necessary to take into account the presence and peculiarity of the operation of the upper levels of the memory hierarchy in order to achieve the highest performance.

see also

Write a review on "Memory Hierarchy"

Literature

• Mikhail Guk "Hardware IBM PC" St. Petersburg 1998
• , John L. Hennessy, 2012

Notes (edit)

Calculating machine (Computer configuration) Non-volatile memory: drives, drives and media Information output device and Multimedia Information input device (by main function) Games and entertainment Communication devices and (tele) communications Power supply Other

Excerpt Characterizing the Hierarchy of Memory

Sonya, as if not believing her ears, looked with all her eyes at Natasha.
- And Bolkonsky? - she said.
- Oh, Sonya, if only you could know how happy I am! - said Natasha. - You don't know what love is ...
- But, Natasha, is it really all over?
Natasha looked at Sonya with large, open eyes, as if she did not understand her question.
- Well, you refuse to Prince Andrey? - said Sonya.
“Oh, you don’t understand anything, don’t talk nonsense, you listen,” Natasha said with instant annoyance.
“No, I can't believe it,” Sonya repeated. - I do not understand. How did you love one person for a whole year and suddenly ... You saw him only three times. Natasha, I don't believe you, you're being naughty. In three days to forget everything and so ...
“Three days,” Natasha said. - I think I have loved him for a hundred years. It seems to me that I have never loved anyone before him. You cannot understand this. Sonya, wait, sit down here. - Natasha hugged and kissed her.
- I was told that it happens and you heard it correctly, but now I have only experienced this love. This is not what it used to be. As soon as I saw him, I felt that he was my master, and I was his slave, and that I could not help but love him. Yes, a slave! What he tells me, I will do. You don't understand this. What am I supposed to do? What am I to do, Sonya? - Natasha said with a happy and frightened face.
“But think about what you’re doing,” Sonya said, “I cannot leave it like that. These secret letters ... How could you let him do this? She said with horror and disgust, which she could hardly hide.
“I told you,” Natasha answered, “that I have no will, how you don’t understand this: I love him!
“So I won't allow it to happen, I’ll tell you,” Sonya cried out with tears bursting out.
- What are you, for God's sake ... If you tell me, you are my enemy, - Natasha spoke up. - You want my misfortune, you want us to be torn apart ...
Seeing this fear of Natasha, Sonya cried with tears of shame and pity for her friend.
- But what happened between you? She asked. - What did he tell you? Why doesn't he go to the house?
Natasha did not answer her question.
“For God's sake, Sonya, don’t tell anyone, don’t torture me,” Natasha begged. - You remember that you cannot interfere in such matters. I opened it for you ...
- But why these secrets! Why doesn't he go to the house? - asked Sonya. - Why is he not looking for your hand directly? After all, Prince Andrew gave you complete freedom, if so; but I don't believe it. Natasha, did you think what secret reasons could be?
Natasha looked at Sonya with surprised eyes. Apparently, for the first time this question presented itself to her and she did not know how to answer it.
- What are the reasons, I do not know. But there are reasons!
Sonya sighed and shook her head in disbelief.
“If there were reasons…” she began. But Natasha, guessing her doubt, interrupted her in fright.
- Sonya, you can't doubt him, you can't, you can't, do you understand? She shouted.
- Does he love you?
- Does she love? Natasha repeated with a smile of regret about her friend's lack of understanding. - You read the letter, did you see it?
- But if he is an ignoble person?
"He! ... an ignoble man?" If you only knew! - Natasha said.
- If he is a noble person, then he must either declare his intention, or stop seeing you; and if you don’t want to do it, then I’ll do it, I’ll write to him, I’ll tell Daddy, ”Sonya said resolutely.
- Yes, I can not live without him! Cried Natasha.
- Natasha, I don't understand you. And what are you saying! Remember your father, about Nicolas.
- I do not need anyone, I do not love anyone but him. How dare you say that he is ignoble? Don't you know that I love him? - Natasha shouted. “Sonia, go away, I don’t want to quarrel with you, go away, for God's sake, go away: you see how I suffer,” Natasha shouted angrily in a restrained irritated and desperate voice. Sonya burst into tears and ran out of the room.
Natasha went to the table and, without thinking for a minute, wrote the answer to Princess Marya, which she could not write the whole morning. In this letter, she briefly wrote to Princess Marya that all their misunderstandings were over, that, taking advantage of the generosity of Prince Andrei, who gave her freedom when leaving, she asks her to forget everything and forgive her if she is to blame for her, but that she cannot be his wife ... All this seemed to her so easy, simple and clear at that moment.

On Friday, the Rostovs were supposed to go to the village, and on Wednesday the count went with the buyer to his Moscow region.
On the day the count left, Sonya and Natasha were invited to a big dinner at the Karagins', and Marya Dmitrievna took them. At this dinner Natasha again met with Anatole, and Sonya noticed that Natasha was saying something to him, wishing not to be heard, and during the dinner she was even more agitated than before. When they returned home, Natasha, first with Sonya, began the explanation that her friend had been waiting for.
- Here you, Sonya, said various nonsense about him, - Natasha began in a meek voice, the voice that children say when they want to be praised. - We have explained to him today.
- Well, what, what? Well, what did he say? Natasha, how glad I am that you are not angry with me. Tell me everything, the whole truth. What did he say?
Natasha pondered.
- Oh Sonya, if you knew him as I do! He said ... He asked me how I promised Bolkonsky. He was glad that it was up to me to refuse him.
Sonya sighed sadly.
“But you didn’t refuse Bolkonsky,” she said.
- Or maybe I refused! Maybe it's all over with Bolkonsky. Why do you think so badly of me?
- I don't think anything, I just don't understand it ...
- Wait, Sonya, you will understand everything. You will see what kind of person he is. Do not think badly about me or about him.
- I do not think bad about anyone: I love everyone and feel sorry for everyone. But what should I do?
Sonya did not give up on the gentle tone with which Natasha addressed her. The softer and more searching was the expression on Natasha's face, the more serious and stern Sonya's face was.
“Natasha,” she said, “you asked me not to speak to you, I didn’t speak, now you yourself started. Natasha, I don't believe him. Why is this secret?
- Again, again! Natasha interrupted.
- Natasha, I'm afraid for you.
- Why be afraid?
“I’m afraid that you will ruin yourself,” Sonya said resolutely, herself frightened by what she said.
Natasha's face again expressed anger.
- And I will destroy, I will destroy, I will destroy myself as soon as possible. None of your business. Not you, but I will feel bad. Leave, leave me. I hate you.
- Natasha! - Sonya cried out in fright.
- I hate, I hate! And you are my enemy forever!
Natasha ran out of the room.
Natasha did not speak to Sonya anymore and avoided her. With the same expression of agitated surprise and criminality, she walked about the rooms, taking now and then for another occupation and immediately abandoning them.
No matter how hard it was for Sonya, she, without taking her eyes off, watched her friend.
On the eve of the day on which the count was supposed to return, Sonya noticed that Natasha had been sitting all morning at the drawing-room window, as if expecting something, and that she had made some sign to the passing soldier whom Sonya took for Anatole.

ORGANIZATION OF COMPUTER MEMORY

At the heart of the memory hierarchy modern computers there are two principles:

The principle of locality of references;

Cost / performance ratio.

The locality of references is subdivided into spatial and temporal locality. Spatial locality means that after accessing a certain memory cell, it is most likely that the next access will occur to one of the neighboring (or nearby) cells. Temporary locality means that in the nearest time after accessing a memory cell, the contents of this cell are most likely to be required again.

The memory hierarchy of modern computers, shown in Figure 5.1, is built on several levels, with the higher level being smaller, faster, and having a higher cost per byte than the lower level. This approach optimizes the cost / performance ratio.

The levels of a hierarchy are interrelated: all data at one level can also be found at a lower level, and all data at that lower level can be found at the next lower level, and so on, until we reach the bottom of the hierarchy. The smallest piece of information that can either be extracted or moved between levels is called block... The block size can be either fixed or variable. If this size is fixed, then the memory size is a multiple of the block size.

The memory levels located between the register and main memory are called cache memory. The cache block is called string(line).

Although the memory hierarchy consists of many levels, at any given time we are dealing with only two adjacent levels. Successful or unsuccessful access to a higher level are named accordingly hit or miss... A hit means that a block of memory was found at a higher level, while a miss means that it was not found at that level. On a miss, the required block is moved to a higher level.

Hit rate(hit rate) or hit ratio (hit ratio) is the proportion of hits found at a higher level (often represented as a percentage). Share of misses(miss rate) is the percentage of hits that were not found at a higher level.

Since improved performance is the main reason for the emergence of memory hierarchy, the proportion of hits and misses is important characteristic... The hit time is the time to access a higher level of the hierarchy, which includes, in particular, the time it takes to determine whether the hit is a hit or a miss. The miss penalty is the time to replace a block at a higher level with a block from a lower level, plus the time to transfer that block to the desired device (usually the processor). Loss on a miss includes two components: access time - the time to access the first word of the block in case of a miss, and transfer time - additional time to transfer the remaining words of the block. Access time is related to the latency of the lower layer memory, while the transfer time is related to the channel bandwidth between the memory devices of two adjacent layers.

To describe a certain level of the memory hierarchy, it is necessary to answer the following four questions:

1) Where can a block be placed at the top level of the hierarchy? (block placement).

2) How to find a block when it is at the top level? (block identification).

3) Which block should be replaced in case of a miss? (block replacement).

4) What happens during the recording? (write strategy).