Memory Hierarchy: An Organization of Memory Types in Levels

Computer memory is an essential component that functions similarly to a human brain. Memory, one of the basic functions of a computer , is an electronic place for storing data, instructions, and the results of executed programs.

Different types of memory exist, such as primary memory , cache memory, secondary memory (magnetic tapes or disks), registers, etc. Besides data storage as a common purpose, each memory type has its own characteristics, speed, access time, and storage capacity.

These different types of memory are organized into levels based on access time and cost per bit. This organization, called memory hierarchy , helps the processor access data from memory in the shortest possible time.

Cost Per Bit: It is the monetary cost in cents or dollars per memory unit.

This blog post aims to enlighten you on memory hierarchy and its design and characteristics.

So, let us get started!

What is Memory Hierarchy?

The memory hierarchy is an organization of different types of memory into levels to minimize the access time for a computer’s CPU ( Central Processing Unit ) and maximize a system’s overall performance.

In other words, memory hierarchy is a meaningful arrangement and visualization of different memory types based on their performance, access time, and cost per bit. This hierarchy helps developers create new systems that have a perfect performance-to-cost ratio.

Here is the organization of types of memory into different levels:

• Level 0: CPU Registers
• Level 1: Cache Memory
• Level 2: Main Memory
• Level 3: Magnetic Disk
• Level 5: Optical Disk / Magnetic Tape

The concept of the memory hierarchy is based on the “ locality of references .” It is a phenomenon where a computer program accesses the same set of memory locations for a specific timeframe. In other words, it implies that a specific computer program accesses instructions whose addresses are close to each other.

Why Do Computers Need Memory Hierarchy?

As discussed earlier, each type of computer memory has a different access time, cost per bit, storage capacity, and performance.

For instance, cache memory, main memory, and CPU registers have faster access times but have limited storage densities. On the other hand, magnetic disks/tapes and optical disks have slower access times but higher storage capacities.

The memory hierarchy arranges these storage types into levels based on their shortest access times. This makes it easy for the computer’s processor to quickly retrieve data it requires for the execution of ongoing processes.

The faster the access time of memory, the higher the cost per bit.

CPU registers, followed by the cache memory and main memory, have faster access times. The further sequence is magnetic disks → optical disks → magnetic tapes.

Types of Memory Hierarchy

The two broad categories of memory hierarchy are – External/Secondary Memory and Internal/Primary Memory.

• Internal/Primary Memory

Internal memory stores data that a computer’s processor needs to access quickly.

CPU registers, cache memory, and main memory fall under this category. A computer’s CPU can directly access them, increasing the access time. However, these memory types have limited storage capacities.

• External/Secondary Memory

External memory stores enormous amounts of data for long terms. Hence, it is ideal for persistent data storage.

Magnetic disks, magnetic tapes, and optical disks fall under this category. They are not directly accessible by the processor. Instead, they need an I/O module.

Memory Hierarchy Design: Level 0 to Level 4

Let us now discuss the different levels of memory hierarchy in detail below.

The first 3 levels – Level 0, Level 1, and Level 2, are primary storage devices or internal memory.

1. CPU Registers – Level 0

A CPU register, or a processor register, has level 0 in the memory hierarchy. It is the smallest electronic place to hold data, part of a computer's CPU. It can hold any kind of data, such as a memory address (location) or instruction.

CPU registers store data and instructions frequently accessed by a processor. They have the fastest access time as they are located inside the processor.

As registers are the fastest computer memory, they are the costliest too. However, registers have the lowest storage capacity, i.e., 16 to 64 bits.

The primary purpose of registers is to allow a processor to access the data they hold quickly. Different types of registers are available, such as general-purpose registers, accumulators, data registers, instruction registers (IR), program counters (PC), address registers, etc.

2. Cache Memory – Level 1

The next level, level 1, in the memory hierarchy, is cache memory. It is a high-speed memory and stores data a CPU frequently uses for the execution of ongoing operations. Its primary objective is to minimize the time required to access the data from the main memory.

Cache memory has a faster access time but is slower than CPU registers. Some computers have cache memory placed on the processor, while some have close to it. This makes it easy for the processor to access data as quickly as possible. Also, it is expensive but less than CPU registers.

3. Main Memory – Level 2

The main memory, or primary memory, stores data and instructions a computer’s CPU requires to perform certain ongoing tasks.

Random Access Memory (RAM) is the best example of the main memory. The processor accesses the RAM directly and retrieves data as and when required. Also, it stores the results of the executed tasks or programs to access them for subsequent operations quickly.

RAM also has a faster access time but is slower than CPU registers and cache memory. Though it is more expensive than secondary storage devices, it is less pricey than CPU registers and cache memory.

Two types of RAM are - SRAM and DRAM.

• Static RAM

SRAM leverages flip-flops to store bits of data. It is an electronic circuitry of 6 transistors, which requires a consistent power supply to retain data.

• Dynamic RAM

DRAM uses a single capacitor and a single transistor to store data bits. As a capacitor tends to lose its charge and electrons, it needs to be refreshed periodically.

Hence, SRAM is much faster than DRAM but has a limited storage density.

All the types we discussed above are volatile memory and primary storage devices. They need consistent power to retain their data.

Now, there are secondary storage devices in the memory hierarchy.

4. Magnetic Disk (Disk Storage) – Level 3

A magnetic disk is a secondary storage device. It uses the magnetization process to read, write, and access data. It is simply a circular plate covered with a magnetic coating. It has tracks and sectors to store data. Tracks are circular divisions, further divided into sectors that store data blocks.

Hard disks, zip disks, and floppy disks are common examples of magnetic disks.

Magnetic storage media have large storage capacities, unlike primary storage devices. Also, they are cost-effective. However, a computer’s CPU cannot directly access data stored in secondary storage devices, making the access time longer.

The primary objective of secondary storage is to store data persistently for long periods of time. In short, data backup is its major goal.

5. Optical Disk – Level 4

Level 4 devices are commonly referred to as tertiary storage devices.

An optical disk is an electronic storage device that leverages a low-powered laser beam to read and write data. It is a flat, circular disk that stores data in the form of microscopic data pits and lands.

Compact disks (CDs), Digital Versatile Disks (DVDs), and Blu-Ray Disks are three major types of optical disks available today.

Optical disks offer extremely high storage capacities at affordable prices. They are removable devices and are primarily used as a solution to transfer data between computers or maintain data backup.

6. Magnetic Tape – Level 4

A magnetic tape is a magnetic data storage medium consisting of a long, narrow strip of plastic film with a magnetic coating. It comes with a huge storage capacity. The access time of a magnetic tape is very large. Hence, retrieving data from the strip takes a lot of time.

When you need information stored on magnetic tape, you can mount it to allow the CPU to access the required data. Once the required data is retrieved, you can unmount the magnetic tape.

Characteristics of Memory Hierarchy

The following are the parameters the memory hierarchy uses to organize memory into different levels –

1. Access Time

In terms of memory, the access time is the time required for the computer processor to read data from memory.

As we move from level 0 to level 4 in the memory hierarchy, the access time increases.

The following table depicts the access time of each memory type in the hierarchy:

2. Performance

The memory hierarchy has improved the overall performance of computer systems. Before it was designed, there was a speed gap between CPU registers and other memory types, which resulted in low system performance. In other words, there was a significant speed gap between CPU registers and other large storage media.

To improve the speed gap and the system performance, the memory hierarchy was designed. It introduced other storage mediums between CPU registers and high-capacity storage devices.

3. Cost Per Bit

We have already stated, “The faster the access time, the higher the cost per bit”. This implies that CPU registers have the fastest access time; hence, they are the costliest. They are followed by cache memory and the main memory.

On the other hand, secondary and tertiary storage mediums are less expensive, as they have slower access times.

4. Capacity

The capacity of storage mediums increases as we move from the top to the bottom of the memory hierarchy. It refers to the volume of data memory can store.

Advantages of Memory Hierarchy

Here are some remarkable advantages of the memory hierarchy:

• Faster Access

The memory hierarchy organizes memory types into levels to minimize data access time. The access time increases as we move down the hierarchy. CPU registers are placed on the topmost level, as they have the fastest access time, allowing the processor to fetch data quickly.

• Cost-Effective

The organization of memory types into levels makes the system more cost-effective. This means the system can use the costliest memory only when required rather than using it everywhere in the system.

For instance, CPU registers or cache memory are expensive and have limited data storage capacity. If the system uses them only when required, it saves much money.

• Improved Processing Speed

The memory hierarchy has significantly improved the system’s performance by allowing the CPU to access data from the fastest memory. This helps the CPU to perform operations quickly and efficiently.

• Efficient Use of Resources

Another advantage of the memory hierarchy is that it enables the efficient use of resources. For instance, it stores frequently accessed information in cache memory with a faster access time rather than in memory with slow access times.

Conclusion

Here ends our discussion on the memory hierarchy. It organizes various memory types into levels according to minimum access time. Its primary aim is to minimize access time and improve the system’s overall performance.

The first three levels – Level 0, Level 1, and Level 2 – are primary storage mediums. The last two levels – Level 3 and Level 4 – are secondary and tertiary storage mediums, respectively.

Primary storage devices – CPU registers, cache memory, and main memory – have faster access times than secondary storage devices. The CPU can directly fetch the required data. However, they offer limited storage at very high prices.

On the other hand, secondary and tertiary storage devices – magnetic disks, optical disks, and magnetic tapes – have slower access times. But they offer high storage capacities at lower prices.

People are also reading:

• What is Recycle Bin?
• Advantages and Disadvantages of Computer
• Limitations of Computer
• Computer Memory Units
• Functions of Computer
• Block Diagram of Computer
• What is Control Unit?
• Input Devices of Computer
• Output Devices of Computer
• Difference Between SSD and HDD