Cache Cohernce with Multi-Processor Author: BNN Reprinted from: linuxforum CPU and compiler plate Essentials
Just finished the article about Cache Coherence, I found that BNN2 had a good text before, I know that I didn't have such a business:) Recently Work with Dual CPU Kernel Part. For Dual CPU, or We Say, Multi- processor, the big challenge part for a kernel is how to handle the cache coherence Conceptually, two choices -... Write Invalidate and Write Update We will talk about Write Invalidate today Typically, there are two protocols falling into the Write Invalidate protocol, namely, the Write-Through Write Invalidate Protocol and the Write Once (Or Write-Back Write Invalidate Protocol) Note that the well-known MESI protocol is derived from the Write Once that's why we will focus on the Write Once here... - ------------------------- WRITE: WRITE PROTOCOL IS TO OFFSET The Shortcomings of Write-Through Write-Invalidate Protocol, Which Will Introduce Extra Bus traffic onto the system bus Write Once works basically as follows:. (. Assume * the hardware snoop enabled over the shared system bus * The cache is Write Back) There are four states for Write Once protoco l - Valid, Reserved, Dirty and Invalid Initial State:. * When a LOAD MISS, the data will be loaded into cache line and the state goes to VALID state // Please note here, Write Once protocol will promise that your loaded data .? in memory will be the latest Why If at the time you tried to load a cache line, there is an modified copy in another CPU, the snoop protocol will abort the load bus transaction; flush another cpu's data into main memory and then resume THE ABORTED THE The Requesting CPU WILL THEN GET The Updated Data .... Now, Let's Investigate The State Machine of Write Once Protocol. ************* VALID State: ** ************* WHEN a LOAD HIT, WE Do Nothing. That '
S Right. The cache line is already here. cpu is happy to find the data in the cache. When a loading miss, we will re-start the init process init init init init init. When a cpu store hit (a store hit from the current processor), Now comes to the key part of Write Once protocol. When having a write / store behavior, for UP (unique processor) system, we all understand that the cache state will go to DIRTY state and ** ** did not write data back to main memory ****. However, Write-Once protocol works like this below in order to achieve multiple processor cache coherence. The stored data will be flushed back to main memory (why? We need A Bus Transaction Over the bus !!!) And the cache state. this is exactly why this protocol is is exactly why this protocol is than one name of "write" - Write the first time write access to a write-back Cache line into main memory *****!!! SO That Other processor cache controller will be awrecped and the invalidate the corresponding cache lines, and thus the whole system will only one copy of the cache line. After the first time write once, the subsequent write access will only change the state to DIRTY state and the data will stay in cache line and will not be flushed into main memory, the same case as we see in UP write-back approach. When a SNOOP STOEE HIT (we found another CPU is trying to do a store on that cached address), then, with the write-invalidate semantics, we know, the system will THEN INVALIDATE OWN COPY TO Keep Only ONE LEGAL COPY for That Cache Line. in Other Words, The State Will Go To Invalid State from Valid State. Note That, WE DON '
THE REASON IS SIMPLE: IN THIS, WE DIDN 'THIS, INVALIDATEDOUR' Well ONLY INVALIDATE OUWN CON THIS, IF WE WANT TO Read this Particular Data, we will have to load it from main memory again. For VALID state, we have other input needed to be considered, like snoop load hit cpu store miss How cpu will react with these two actions? I will leave questions to you guys ... ...
Recently work with dual cpu kernel part For dual cpu, or we say, multi-processor, the big challenge part for a kernel is how to handle the cache coherence Conceptually, two choices -... Write Invalidate and Write Update We will talk about Write Invalidate today. Typically, there are two protocols falling into the Write Invalidate protocol, namely, the Write-Through Write Invalidate Protocol and the Write Once (Or Write-Back Write Invalidate Protocol). Note that the well-known MESI protocol is Derived from the write overce. That's why we will focus on the write overce here. -------------------------- Write: Write Once Protocol IS to offset the shortcomings of Write-Through Write-Invalidate Protocol, which will introduce extra bus traffic onto the system bus Write Once works basically as follows:.. (Assume * the hardware snoop enabled over the shared system bus * The cache is Write Back ) There Are Four States for Write Once Protocol - Valid, Reserved, Dirty and Invalid. Initial State: * W hen a LOAD MISS, the data will be loaded into cache line and the state goes to VALID state // Please note here, Write Once protocol will promise that your loaded data in memory will be the latest. Why? If at the time you tried to load a cache line, there is an modified copy in another CPU, the snoop protocol will abort the load bus transaction; flush another cpu's data into main memory and then resume the aborted transaction so that the requesting CPU will then get the updated data. ... now, let's investigate the state machine of write overce protocol. ************* VALID State: ************* WHEN A Load Hit, We do nothing. That '
S Right. The cache line is already here. cpu is happy to find the data in the cache. When a loading miss, we will re-start the init process init init init init init. When a cpu store hit (a store hit from the current processor), Now comes to the key part of Write Once protocol. When having a write / store behavior, for UP (unique processor) system, we all understand that the cache state will go to DIRTY state and ** ** did not write data back to main memory ****. However, Write-Once protocol works like this below in order to achieve multiple processor cache coherence. The stored data will be flushed back to main memory (why? We need A Bus Transaction Over the bus !!!) And the cache state. this is exactly why this protocol is is exactly why this protocol is than one name of "write" - Write the first time write access to a write-back Cache line into main memory *****!!! SO That Other processor cache controller will be awrecped and the invalidate the corresponding cache lines, and thus the whole system will only one copy of the cache line. After the first time write once, the subsequent write access will only change the state to DIRTY state and the data will stay in cache line and will not be flushed into main memory, the same case as we see in UP write-back approach. When a SNOOP STOEE HIT (we found another CPU is trying to do a store on that cached address), then, with the write-invalidate semantics, we know, the system will THEN INVALIDATE OWN COPY TO Keep Only ONE LEGAL COPY for That Cache Line. in Other Words, The State Will Go To Invalid State from Valid State. Note That, WE DON '
THE REASON IS SIMPLE: IN THIS, WE DIDN 'THIS, INVALIDATEDOUR' Well ONLY INVALIDATE OUWN CON THIS, IF WE WANT TO Read this Particular Data, we will have to load it from main memory again. For VALID state, we have other input needed to be considered, like snoop load hit cpu store miss How cpu will react with these two actions? I will leave questions to you guys ... ...---------------- Valid State ---------------- Snoop Load Hit: That Means That Current CPU Cache Controller ** * see ** a bus transaction issued by another device, for instance, another CPU, to access the data that is currentily cached When this happens, current CPU cache will do nothing but stay the VALID state The reason is simple:.. Another device / cpu will fetch the data from the memory that STILL holds the latest data CPU STORE MISS:.? How / when this situation could happen, after a cache line WAS loaded before Yeah, you are right that cache line could be ** re. placed ** by other datas (Remember the LRU algorithm of cache management; Set associative concepts / mechanisms and so such). when a CPU store miss happens, the data will be written into main memory and also have a copy in the cache (That's The "write allocate"
when handling a write miss). After that, the cache snoop protocol will move its state into Reserved from the VALID. Why? That means that a write and more importantly, the first write, has just completed! ------- ------- Reserved -------------- As I Said Before, the Reserved State Was Introduces - The First Write Will Flush Data Into Memory So That Data INTO MEMORY SO THAT other devices / CPUs are able to see / be notified a bus transaction For this state, same as VALID state, its input could vary as follows:! * CPU LOAD HIT Simply feed the data back to CPU and keep the state stay * CPU. WRITE HIT Since we are using WT (write back) for the cache lines, then we will change our state to DIRTY. in other words, we will not flush data into memory, in contrast with the WT (Write Through) approach. * CPU Load Miss System Will SIMPLY LOAD A COPY IN THE CACHE (IT IS NOT THROUGH). AND RESET The System As Valid State. * CPU Write Miss this Cache Line Go t replaced some time after it was loaded into cache. System will simply write the data back to main memory and then keep the latest copy in the cache and set the state still as Reserved. * SNOOP LOAD HIT We ** see ** another device .. is to read the cached data Here we do not have to do any cache flush / invalidating behavior What need to do is to change the cache coherent protocol state to VALID Question:. what would happen if we stay Reserved state For example? , if there is a coming cpu store hit? :-) * snoop write hit we ** See ** a write access by Another Device / CPU. WE WILL INVALIDATE OWN Cache, Which Will Then Become Stale And Move Our State To INVALID. The Reason Why We Don '
