# [Study] Highlight in RC-NVM source code
###### tags: `research-GraphRC`
## Introduction
[RC-NVM](https://ieeexplore.ieee.org/document/8327034) is a non-volatile memory based simulator based on [NVMain](https://ieeexplore.ieee.org/document/6296505). We acquire the source code of RC-NVM and this chapter will focus on the modification made by RC-NVM team to enable the dual-address mode (row and column) on NVMain simulator.

## RC-NVM based RRAM config
In the NVMain simulator, there's a config file specifying the configuration of the memory architecture and the electrical performance of non-volative memory cell in case the simulation target is based on NVM. Similarily, RC-NVM follows the general specification of NVMain simulator; however, some modifications have to be made in this file to enable the row-column access in RC-NVM. This file is important while decoding the underlying architecture of RC-NVM. A sample RC-NVM configuration file based on RRAM can be fonud in the following section. Notice that only architecural info is contained, electrical performance of NVM cell is tedious so excluded here.

;********************************************************************************
; General memory system configuration

; Number of banks per rank 
BANKS 8

; Number of ranks per channel
RANKS 4

; Number of channels in the system
CHANNELS 2

; Number of rows in one bank
ROWS 8192 ; xx WL / macro * x vertical macros

; Number of VISIBLE columns in one LOGIC bank 
;COLS 128 ; xx bitlines/macro * xx horiz. macros / (8 device width * 8 burst cycles)
         ; should change to 1024
         ; Add another parameter, such as COLSInCacheline
COLS 1024
; COLSInCacheline 128

; whether enable sub-array level parallelism (SALP)
; options: 
;   No SALP: MATHeight = ROWS
;   SALP: number of subarrays = ROWS / MATHeight
; Assume 8 subarrays
MATHeight 1024

; RBSize - 4:1 mux (?)
;RBSize 32
RBSize 1024
CBSize 1024

; No refresh needed in RRAM
UseRefresh false

### Observation
This configuration file provides the architecural info:
| BANKS | RANKS | CHANNELS | ROWS | COLS | MATHeight | RBSize | CBSize |
| ----- | ----- | -------- | ---- | ---- | --------- | ------ | ------ |
| 8 | 4 | 2 | 8192 | 1024 | 1024 | 1024 | 1024 |
- BANKS, RANKS, CHANNELS: These are the standard DRAM-style architectural parameter
- ROWS: The number of rows in
- COLS: The number of columns in
- MATHeight:
- RBSize: Row buffer size
- CBSize: Column buffer size
## Source code
### Add new instruction "CR" & "CW"
- 
- 

### New ColumnAddressMappingScheme
- 

### New columnDecoder
- 
- 

### Add Activate/Read/Write operation in Subarray level
- Decode physical address into row & col address
- 
- Activate row/column
- 
- Modify scheduling policy
- 

### Add Activate/Read/Write operation in Bank level
- Translate column number from physical address
- 
- Important insight: A row/column CANNOT be activate at the same time
- 

### Add Column access in Rank level
- Scheduling policy
- 
### Memory Controller
- FindStarvedRequest

bool MemoryController::FindStarvedRequest( std::list<NVMainRequest *>& transactionQueue, 
                                           NVMainRequest **starvedRequest, 
                                           SchedulingPredicate& pred )
        ...
        /* By design, mux level can only be a subset of the selected columns. */
        ncounter_t muxLevel = static_cast<ncounter_t>(col / p->RBSize);
        ncounter_t col_muxLevel = static_cast<ncounter_t>(row / p->CBSize);
        ...

- FindRowBufferHit

bool MemoryController::FindRowBufferHit( std::list<NVMainRequest *>& transactionQueue, 
                                         NVMainRequest **hitRequest, SchedulingPredicate& pred )
        ...
        /* By design, mux level can only be a subset of the selected columns. */
        ncounter_t muxLevel = static_cast<ncounter_t>(col / p->RBSize);
        ncounter_t col_muxLevel = static_cast<ncounter_t>(row / p->CBSize);
        ...

- IssueMemoryCommands

bool MemoryController::IssueMemoryCommands( NVMainRequest *req )
        ...
        ncounter_t muxLevel = static_cast<ncounter_t>(col / p->RBSize);
        ncounter_t col_muxLevel = static_cast<ncounter_t>(row / p->CBSize);
        ...

:::spoiler FindStarvedRequest(. . . . ) complete function

bool MemoryController::FindStarvedRequest( std::list<NVMainRequest *>& transactionQueue, 
                                           NVMainRequest **starvedRequest, 
                                           SchedulingPredicate& pred )
{
    bool rv = false;
    std::list<NVMainRequest *>::iterator it;

    *starvedRequest = NULL;

    for( it = transactionQueue.begin(); it != transactionQueue.end(); it++ )
    {
        ncounter_t rank, bank, row, subarray, col;
        ncounter_t queueId = GetCommandQueueId( (*it)->address );

        if( !commandQueues[queueId].empty() ) continue;

        (*it)->address.GetTranslatedAddress( &row, &col, &bank, &rank, NULL, &subarray );
        
        /* By design, mux level can only be a subset of the selected columns. */
        ncounter_t muxLevel = static_cast<ncounter_t>(col / p->RBSize); /*!!!*/
        ncounter_t col_muxLevel = static_cast<ncounter_t>(row / p->CBSize); /*!!!*/

        if( !(*it)->address.IsColumnAddress()
            && activateQueued[rank][bank]
            && ( !activeSubArray[rank][bank][subarray]          /* The subarray is inactive */
                || effectiveRow[rank][bank][subarray] != row    /* Row buffer miss */
                || effectiveMuxedRow[rank][bank][subarray] != muxLevel )  /* Subset of row buffer is not at the sense amps */
            && !bankNeedRefresh[rank][bank]                     /* The bank is not waiting for a refresh */
            && !refreshQueued[rank][bank]                       /* Don't interrupt refreshes queued on bank group head. */
            && starvationCounter[rank][bank][subarray] 
                >= starvationThreshold                          /* This subarray has reached starvation threshold */
            && (*it)->arrivalCycle != GetEventQueue()->GetCurrentCycle()
            && commandQueues[queueId].empty()                   /* The request queue is empty */
            && pred( (*it) ) )                                  /* User-defined predicate is true */
        {
            *starvedRequest = (*it);
            transactionQueue.erase( it );

            /* Different row buffer management policy has different behavior */ 

            /* 
             * if Relaxed Close-Page row buffer management policy is applied,
             * we check whether there is another request has row buffer hit.
             * if not, this request is the last request and we can close the
             * row.
             */
            if(  IsLastRequest( transactionQueue, (*starvedRequest) ) )
                (*starvedRequest)->flags |= NVMainRequest::FLAG_LAST_REQUEST;

            rv = true;
            break;
        }
        else if( (*it)->address.IsColumnAddress()
            && activateQueued[rank][bank]
            && ( !activeSubArray[rank][bank][subarray]          /* The subarray is inactive */
                || effectiveCol[rank][bank][subarray] != col    /* Column buffer miss */
                || effectiveMuxedCol[rank][bank][subarray] != col_muxLevel )  /* Subset of column buffer is not at the sense amps */
            && !bankNeedRefresh[rank][bank]                     /* The bank is not waiting for a refresh */
            && !refreshQueued[rank][bank]                       /* Don't interrupt refreshes queued on bank group head. */
            && starvationCounter[rank][bank][subarray]
                >= starvationThreshold                          /* This subarray has reached starvation threshold */
            && (*it)->arrivalCycle != GetEventQueue()->GetCurrentCycle()
            && commandQueues[queueId].empty()                   /* The request queue is empty */
            && pred( (*it) ) )                                  /* User-defined predicate is true */
        {
            *starvedRequest = (*it);
            transactionQueue.erase( it );

            if(  IsLastRequest( transactionQueue, (*starvedRequest) ) )
                (*starvedRequest)->flags |= NVMainRequest::FLAG_LAST_REQUEST;

            rv = true;
            break;
        }

    }

    return rv;
}

:::
## Output file
### RRAM.stats
:::spoiler

i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.subArrayEnergy 243906nJ
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.activeEnergy 80.7128nJ
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.burstEnergy 5337.89nJ
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.writeEnergy 238487nJ
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.refreshEnergy 0nJ
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.cancelledWrites 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.cancelledWriteTime 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.pausedWrites 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.averagePausesPerRequest 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.measuredPauses 3165
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.averagePausedRequestProgress 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.measuredProgresses 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.reads 3379
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.writes 3165
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.activates 994
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.precharges 993
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.refreshes 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.col_reads 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.col_writes 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.col_activates 0
i0.defaultMemory.channel0.FCFS.channel0.rank0.bank0.subarray0.worstCaseEndurance 18446744073709551615

:::
### rram.trace
- Just memory trace