FreeRTOS SMP

丁語婕, 鄭煦霖, 王信智

Overview of FreeRTOS

FreeRTOS is a lightweight real-time operating system designed for embedded systems, offering efficient scheduling and scalability. Its robust kernel and modular design make it ideal for responsive and reliable applications across various hardware and industries.

Symmetric Multiprocessing (SMP)

SMP enables multiple processors to share memory and execute tasks in parallel, enhancing performance for applications. In FreeRTOS, a single instance operates across cores with shared memory and the same architecture, allowing multiple tasks to run simultaneously, which challenges traditional priority-based task execution.

Purpose

This project focuses on implementing FreeRTOS SMP for the RISC-V architecture, exploring two key aspects: FreeRTOS with RISC-V, which involves adapting FreeRTOS to the RISC-V environment by modifying its port layer for hardware compatibility, and SMP with FreeRTOS, which extends the system to support multiprocessor execution by updating task scheduling, resource management, and synchronization mechanisms.

Environment Setup

GNU Toolchain for RISC-V

  1. Install xPack Project Manager(xpm)
    ​​​​$ sudo apt-get install npm
    ​​​​$ sudo npm install --global xpm
    
  2. Install xPack GNU RISC-V embedded GCC
    ​​​​$ xpm install --global @xpack-dev-tools/riscv-none-elf-gcc@14.2.0-2.1
    
  3. Configure PATH
    • Toolchain will be installed in ~/.local/xPacks/@xpack-dev-tools/riscv-none-elf-gcc/14.2.0-2.1/.content/bin
    • Add to PATH
      ​​​​​​​​ $ echo 'export PATH=$PATH:$HOME/.local/xPacks/@xpack-dev-tools/riscv-none-elf-gcc/14.2.0-2.1/.content/bin' >> ~/.bashrc
      ​​​​​​​​ $ source ~/.bashrc
      

QEMU

  1. Create a directory for QEMU
    ​​​​$ sudo mkdir -p /opt/qemu
    ​​​​$ sudo chown $USER:$USER /opt/qemu
    
  2. Clone the QEMU repo from GitHub
    ​​​​$ git clone https://github.com/qemu/qemu.git
    
  3. Build and install QEMU
    ​​​​$ cd qemu
    ​​​​$ ./configure --prefix=/opt/qemu
    ​​​​$ make -j$(nproc)
    ​​​​$ make install
    
  4. Add QEMU path to PATH
    ​​​​$ echo 'export PATH=$PATH:/opt/qemu/bin' >> ~/.bashrc
    ​​​​$ source ~/.bashrc
    

Building and Executing FreeRTOS

  1. Clone the FreeRTOS repo from Github
    ​​​​$ git config --global core.symlinks true
    ​​​​$ git clone https://github.com/FreeRTOS/FreeRTOS --recurse-submodules
    
  2. Build FreeRTOS
    ​​​​$ cd FreeRTOS/FreeRTOS/Demo/RISC-V_RV32_QEMU_VIRT_GCC
    ​​​​$ make -C build/gcc DEBUG=1
    
  3. Execute the binary using QEMU
    ​​​​ $ qemu-system-riscv32 -nographic -machine virt -net none \
    ​​​​   -chardev stdio,id=con,mux=on -serial chardev:con \
    ​​​​   -mon chardev=con,mode=readline -bios none \
    ​​​​   -smp 4 -kernel ./build/gcc/output/RTOSDemo.elf
    
    • Parameters

      Parameters Description
      -nographic Disable graphical output and redirect serial I/Os to console
      -machine virt Select emulated machine, virt is RISC-V VirtIO board
      -net none Disable all network functionality in the virtual machine
      -chardev stdio,id=con,mux=on A character device connects to the host's standard input(stdin) and standard output(stdout). id=id assigns a unique identifier to the device. mux=on enables multiplexing, allowing multiple virtual devices to share the same terminal connection
      -serial chardev:con Redirect the serial port to character device named con, enabling communication through it
      -mon chardev=con,mode=realine Set up a QEMU monitor interface connected to the character device con and enable readline mode for interactive command-line input
      -bios none Not to load a BIOS for the virtual machine
      -smp 4 Set the number of initial CPUs to 4
      -kernel ./build/gcc/output/RTOSDemo.elf Use ./build/gcc/output/RTOSDemo.elf as kernel image
    • Output

      ​​​​​​​​ FreeRTOS Demo Start: : 5032
      ​​​​​​​​ FreeRTOS Demo Start: : 10032
      ​​​​​​​​ FreeRTOS Demo Start: : 15033
      ​​​​​​​​ FreeRTOS Demo Start: : 20032
      ​​​​​​​​ FreeRTOS Demo Start: : 25032
      ​​​​​​​​ FreeRTOS Demo Start: : 30033
      ​​​​​​​​ FreeRTOS Demo Start: : 35032
      ​​​​​​​​ FreeRTOS Demo Start: : 40032
      ​​​​​​​​ FreeRTOS Demo Start: : 45032
      ​​​​​​​​ FreeRTOS Demo Start: : 50032
      

Remote Debugging with GDB

  1. Execute the binary using QEMU
    ​​​​ $ qemu-system-riscv32 -nographic -machine virt -net none \
    ​​​​   -chardev stdio,id=con,mux=on -serial chardev:con \
    ​​​​   -mon chardev=con,mode=readline -bios none \
    ​​​​   -smp 4 -kernel ./build/gcc/output/RTOSDemo.elf -s -S
    
    • -s: Starts a GDB server on default TCP port 1234 for debugging.
    • -S: Starts QEMU in a paused state, waiting for a GDB to connect and resume execution.
  2. GDB command
    ​​​​$ riscv-none-elf-gdb ./build/gcc/output/RTOSDemo.elf -ex "target remote:1234"
    
    • ./build/gcc/output/RTOSDemo.elf: Specify the ELF file to debug, containing symbols and code for debugging.
    • -ex "target remote :1234": Execute the GDB command target remote :1234 to connect to the QEMU GDB server running on TCP port 1234.

FreeRTOS with RISC-V

MTIME Settings

/* FreeRTOSConfig.h */
#define configMTIME_BASE_ADDRESS        ( CLINT_ADDR + CLINT_MTIME )
#define configMTIMECMP_BASE_ADDRESS     ( CLINT_ADDR + CLINT_MTIMECMP )
Parameters Description
configMTIME_BASE_ADDRESS set configMTIME_BASE_ADDRESS to the MTIME base address
configMTIMECMP_BASE_ADDRESS set configMTIMECMP_BASE_ADDRESS to the MTIMECMP base address

Interrupt (System) Stack Setup

/* FreeRTOSConfig.h */
#define configISR_STACK_SIZE_WORD       ( 300 )

To achieve the transition to a dedicated interrupt stack before any C functions are called from an interrupt service routine (ISR) in the FreeRTOS RISC-V port, two methods are available: using a linker script or a static allocated array.We chose the latter method for its simplicity and efficiency, declaring a 300-word stack. The stack size is defined in FreeRTOSConfig.h, ensuring consistent memory usage and avoiding complex linker script modifications.

Installing the FreeRTOS Trap Handler

/* freertos_risc_v_chip_specific_extensions.h */
#define portasmHAS_MTIME    1

freertos_risc_v_trap_handler() is the FreeRTOS trap handler and serves as the central entry point for all interrupts and exceptions. To automatically install the trap handler, set portasmHAS_MTIME to 1.

Porting to 32-bit RISC-V Implmentations

/* freertos_risc_v_chip_specific_extensions.h */
#define portasmHAS_SIFIVE_CLINT           1
#define portasmADDITIONAL_CONTEXT_SIZE    0
Parameters Description
portasmHAS_SIFIVE_CLINT target RISC-V chip includes a CLINT and trap handler to be installed automatically then set to 1
portasmADDITIONAL_CONTEXT_SIZE the number of additional registers that exist on the target chip; temporarily assume it to be 0

SMP with FreeRTOS

FreeRTOS/SMP Boot Procedure

Compilation, Assemble and Linking

This Phase is responsible for converting the source files(.c and .S) into a final executable binary, RTOSDemo.elf, that meets RISC-V architecture requirements. The process begins with preparing the necessary tools and flags for the build. Source files are compiled into object files, and dependencies are tracked to ensure only modified files are recompiled. Afterward, the object files are linked together to generate the final executable. The process is managed through a Makefile that defines the necessary steps for compilation and linking, including memory layout and stack size configurations. The result elf file is a binary ready for execution on a RISC-V system.

Flow Chart of Boot Process

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Boot Process and Implementation

Finished!
Shrink the code listing. You should only mention the critical parts.

  • start.S
    .section .data
    .global hart_ready_flag
    .align 4
    /* set hart_ready_flag 0, means primary hart does not initialize */
    hart_ready_flag:
        .word 0

    .section .bss
    .align 4
    _secondary_stack_top:
        /* set 2048 size for secondary hart stack */
        .space 2048
	
	.extern xPortStartScheduler

    /* set the entry point and initialize global pointers */
    /* check the core ID and branch non-primary cores to secondary */
    /* initialize the stack pointer */
    /* copy initialized data from _data_lma to _data */
    /* clear the BSS section (_bss to _ebss) */
    /* call the main function */
    /* enter a low-power wait loop(wfi) */
    
secondary:
    /* secondary hart waits until primary signals readiness */
1: 
    la a0, hart_ready_flag
    LOAD t0, (a0)
    beqz t0, 1b

    /* setup secondary hart stack pointer*/
    la sp, _secondary_stack_top

    /* jump to FreeRTOS secondary hart entry point */
    j xPortStartScheduler
	
1:
    wfi
    j secondary

.cfi_endproc
  • port.c
BaseType_t xPortStartScheduler( void )
{
    extern volatile uint32_t hart_ready_flag;
    
    BaseType_t xCoreID = xGetCoreID();

    if (xCoreID == PRIM_HART) {
        // volatile uint32_t *p_hart_ready_flag = &hart_ready_flag;
        // *p_hart_ready_flag = 1;
        hart_ready_flag = 1;
        
        /* ensure memory synchronization so that the secondary cores
        * can read the latest value of the hart_ready_flag */
        __asm volatile ("fence rw, rw");
    }

    /* Check alignment of the interrupt stack - which is the same as the
     * stack that was being used by main() prior to the scheduler being
     * started. */

    /* If there is a CLINT then it is ok to use the default implementation
     * in this file, otherwise vPortSetupTimerInterrupt() must be implemented to
     * configure whichever clock is to be used to generate the tick interrupt. */
    
    /* Enable mtime and external interrupts.  1<<7 for timer interrupt,
     * 1<<11 for external interrupt.  _RB_ What happens here when mtime is
     * not present as with pulpino? */
        
    xPortStartFirstTask();

    /* Should not get here as after calling xPortStartFirstTask() only tasks
     * should be executing. */
    return pdFAIL;
}

Comment section /* ----- [SMP-Specify] ----- */ represents modifications that satisfy SMP requirements.

  1. Hart Initialization and Sychronization Mechansim
    In start.S, the primary hart performs system initialization, including setting up .data and .bss sections and calling the main function. It signals its readiness to secondary harts using a shared hart_ready_flag. Secondary harts poll this flag and only proceed to their designated entry point (xPortStartScheduler) after the primary hart completes initialization. Additionally, stack memory for secondary harts is allocated and aligned to support their execution.
  2. Scheduler Initialization and Task Execution
    In xPortStartScheduler(), the primary hart finalizes its role in initializing the system by setting the hart_ready_flag and ensuring memory synchronization, allowing secondary harts to proceed. Once all harts are synchronized, the scheduler sets up the interrupt mechanism, including the timer and external interrupts, and transitions to task execution by calling xPortStartFirstTask().

GDB Experiment

  1. Set two breakpoint at the entry of primary hart and secondary harts
    ​​​​(gdb) b main
    ​​​​Breakpoint 1 at 0x80004656: file ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/main.c, line 128.
    ​​​​(gdb) b port.c:xPortStartScheduler 
    ​​​​Breakpoint 2 at 0x800045ec: file ./../../../../Source/portable/GCC/RISC-V/port.c, line 214.
    
  2. Watch the value of hart_ready_flag to verify that the primary hart initializes before starting the secondary harts
    ​​​​(gdb) watch (volatile uint32_t)hart_ready_flag
    ​​​​Hardware watchpoint 3: (volatile uint32_t)hart_ready_flag
    ​​​​(gdb) c
    ​​​​Continuing.
    
    ​Thread 1 hit Watchpoint 3: (volatile uint32_t)hart_ready_flag
    
    ​Old value = 0
    ​New value = 1
    ​xPortStartScheduler ()
    ​	at ./../../../../Source/portable/GCC/RISC-V/port.c:223
    ​223             __asm volatile ("fence rw, rw");
    
  3. Observe the behavior of the primary hart and secondary harts
    We can see that the secondary harts remain in the secondary loop until the primary hart set the hart_ready_flag to 1. Once the hart_ready_flag is set to 1, the secondary harts will enter the entry point xxPortStartScheduler in port.c.
    ​​​​(gdb) info threads
    ​​​​info threads
    ​  Id   Target Id                    Frame 
    ​* 1    Thread 1.1 (CPU#0 [running]) xPortStartScheduler ()
    ​	at ./../../../../Source/portable/GCC/RISC-V/port.c:223
    ​  2    Thread 1.2 (CPU#1 [running]) secondary () at start.S:123
    ​  3    Thread 1.3 (CPU#2 [running]) secondary () at start.S:123
    ​  4    Thread 1.4 (CPU#3 [running]) secondary () at start.S:123
    ​​​​(gdb) c
    ​Continuing.
    ​[Switching to Thread 1.4]
    
    ​Thread 4 hit Breakpoint 2, xPortStartScheduler ()
    ​	at ./../../../../Source/portable/GCC/RISC-V/port.c:214
    ​214         BaseType_t xCoreID = xGetCoreID();
    ​​​​(gdb) info threads
    ​  Id   Target Id                    Frame 
    ​* 1    Thread 1.1 (CPU#0 [running]) xPortStartScheduler ()
    ​	at ./../../../../Source/portable/GCC/RISC-V/port.c:238
    ​  2    Thread 1.2 (CPU#2 [running]) xGetCoreID ()
    ​	at ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/riscv-virt.c:35
    ​  3    Thread 1.3 (CPU#2 [running]) xGetCoreID ()
    ​	at ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/riscv-virt.c:35
    ​  4    Thread 1.4 (CPU#2 [running]) xGetCoreID ()
    ​	at ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/riscv-virt.c:35
    ​​​​(gdb) n
    ​247         vPortSetupTimerInterrupt();
    

Mention what you have found by means of GDB tracing.

Functions in Portable Layer Required for SMP Implementation

In the FreeRTOSConfig.h, set the number of cores with #define configNUMBER_OF_CORES 4 (assuming a core number greater than 2), and use the error output to modify the program to enable SMP.

./../../../../Source/include/FreeRTOS.h:190:10: error: #error Missing definition: configUSE_PASSIVE_IDLE_HOOK must be defined in FreeRTOSConfig.h as either 1 or 0. See the Configuration section of the FreeRTOS API documentation for details.
  190 |         #error Missing definition:  configUSE_PASSIVE_IDLE_HOOK must be defined in FreeRTOSConfig.h as either 1 or 0.  See the Configuration section of the FreeRTOS API documentation for details.
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:414:10: error: #error configNUMBER_OF_CORES is set to more than 1 then portGET_CORE_ID must also be defined.
  414 |         #error configNUMBER_OF_CORES is set to more than 1 then portGET_CORE_ID must also be defined.
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:424:10: error: #error configNUMBER_OF_CORES is set to more than 1 then portYIELD_CORE must also be defined.
  424 |         #error configNUMBER_OF_CORES is set to more than 1 then portYIELD_CORE must also be defined.
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:432:10: error: #error portSET_INTERRUPT_MASK is required in SMP
  432 |         #error portSET_INTERRUPT_MASK is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:440:10: error: #error portCLEAR_INTERRUPT_MASK is required in SMP
  440 |         #error portCLEAR_INTERRUPT_MASK is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:450:10: error: #error portRELEASE_TASK_LOCK is required in SMP
  450 |         #error portRELEASE_TASK_LOCK is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:460:10: error: #error portGET_TASK_LOCK is required in SMP
  460 |         #error portGET_TASK_LOCK is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:470:10: error: #error portRELEASE_ISR_LOCK is required in SMP
  470 |         #error portRELEASE_ISR_LOCK is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:480:10: error: #error portGET_ISR_LOCK is required in SMP
  480 |         #error portGET_ISR_LOCK is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:488:10: error: #error portENTER_CRITICAL_FROM_ISR is required in SMP
  488 |         #error portENTER_CRITICAL_FROM_ISR is required in SMP
      |          ^~~~~
./../../../../Source/include/FreeRTOS.h:496:10: error: #error portEXIT_CRITICAL_FROM_ISR is required in SMP
  496 |         #error portEXIT_CRITICAL_FROM_ISR is required in SMP
      |          ^~~~~

portGET_CORE_ID()

  • Return the core ID of the calling core
/* portmacro.h */
#define portGET_CORE_ID()      \
    ({                         \
        uint32_t coreID;       \
        __asm volatile (       \
            "csrr %0, mhartid" \
            : "=r" (coreID)    \
        );                     \
        coreID;                \
    })

portYIELD_CORE()

  • Interrupt the core with ID
/* portmacro.h */
#define portYIELD_CORE(coreID)                                                            \
    {                                                                                     \
        volatile uint32_t* msip = (uint32_t*)(configMSIP_BASE_ADDRESS + (4U * (coreID))); \
        /* write the value 1 to the MSIP register triggers a software interrupt on the specified core */ \
        *msip = 1U;                                                                       \
    }

portSET_INTERRUPT_MASK() & portCLEAR_INTERRUPT_MASK()

  • Interrupt mask should be returned and cleared with macros
/* portmacro.h */
#define portSET_INTERRUPT_MASK()                                  \
    ( {                                                           \
        uint32_t ulState;                                         \
        __asm volatile ( "csrr %0, mstatus" : "=r" ( ulState ) ); /* Read current mstatus */ \
        __asm volatile ( "csrc mstatus, %0" :: "r" (0x8) : "memory" ); /* Clear MIE bit to disable interrupts */ \
        ulState;                                                  \
    } )

#define portCLEAR_INTERRUPT_MASK( ulState )    \
    __asm volatile ( "csrw mstatus, %0" :: "r" ( ulState ) : "memory" )

portRELEASE_TASK_LOCK & portGET_TASK_LOCK & portRELEASE_ISR_LOCK & portGET_ISR_LOCK

  • Acquire and release the task or ISR lock
/* portmacro.h */
static inline void vPortRecursiveLock( uint32_t ulLockNum,
                                       spin_lock_t * pxSpinLock,
                                       BaseType_t uxAcquire )
{
    configASSERT( ulLockNum < NUM_SPINLOCKS );
    uint32_t ulCoreID = portGET_CORE_ID();
    /* Pointer to metadata for the lock. */
    RecursiveLock_t *pxLockMetadata = &lockMetadata[ulLockNum];

    if (uxAcquire)
    {
        /* Acquire the lock. */
        while (__builtin_expect(!__sync_bool_compare_and_swap(pxSpinLock, 1, 0), 0))
        {
            /* Spin until the lock is acquired. */
        }

        __mem_fence_acquire(); /* Ensure memory consistency. */

        /* Check if the lock is already owned by the current core. */
        if (pxLockMetadata->ownerCoreID == ulCoreID)
        {
            pxLockMetadata->recursionCount++;
            return;
        }

        /* Lock is not owned by this core, take ownership. */
        pxLockMetadata->ownerCoreID = ulCoreID;
        pxLockMetadata->recursionCount = 1;
    }
    else
    {
        /* Release the lock. */
        configASSERT(pxLockMetadata->ownerCoreID == ulCoreID); /* Ensure this core owns the lock. */
        configASSERT(pxLockMetadata->recursionCount > 0);      /* Ensure recursion count is valid. */

        pxLockMetadata->recursionCount--;

        /* Fully release the lock if recursion count reaches zero. */
        if (pxLockMetadata->recursionCount == 0)
        {
            pxLockMetadata->ownerCoreID = -1;
            __mem_fence_release(); /* Ensure memory consistency. */
            *pxSpinLock = 1;       /* Release the spinlock. */
        }
    }
}

#if ( configNUMBER_OF_CORES == 1 )
    #define portGET_ISR_LOCK()
    #define portRELEASE_ISR_LOCK()
    #define portGET_TASK_LOCK()
    #define portRELEASE_TASK_LOCK()
#else
    #define portGET_ISR_LOCK()         vPortRecursiveLock( 0, spin_lock_instance( 0 ), pdTRUE )
    #define portRELEASE_ISR_LOCK()     vPortRecursiveLock( 0, spin_lock_instance( 0 ), pdFALSE )
    #define portGET_TASK_LOCK()        vPortRecursiveLock( 1, spin_lock_instance( 1 ), pdTRUE )
    #define portRELEASE_TASK_LOCK()    vPortRecursiveLock( 1, spin_lock_instance( 1 ), pdFALSE )
#endif

portENTER_CRITICAL_FROM_ISR & portEXIT_CRITICAL_FROM_ISR

  • Enter/exit critical section from ISR
/* portmacro.h */
extern UBaseType_t vTaskEnterCriticalFromISR( void );

#define portENTER_CRITICAL_FROM_ISR()      vTaskEnterCriticalFromISR()
#define portEXIT_CRITICAL_FROM_ISR( x )    vTaskExitCriticalFromISR( x )

CLINT/ACLINT and FreeRTOS portable

CLINT ( Core Local Interrupt )

Introduction

CLINT is a simple core local interrupt controller designed for embedded systems to provide timer interrupts, software interrupts, and local interrupt management. It is used for handling asynchronous CPU events such as timers and software-triggered interrupts.

Features

  • Timer Interrupts
    • Managed via mtime and mtimecmp registers.
    • Triggers periodic operations when mtime reaches the value in mtimecmp.
  • Software Interrupts
    • Managed via msip, enabling inter-core communication by notifying a CPU core to execute specific operations.
  • Configuration Registers
    • mtime: Provides a monotonically increasing time value for precise timing.
    • mtimecmp: Configures the threshold for triggering timer interrupts.
    • msip: Used to trigger software interrupts for inter-core signaling.

Limitations

  • Fixed interrupt priorities, with machine mode having the highest priority.
  • Does not support dynamic interrupt prioritization or advanced features.
  • Primarily supports machine-level interrupts.
  • Does not handle external interrupts (these are managed by PLIC).

Operation Modes

  1. Direct Mode: All interrupts and exceptions use the same base address for their handler. The software determines the type of interrupt.
  2. Vectored Mode: Each interrupt type has a unique entry point, enabling faster processing by jumping directly to the corresponding handler.

ACLINT ( Advanced Core Local Interrupt )

Introduction

ACLINT is an enhanced version of CLINT, designed for multi-core systems and modular architectures. It provides machine-level and supervisor-level timer and software interrupt functionalities. Its modular design allows selective integration of features like inter-processor interrupts (IPI) and time synchronization.

Features

  • Backward Compatibility
    ACLINT retains compatibility with CLINT, using the same register mapping for mtime, mtimecmp, and msip.
  • Additional Functionality
    • Supervisor-Level Software Interrupts (SSWI): Enables IPIs in supervisor mode.
    • Modular design supports multiple MTIMER and MSWI devices.
    • Synchronization mechanisms for multiple mtime registers.
  • Multi-Cluster Support
    ACLINT can handle multiple clusters, each with independent timers and IPIs, making it suitable for distributed systems.

ACLINT Register Map

  • MTIMER Device: Manages mtime and mtimecmp for timekeeping and timer interrupts.
  • MSWI Device: Implements msip for managing software interrupts on each core.

Core Affinity Settings

In FreeRTOSConfig.h, adding #define configUSE_CORE_AFFINITY = 1 to enable the function to set which core the task shall run on.

vTaskCoreAffinitySet

  • Use UBaseType_t uxCoreAffinityMask to record which cores a task can run on using a bitwise value.
  • For example, 0101 indicates that the task can run on core 0 and core 2.

vTaskCoreAffinityGet

  • Retrieve the core affinity mask of a specific task.

Create Task

Type Create Task Create Affinity Set
Dynamic xTaskCreate xTaskCreateAffinitySet
Static xTaskCreateStatic xTaskCreateStaticAffinitySet
Restricted xTaskCreateRestricted xTaskCreateRestrictedAffinitySet
Restricted Static xTaskCreateRestrictedStatic xTaskCreateRestrictedStaticAffinitySet
  • Create Task
    • Set TCB of the task uxCoreAffinityMask to allow the task to run on the default core(s) defined by configTASK_DEFAULT_CORE_AFFINITY (usually all cores).
  • Create Affinity Set
    • Set/modify the task TCB's uxCoreAffinityMask to run on the specified core.
    • If the task is currently running and it isn't on a core where it can run, a core switch is required, and the task will yield to the specified core with prvYieldCore(xCoreID);.

Task Scheduler

  • vTaskStartScheduler
    • Affinity here confirms that UBaseType_t can represent all cores.
  • prvYieldForTask
    • Check uxCoreAffinityMask of the task to ensure it can only run on the allowed core(s) (xCore).
  • prvSelectHighestPriorityTask
    • Record pxPreviousTCB, which points to the TCB of the task that was previously executed on that core, used to preserve the state before the scheduling occurs.
    • Check uxCoreAffinityMask of the task to ensure it can only run on the allowed core(s) (xCore).

Future Works

  • Enable task execution on multiple cores by resolving scheduling and placement issues.
  • Improve inter-core communication through better synchronization and resource management.
  • Optimize performance by enhancing workload distribution and reducing overhead.
  • Conduct comprehensive testing and validation to ensure system stability and reliability.

Reference

  1. xPack GNU RISC-V Embedded GCC
  2. Emulating generic RISC-V 32bit machine on QEMU
  3. Using FreeRTOS on RISC-V Microcontrollers
  4. SiFive Interrupt Cookbook
  5. SMP portable function list
  6. Second core Wake up sequence in SMP
  7. The startup process for each core in FreeRTOS SMP