丁語婕, 鄭煦霖, 王信智
FreeRTOS is a lightweight real-time operating system designed for embedded systems, offering efficient scheduling and scalability. Its robust kernel and modular design make it ideal for responsive and reliable applications across various hardware and industries.
SMP enables multiple processors to share memory and execute tasks in parallel, enhancing performance for applications. In FreeRTOS, a single instance operates across cores with shared memory and the same architecture, allowing multiple tasks to run simultaneously, which challenges traditional priority-based task execution.
This project focuses on implementing FreeRTOS SMP for the RISC-V architecture, exploring two key aspects: FreeRTOS with RISC-V, which involves adapting FreeRTOS to the RISC-V environment by modifying its port layer for hardware compatibility, and SMP with FreeRTOS, which extends the system to support multiprocessor execution by updating task scheduling, resource management, and synchronization mechanisms.
$ sudo apt-get install npm
$ sudo npm install --global xpm
$ xpm install --global @xpack-dev-tools/riscv-none-elf-gcc@14.2.0-2.1
~/.local/xPacks/@xpack-dev-tools/riscv-none-elf-gcc/14.2.0-2.1/.content/bin
$ echo 'export PATH=$PATH:$HOME/.local/xPacks/@xpack-dev-tools/riscv-none-elf-gcc/14.2.0-2.1/.content/bin' >> ~/.bashrc
$ source ~/.bashrc
$ sudo mkdir -p /opt/qemu
$ sudo chown $USER:$USER /opt/qemu
$ git clone https://github.com/qemu/qemu.git
$ cd qemu
$ ./configure --prefix=/opt/qemu
$ make -j$(nproc)
$ make install
$ echo 'export PATH=$PATH:/opt/qemu/bin' >> ~/.bashrc
$ source ~/.bashrc
$ git config --global core.symlinks true
$ git clone https://github.com/FreeRTOS/FreeRTOS --recurse-submodules
$ cd FreeRTOS/FreeRTOS/Demo/RISC-V_RV32_QEMU_VIRT_GCC
$ make -C build/gcc DEBUG=1
$ qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -bios none \
-smp 4 -kernel ./build/gcc/output/RTOSDemo.elf
Parameters
Parameters | Description |
---|---|
-nographic |
Disable graphical output and redirect serial I/Os to console |
-machine virt |
Select emulated machine, virt is RISC-V VirtIO board |
-net none |
Disable all network functionality in the virtual machine |
-chardev stdio,id=con,mux=on |
A character device connects to the host's standard input(stdin) and standard output(stdout). id=id assigns a unique identifier to the device. mux=on enables multiplexing, allowing multiple virtual devices to share the same terminal connection |
-serial chardev:con |
Redirect the serial port to character device named con , enabling communication through it |
-mon chardev=con,mode=realine |
Set up a QEMU monitor interface connected to the character device con and enable readline mode for interactive command-line input |
-bios none |
Not to load a BIOS for the virtual machine |
-smp 4 |
Set the number of initial CPUs to 4 |
-kernel ./build/gcc/output/RTOSDemo.elf |
Use ./build/gcc/output/RTOSDemo.elf as kernel image |
Output
FreeRTOS Demo Start: : 5032
FreeRTOS Demo Start: : 10032
FreeRTOS Demo Start: : 15033
FreeRTOS Demo Start: : 20032
FreeRTOS Demo Start: : 25032
FreeRTOS Demo Start: : 30033
FreeRTOS Demo Start: : 35032
FreeRTOS Demo Start: : 40032
FreeRTOS Demo Start: : 45032
FreeRTOS Demo Start: : 50032
$ qemu-system-riscv32 -nographic -machine virt -net none \
-chardev stdio,id=con,mux=on -serial chardev:con \
-mon chardev=con,mode=readline -bios none \
-smp 4 -kernel ./build/gcc/output/RTOSDemo.elf -s -S
-s
: Starts a GDB server on default TCP port 1234 for debugging.-S
: Starts QEMU in a paused state, waiting for a GDB to connect and resume execution.$ riscv-none-elf-gdb ./build/gcc/output/RTOSDemo.elf -ex "target remote:1234"
./build/gcc/output/RTOSDemo.elf
: Specify the ELF file to debug, containing symbols and code for debugging.-ex "target remote :1234"
: Execute the GDB command target remote :1234 to connect to the QEMU GDB server running on TCP port 1234./* FreeRTOSConfig.h */
#define configMTIME_BASE_ADDRESS ( CLINT_ADDR + CLINT_MTIME )
#define configMTIMECMP_BASE_ADDRESS ( CLINT_ADDR + CLINT_MTIMECMP )
Parameters | Description |
---|---|
configMTIME_BASE_ADDRESS |
set configMTIME_BASE_ADDRESS to the MTIME base address |
configMTIMECMP_BASE_ADDRESS |
set configMTIMECMP_BASE_ADDRESS to the MTIMECMP base address |
/* FreeRTOSConfig.h */
#define configISR_STACK_SIZE_WORD ( 300 )
To achieve the transition to a dedicated interrupt stack before any C functions are called from an interrupt service routine (ISR) in the FreeRTOS RISC-V port, two methods are available: using a linker script or a static allocated array.We chose the latter method for its simplicity and efficiency, declaring a 300-word stack. The stack size is defined in FreeRTOSConfig.h
, ensuring consistent memory usage and avoiding complex linker script modifications.
/* freertos_risc_v_chip_specific_extensions.h */
#define portasmHAS_MTIME 1
freertos_risc_v_trap_handler()
is the FreeRTOS trap handler and serves as the central entry point for all interrupts and exceptions. To automatically install the trap handler, set portasmHAS_MTIME
to 1.
/* freertos_risc_v_chip_specific_extensions.h */
#define portasmHAS_SIFIVE_CLINT 1
#define portasmADDITIONAL_CONTEXT_SIZE 0
Parameters | Description |
---|---|
portasmHAS_SIFIVE_CLINT |
target RISC-V chip includes a CLINT and trap handler to be installed automatically then set to 1 |
portasmADDITIONAL_CONTEXT_SIZE |
the number of additional registers that exist on the target chip; temporarily assume it to be 0 |
This Phase is responsible for converting the source files(.c and .S) into a final executable binary, RTOSDemo.elf, that meets RISC-V architecture requirements. The process begins with preparing the necessary tools and flags for the build. Source files are compiled into object files, and dependencies are tracked to ensure only modified files are recompiled. Afterward, the object files are linked together to generate the final executable. The process is managed through a Makefile that defines the necessary steps for compilation and linking, including memory layout and stack size configurations. The result elf file is a binary ready for execution on a RISC-V system.
Finished!
Shrink the code listing. You should only mention the critical parts.
start.S
.section .data
.global hart_ready_flag
.align 4
/* set hart_ready_flag 0, means primary hart does not initialize */
hart_ready_flag:
.word 0
.section .bss
.align 4
_secondary_stack_top:
/* set 2048 size for secondary hart stack */
.space 2048
.extern xPortStartScheduler
/* set the entry point and initialize global pointers */
/* check the core ID and branch non-primary cores to secondary */
/* initialize the stack pointer */
/* copy initialized data from _data_lma to _data */
/* clear the BSS section (_bss to _ebss) */
/* call the main function */
/* enter a low-power wait loop(wfi) */
secondary:
/* secondary hart waits until primary signals readiness */
1:
la a0, hart_ready_flag
LOAD t0, (a0)
beqz t0, 1b
/* setup secondary hart stack pointer*/
la sp, _secondary_stack_top
/* jump to FreeRTOS secondary hart entry point */
j xPortStartScheduler
1:
wfi
j secondary
.cfi_endproc
port.c
BaseType_t xPortStartScheduler( void )
{
extern volatile uint32_t hart_ready_flag;
BaseType_t xCoreID = xGetCoreID();
if (xCoreID == PRIM_HART) {
// volatile uint32_t *p_hart_ready_flag = &hart_ready_flag;
// *p_hart_ready_flag = 1;
hart_ready_flag = 1;
/* ensure memory synchronization so that the secondary cores
* can read the latest value of the hart_ready_flag */
__asm volatile ("fence rw, rw");
}
/* Check alignment of the interrupt stack - which is the same as the
* stack that was being used by main() prior to the scheduler being
* started. */
/* If there is a CLINT then it is ok to use the default implementation
* in this file, otherwise vPortSetupTimerInterrupt() must be implemented to
* configure whichever clock is to be used to generate the tick interrupt. */
/* Enable mtime and external interrupts. 1<<7 for timer interrupt,
* 1<<11 for external interrupt. _RB_ What happens here when mtime is
* not present as with pulpino? */
xPortStartFirstTask();
/* Should not get here as after calling xPortStartFirstTask() only tasks
* should be executing. */
return pdFAIL;
}
Comment section /* ----- [SMP-Specify] ----- */
represents modifications that satisfy SMP requirements.
start.S
, the primary hart performs system initialization, including setting up .data
and .bss
sections and calling the main function. It signals its readiness to secondary harts using a shared hart_ready_flag
. Secondary harts poll this flag and only proceed to their designated entry point (xPortStartScheduler
) after the primary hart completes initialization. Additionally, stack memory for secondary harts is allocated and aligned to support their execution.xPortStartScheduler()
, the primary hart finalizes its role in initializing the system by setting the hart_ready_flag
and ensuring memory synchronization, allowing secondary harts to proceed. Once all harts are synchronized, the scheduler sets up the interrupt mechanism, including the timer and external interrupts, and transitions to task execution by calling xPortStartFirstTask()
.(gdb) b main
Breakpoint 1 at 0x80004656: file ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/main.c, line 128.
(gdb) b port.c:xPortStartScheduler
Breakpoint 2 at 0x800045ec: file ./../../../../Source/portable/GCC/RISC-V/port.c, line 214.
hart_ready_flag
to verify that the primary hart initializes before starting the secondary harts
(gdb) watch (volatile uint32_t)hart_ready_flag
Hardware watchpoint 3: (volatile uint32_t)hart_ready_flag
(gdb) c
Continuing.
Thread 1 hit Watchpoint 3: (volatile uint32_t)hart_ready_flag
Old value = 0
New value = 1
xPortStartScheduler ()
at ./../../../../Source/portable/GCC/RISC-V/port.c:223
223 __asm volatile ("fence rw, rw");
hart_ready_flag
to 1. Once the hart_ready_flag
is set to 1, the secondary harts will enter the entry point xxPortStartScheduler
in port.c
.
(gdb) info threads
info threads
Id Target Id Frame
* 1 Thread 1.1 (CPU#0 [running]) xPortStartScheduler ()
at ./../../../../Source/portable/GCC/RISC-V/port.c:223
2 Thread 1.2 (CPU#1 [running]) secondary () at start.S:123
3 Thread 1.3 (CPU#2 [running]) secondary () at start.S:123
4 Thread 1.4 (CPU#3 [running]) secondary () at start.S:123
(gdb) c
Continuing.
[Switching to Thread 1.4]
Thread 4 hit Breakpoint 2, xPortStartScheduler ()
at ./../../../../Source/portable/GCC/RISC-V/port.c:214
214 BaseType_t xCoreID = xGetCoreID();
(gdb) info threads
Id Target Id Frame
* 1 Thread 1.1 (CPU#0 [running]) xPortStartScheduler ()
at ./../../../../Source/portable/GCC/RISC-V/port.c:238
2 Thread 1.2 (CPU#2 [running]) xGetCoreID ()
at ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/riscv-virt.c:35
3 Thread 1.3 (CPU#2 [running]) xGetCoreID ()
at ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/riscv-virt.c:35
4 Thread 1.4 (CPU#2 [running]) xGetCoreID ()
at ./../../../../Demo/RISC-V_RV32_QEMU_VIRT_GCC/riscv-virt.c:35
(gdb) n
247 vPortSetupTimerInterrupt();
Mention what you have found by means of GDB tracing.
In the FreeRTOSConfig.h
, set the number of cores with #define configNUMBER_OF_CORES 4
(assuming a core number greater than 2), and use the error output to modify the program to enable SMP.
./../../../../Source/include/FreeRTOS.h:190:10: error: #error Missing definition: configUSE_PASSIVE_IDLE_HOOK must be defined in FreeRTOSConfig.h as either 1 or 0. See the Configuration section of the FreeRTOS API documentation for details.
190 | #error Missing definition: configUSE_PASSIVE_IDLE_HOOK must be defined in FreeRTOSConfig.h as either 1 or 0. See the Configuration section of the FreeRTOS API documentation for details.
| ^~~~~
./../../../../Source/include/FreeRTOS.h:414:10: error: #error configNUMBER_OF_CORES is set to more than 1 then portGET_CORE_ID must also be defined.
414 | #error configNUMBER_OF_CORES is set to more than 1 then portGET_CORE_ID must also be defined.
| ^~~~~
./../../../../Source/include/FreeRTOS.h:424:10: error: #error configNUMBER_OF_CORES is set to more than 1 then portYIELD_CORE must also be defined.
424 | #error configNUMBER_OF_CORES is set to more than 1 then portYIELD_CORE must also be defined.
| ^~~~~
./../../../../Source/include/FreeRTOS.h:432:10: error: #error portSET_INTERRUPT_MASK is required in SMP
432 | #error portSET_INTERRUPT_MASK is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:440:10: error: #error portCLEAR_INTERRUPT_MASK is required in SMP
440 | #error portCLEAR_INTERRUPT_MASK is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:450:10: error: #error portRELEASE_TASK_LOCK is required in SMP
450 | #error portRELEASE_TASK_LOCK is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:460:10: error: #error portGET_TASK_LOCK is required in SMP
460 | #error portGET_TASK_LOCK is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:470:10: error: #error portRELEASE_ISR_LOCK is required in SMP
470 | #error portRELEASE_ISR_LOCK is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:480:10: error: #error portGET_ISR_LOCK is required in SMP
480 | #error portGET_ISR_LOCK is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:488:10: error: #error portENTER_CRITICAL_FROM_ISR is required in SMP
488 | #error portENTER_CRITICAL_FROM_ISR is required in SMP
| ^~~~~
./../../../../Source/include/FreeRTOS.h:496:10: error: #error portEXIT_CRITICAL_FROM_ISR is required in SMP
496 | #error portEXIT_CRITICAL_FROM_ISR is required in SMP
| ^~~~~
portGET_CORE_ID()
/* portmacro.h */
#define portGET_CORE_ID() \
({ \
uint32_t coreID; \
__asm volatile ( \
"csrr %0, mhartid" \
: "=r" (coreID) \
); \
coreID; \
})
portYIELD_CORE()
/* portmacro.h */
#define portYIELD_CORE(coreID) \
{ \
volatile uint32_t* msip = (uint32_t*)(configMSIP_BASE_ADDRESS + (4U * (coreID))); \
/* write the value 1 to the MSIP register triggers a software interrupt on the specified core */ \
*msip = 1U; \
}
portSET_INTERRUPT_MASK()
& portCLEAR_INTERRUPT_MASK()
/* portmacro.h */
#define portSET_INTERRUPT_MASK() \
( { \
uint32_t ulState; \
__asm volatile ( "csrr %0, mstatus" : "=r" ( ulState ) ); /* Read current mstatus */ \
__asm volatile ( "csrc mstatus, %0" :: "r" (0x8) : "memory" ); /* Clear MIE bit to disable interrupts */ \
ulState; \
} )
#define portCLEAR_INTERRUPT_MASK( ulState ) \
__asm volatile ( "csrw mstatus, %0" :: "r" ( ulState ) : "memory" )
portRELEASE_TASK_LOCK
& portGET_TASK_LOCK
& portRELEASE_ISR_LOCK
& portGET_ISR_LOCK
/* portmacro.h */
static inline void vPortRecursiveLock( uint32_t ulLockNum,
spin_lock_t * pxSpinLock,
BaseType_t uxAcquire )
{
configASSERT( ulLockNum < NUM_SPINLOCKS );
uint32_t ulCoreID = portGET_CORE_ID();
/* Pointer to metadata for the lock. */
RecursiveLock_t *pxLockMetadata = &lockMetadata[ulLockNum];
if (uxAcquire)
{
/* Acquire the lock. */
while (__builtin_expect(!__sync_bool_compare_and_swap(pxSpinLock, 1, 0), 0))
{
/* Spin until the lock is acquired. */
}
__mem_fence_acquire(); /* Ensure memory consistency. */
/* Check if the lock is already owned by the current core. */
if (pxLockMetadata->ownerCoreID == ulCoreID)
{
pxLockMetadata->recursionCount++;
return;
}
/* Lock is not owned by this core, take ownership. */
pxLockMetadata->ownerCoreID = ulCoreID;
pxLockMetadata->recursionCount = 1;
}
else
{
/* Release the lock. */
configASSERT(pxLockMetadata->ownerCoreID == ulCoreID); /* Ensure this core owns the lock. */
configASSERT(pxLockMetadata->recursionCount > 0); /* Ensure recursion count is valid. */
pxLockMetadata->recursionCount--;
/* Fully release the lock if recursion count reaches zero. */
if (pxLockMetadata->recursionCount == 0)
{
pxLockMetadata->ownerCoreID = -1;
__mem_fence_release(); /* Ensure memory consistency. */
*pxSpinLock = 1; /* Release the spinlock. */
}
}
}
#if ( configNUMBER_OF_CORES == 1 )
#define portGET_ISR_LOCK()
#define portRELEASE_ISR_LOCK()
#define portGET_TASK_LOCK()
#define portRELEASE_TASK_LOCK()
#else
#define portGET_ISR_LOCK() vPortRecursiveLock( 0, spin_lock_instance( 0 ), pdTRUE )
#define portRELEASE_ISR_LOCK() vPortRecursiveLock( 0, spin_lock_instance( 0 ), pdFALSE )
#define portGET_TASK_LOCK() vPortRecursiveLock( 1, spin_lock_instance( 1 ), pdTRUE )
#define portRELEASE_TASK_LOCK() vPortRecursiveLock( 1, spin_lock_instance( 1 ), pdFALSE )
#endif
portENTER_CRITICAL_FROM_ISR
& portEXIT_CRITICAL_FROM_ISR
/* portmacro.h */
extern UBaseType_t vTaskEnterCriticalFromISR( void );
#define portENTER_CRITICAL_FROM_ISR() vTaskEnterCriticalFromISR()
#define portEXIT_CRITICAL_FROM_ISR( x ) vTaskExitCriticalFromISR( x )
CLINT is a simple core local interrupt controller designed for embedded systems to provide timer interrupts, software interrupts, and local interrupt management. It is used for handling asynchronous CPU events such as timers and software-triggered interrupts.
mtime
and mtimecmp
registers.mtime
reaches the value in mtimecmp
.msip
, enabling inter-core communication by notifying a CPU core to execute specific operations.mtime
: Provides a monotonically increasing time value for precise timing.mtimecmp
: Configures the threshold for triggering timer interrupts.msip
: Used to trigger software interrupts for inter-core signaling.ACLINT is an enhanced version of CLINT, designed for multi-core systems and modular architectures. It provides machine-level and supervisor-level timer and software interrupt functionalities. Its modular design allows selective integration of features like inter-processor interrupts (IPI) and time synchronization.
mtime
, mtimecmp
, and msip
.mtime
registers.In
FreeRTOSConfig.h
, adding#define configUSE_CORE_AFFINITY = 1
to enable the function to set which core the task shall run on.
vTaskCoreAffinitySet
UBaseType_t uxCoreAffinityMask
to record which cores a task can run on using a bitwise value.0101
indicates that the task can run on core 0 and core 2.vTaskCoreAffinityGet
Type | Create Task | Create Affinity Set |
---|---|---|
Dynamic | xTaskCreate |
xTaskCreateAffinitySet |
Static | xTaskCreateStatic |
xTaskCreateStaticAffinitySet |
Restricted | xTaskCreateRestricted |
xTaskCreateRestrictedAffinitySet |
Restricted Static | xTaskCreateRestrictedStatic |
xTaskCreateRestrictedStaticAffinitySet |
uxCoreAffinityMask
to allow the task to run on the default core(s) defined by configTASK_DEFAULT_CORE_AFFINITY
(usually all cores).prvYieldCore(xCoreID);
.vTaskStartScheduler
UBaseType_t
can represent all cores.prvYieldForTask
uxCoreAffinityMask
of the task to ensure it can only run on the allowed core(s) (xCore
).prvSelectHighestPriorityTask
pxPreviousTCB
, which points to the TCB of the task that was previously executed on that core, used to preserve the state before the scheduling occurs.uxCoreAffinityMask
of the task to ensure it can only run on the allowed core(s) (xCore
).