Contact: Olivier Certner <olce.freebsd.statusreports@certner.fr>

The goal of the 256-queue runqueues sub-project is to fix FreeBSD’s POSIX compliance in that different priority levels in the SCHED_FIFO or SCHED_RR scheduling classes must lead to immediate preemption by threads having higher priority. It is part of the bigger scheduling priorities revamp project aiming at rationalizing FreeBSD scheduling interfaces, including having consistent rtprio(2) and POSIX interface behaviors (where it makes sense), enhancing POSIX compliance, removing duplicate code and fixing existing bugs, and enhancing the non-standard parts both for better control and security. Expected benefits are increased usage of FreeBSD as a soft real-time platform, e.g., for video and audio processing in casual desktop uses to professional settings. Readers interested in this topic are invited to consult AsiaBSDCon 2024’s paper and EuroBSDCon 2024’s slides for a wider view, and to contact me for questions, feedbacks or discussions.

Currently, priority levels specified either through the prio field of struct rtprio (rtprio(2) interface) or the sched_priority one of struct sched_param, for real-time scheduling classes (RTP_PRIO_FIFO and RTP_PRIO_REALTIME for the former, SCHED_FIFO and SCHED_RR for the latter) as well as idle-time ones (RTP_PRIO_IDLE), are conflated as follows: Each priority level that has the same quotient when divided by 4 is internally treated the same. In particular, threads being in the same such equivalence class but having higher priority will not preempt other threads in the same class, violating POSIX expectations for SCHED_FIFO and SCHED_RR.

To remedy this situation, we have chosen an impacting internal change on the number of queues per runqueue, and defer to the above-mentioned EuroBSDCon 2024’s slides for more details.

The switch to 256-queue runqueues having non-trivial impacts on the ULE scheduler, we have been analyzing it and tuning the scheduler to preserve its previous behavior with respect to anti-starvation and the effect of nice values. With the goal to perform further testing, we have revived Jeff Roberson’s initial ULE’s test tool, called late (currently available on GitHub).

All the modifications made as part of this sub-project are currently under review, starting with Phabricator’s review D45387 (click on the "Stack" tab to see the full series of reviews).

In the course of this project, we have noticed that the effect of nice values is especially weak, and consequently have produced experimental patches to make their effect stronger. However, it is not clear at this point whether we can increase their effect satisfactorily enough in the current ULE setting.

We have also started another scheduler project about adapting ULE to hybrid architectures, which might also trigger more drastic scheduler changes.

Sponsor: The FreeBSD Foundation