FreeBSD The Power to Serve

FreeBSD Status Report Second Quarter 2024

Here is the second 2024 status report, with 20 entries.

It has been very difficult to publish this status report within the usual schedule: indeed we are late. Unfortunately many of us have been busy with a lot of stuff, both inside and outside FreeBSD, thus some reports arrived late and report publication was slower than usual. Hopefully, this quarter was an exception and next quarter we will already be back on track, with 2024Q3 report published within October 2024.

Have a nice read.

Lorenzo Salvadore, on behalf of the Status Team.



FreeBSD Team Reports

Entries from the various official and semi-official teams, as found in the Administration Page.

FreeBSD Core Team

Contact: FreeBSD Core Team <core@FreeBSD.org>

The FreeBSD Core Team is the governing body of FreeBSD.

DevSummit 202405

The Core Team has presented the status update at the DevSummit 202405. Details are available at https://wiki.freebsd.org/DevSummit/202405

Passing of Mike Karels

It is with a heavy heart that we have learned of Mike Karels' recent passing. We want to offer our sincere condolences to his family, friends, and the community for this loss. Mike was an inspirational fellow developer and will be sorely missed.

For more details about him, please visit the FreeBSD Foundation’s page at:

2024 Core Team Election

Core.12 appointed Muhammad Moinur Rahman (bofh@) and Allan Jude (allanjude@) as the managers of the 2024 Core Team election. After Allan decided to run for Core, Moin continued to handle all the election tasks independently.

The result was announced on June 12th, with 8 members:

  • Allan Jude (allanjude@)

  • Dave Cottlehuber (dch@)

  • Gleb Smirnoff (glebius@)

  • Hiroki Sato (hrs@)

  • Li-Wen Hsu (lwhsu@)

  • Mathieu Arnold (mat@)

  • Olivier Cochard (olivier@)

  • Tobias C. Berner (tcberner@)

Mike Karels has run for the 2024 Core Team election, and was in the top nine candidates in the result. After a discussion, core.13 came to the conclusion that the eight members will serve as the new team.

Core.13 takes office on June 12th, and hold the handover meeting with core.12 on June 21st.

The project thanks the outgoing core.12 members for their service in the last two years:

  • Baptiste Daroussin (bapt@)

  • Benedict Reuschling (bcr@)

  • Ed Maste (emaste@)

  • Greg Lehey (grog@)

  • John Baldwin (jhb@)

  • Emmanuel Vadot (manu@)

  • Mateusz Piotrowski (0mp@)

Commit bits

  • Core approved the src commit bit for Osama Abboud (osamaabb@)

  • Core reactivated the src commit bit for Ryan Libby (rlibby@)


FreeBSD Foundation

Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and worldwide community, and helping to advance the state of FreeBSD. We do this in both technical and non-technical ways. We are 100% supported by donations from individuals and corporations and those investments help us fund the:

  • Software development projects to implement features and functionality in FreeBSD

  • Sponsor and organize conferences and developer summits to provide collaborative opportunities and promote FreeBSD

  • Purchase and support of hardware to improve and maintain FreeBSD infrastructure

  • Resources to improve security, quality assurance, and continuous integration efforts

  • Materials and staff needed to promote, educate, and advocate for FreeBSD

  • Collaboration between commercial vendors and FreeBSD developers

  • Representation of the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity

Last quarter we helped FreeBSD celebrate its 31st anniversary! This community sure loves to celebrate milestones like this one. We not only saw more users sharing their stories on social media, but many commercial users stepped in to promote their use cases and love for FreeBSD. It is exciting to see the growth of this project through the improvements made to FreeBSD, as well as the increase in users and contributors.

Over the past few quarters, we have built up our technology, advocacy, and partnership teams to accelerate our work in improving the operating system, increasing the adoption and visibility of FreeBSD, and increasing the number of partners who help fund our work.

Below you will find updates from each team to see the work we have accomplished to support you, the community and the operating system we all love. But, first I want to share a fundraising update. Last quarter we raised $41,154 towards our goal of raising over $2,000,000. You can see our 2024 budget to understand how we are spending your donations here: https://freebsdfoundation.org/wp-content/uploads/2024/03/Budget2024-Approved-Public.pdf.

Over half the budget goes directly into improving and securing FreeBSD. If there is a security vulnerability out there, we have software developers on staff who can quickly step in, evaluate the situation, and put in a change or workaround if needed. We have a full-time developer who leads the continuous integration efforts and investigates ways to improve the tools to test code, improve test coverage, and help developers be more efficient. We have also allocated more funding towards our advocacy efforts. This includes creating content to highlight FreeBSD’s strengths and differentiators, talking to commercial users and documenting their use cases, and promoting the work you are doing.

Please consider funding our efforts to help keep FreeBSD innovative, secure, and stable by making a donation here: https://freebsdfoundation.org/donate/.

If you are a corporate user, please consider becoming a partner! Go here to find out more information about our partnership opportunities: https://freebsdfoundation.org/our-donors/freebsd-foundation-partnership-program/.

OS Improvements

During the second quarter of 2024, 204 src, 50 ports, and 11 doc tree commits identified The FreeBSD Foundation as a sponsor.

The Foundation is sponsoring 13 projects.

  • Christos Margiolis continued to improve FreeBSD’s audio stack and provide audio developers with useful tools and frameworks to facilitate sound development on FreeBSD. Refer to the Audio Stack Improvements entry for details.

  • Pawel Dawidek is in the final stages of a project to add hierarchical rate limits to OpenZFS. For details, refer to the Hierarchical rate limits for OpenZFS report entry and the pull request in the OpenZFS repository.

  • Long-term contractor Olivier Certner was active in a few different parts of the tree:

    • rtprio(2): Updating the number of queues per runqueue from 64 to 256

    • UnionFS: reviewed work from Jason A. Harmening. Jason’s work fixes many locking problems (wild accesses without locks, deadlocks, etc.), particularly in unionfs_rename() and improves locking logic.

    • Vnode recycling/ZFS ARC reclaim: Reviewed a fix for bug #275594, liaised with upstream to obtain and test a backport, had an EN issued and applied as 13.3-RELEASE-p2, and started longer-term work to improve the vnode reclaiming mechanisms and have ZFS pass the right information

    • ULE scheduler: Updated to work on a single runqueue instead of 3 for POSIX compliance with respect to the number of distinct SCHED_FIFO/SCHED_RR priority levels

    • Miscellaneous: Many (26) reviews, ports updates, and investigated DRM problems

    • Published a EuroBSDCon 2023 conference report in the FreeBSD Journal.

  • Pierre Pronchery continued work on a security-focused project with the Foundation that included:

    • working on a conversion tool from VuXML to OSV

    • automating the generation of VuXML reports across all ports with security/osv-scanner

    • running Coverity Scan reports around bhyve and assisting in rectifying the reported defects

  • Work continued on a joint project between Advanced Micro Devices (AMD) and The FreeBSD Foundation to develop a complete FreeBSD AMD IOMMU driver. This work will allow FreeBSD to fully support greater than 256 cores with features such as CPU mapping and will also include bhyve integration. Konstantin Belousov has been working on various parts of the project, including driver attachment, register definitions, an ACPI table parser, and utility functions. Two key components that need to be completed are context handling, which is mostly a generalization of Intel DMAR code, and page table creation. After this, the AMD driver’s enable bit can be turned on for testing. To follow all of Konstantin’s work, look for src commits tagged with Sponsored by fields for Advanced Micro Devices (AMD) and The FreeBSD Foundation.

  • The Vector Packet Processor (VPP) is an open-source, high-performance user space networking stack that provides fast packet processing suitable for software-defined networking and network function virtualization applications. Tom Jones is wrapping up his part on a project to port VPP to FreeBSD. The code has been shared with RG Nets, a co-sponsor of the work, for extensive testing.

  • Björn Zeeb continued to improve wireless networking on FreeBSD. As with last quarter, the focus was mainly on bug fixes and stability improvements. Most of Bjoern’s 30+ src commits were to LinuxKPI and net80211 code.

  • Philip Paeps is working on a 20-hour-per-month, six-month contract to continue modernizing the FreeBSD cluster. This work includes moving more servers to our newest cluster site at NYI in Chicago.

  • Moin Rahman is under two contracts. Moin is nearing completion of a Center for Internet Security (CIS) hardening guide and continues work to establish pre-commit CI.

  • Mina Galić continues efforts to put FreeBSD cloud-init support on par with Linux support.

  • Mitchell Horne presented his RISC-V work at the FreeBSD developer summit. You can read about the latest developments in the FreeBSD/riscv64 report entry.

  • Refer to Chih-Hsin Chang's OpenStack on FreeBSD report entry for the latest updates on the project to port OpenStack components so that OpenStack can be run on FreeBSD hosts.

Other members of the Foundation’s technology team contributed to FreeBSD development efforts. For example, Mark Johnston, along with Andrew Turner, authored basic routines to build a Flattened Device Tree (FDT) for arm64 bhyve guests. The FDT describes various hardware components like CPUs, memory, UART, PCIe controller, interrupt controller, and platform timer, which the guest OS needs to know about. Ed Maste committed a variety of src contributions, including modernization of tzsetup(8) and correcting an issue with diff(1) options. Balancing their regular responsibilities, Li-Wen Hsu and Joe Mingrone contributed updates and fixes to various ports, including addressing pressing security issues.

Continuous Integration and Workflow Improvement

As part of our continued support of the FreeBSD Project, the Foundation supports a full-time staff member dedicated to improving the Project’s continuous integration system and test infrastructure.

Partnerships and Research

In the Second Quarter, Greg Wallace, the Foundation Partnerships lead, attended the Open Source Summit event in Seattle. There he joined Doug Rabson who gave a talk on the work of the FreeBSD OCI Runtime Extension working Group. You can check it out here. Greg also used the event to connect with a number of key tech companies to advance major joint technology initiatives. Greg’s write up on the event is here.

Work continues on other highly-requested features. RG Nets and others have been making great strides to bring CUDA and related AI stack components to FreeBSD. The Foundation is seeking ways to coordinate across users of FreeBSD to get support for a variety of AI technologies on FreeBSD. One idea is to launch a FreeBSD AI lab that would pool money from supporters to get CUDA fully supported on FreeBSD and to round out DPU driver support. Please contact us if you would like to support such an initiative.

Work continues to leverage the heroic work from the FreeBSD Community to get .NET supported on FreeBSD so that downstream dependencies can in turn better support FreeBSD. More to come on this front soon.

Thanks to the generous grant from Alpha-Omega, the FreeBSD Foundation has undertaken two code audits of important subsystems carried out by Synactiv. Alpha-Omega is an open source project with a mission to protect society by catalyzing sustainable security improvements to the most critical open source software projects and ecosystems. Our most recent monthly update can be found here. The code audits will conclude in July and then we will then undertake a process audit and will also run a 2FA pilot.

In Q1 and Q2, Greg participated in several meetings about various government regulations. In March, he represented FreeBSD at the CISA two-day Open Source Software (OSS) Security Summit alongside other Open Policy Alliance members. Previously, Greg collaborated with OPA to submit comments to CISA’s RFC on how the US Government can support the security and sustainability of Open Source. And in June, The FreeBSD Foundation joined the Open Regulatory Compliance Working Group at the Eclipse Foundation. This group aims to accelerate the development of cohesive cybersecurity processes required for regulatory compliance while offering a neutral environment for hosting technical discussions with the open source community at large.

We are thrilled to welcome Alice Sowerby as a part time, contract Partnerships Program Manager. Alice is an experienced, multi-skilled leader, currently active in a number of open source domains. She is the co-host of the CHAOSS podcast and chair of the TODO group review team for the OSPO Book. Alice is providing program and project management for partnership initiatives, like Alpha-Omega, OCI FreeBSD Runtime Extension WG, and the Enterprise Working Group.

Advocacy

During the second quarter of 2024, we continued growing our efforts to drive awareness, advocate for the project, highlight users, and bring educational content to the FreeBSD community. Below are some of those efforts.

Legal/FreeBSD IP

The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

Go to https://freebsdfoundation.org to find more about how we support FreeBSD and how we can help you!


FreeBSD Release Engineering Team

Contact: FreeBSD Release Engineering Team, <re@FreeBSD.org>

The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.

The Team mourns the loss of Mike Karels, who was the Deputy Release Engineering Lead and mere days before his death agreed to take the role of Release Engineer for 13.4-RELEASE.

The Team managed 14.1-RELEASE, leading to the final RELEASE build and announcement in June. During the second quarter of the year, the Team has gained new members: Ed Maste (Deputy Release Engineer Lead), Dave Cottlehuber, John Hixson, Mahdi Mokhtari, Doug Rabson, Muhammad Moinur Rahman. Planning has started for the upcoming 13.4-RELEASE cycle.

The Release Engineering Team continued providing weekly development snapshot builds for the main, stable/14, and stable/13 branches.


Cluster Administration Team

Contact: Cluster Administration Team <clusteradm@FreeBSD.org>

FreeBSD Cluster Administration Team members are responsible for managing the machines the Project relies on to synchronize its distributed work and communications.

In this quarter, the team has worked on the following:

  • Regular support for FreeBSD.org user accounts.

  • Regular disk and parts support (and replacement) for all physical hosts and mirrors.

  • Cluster software refresh.

  • Moving cluster services to Chicago.

Cluster software refresh

Except for the package builders and developer-facing ("dogfood") machines, the FreeBSD cluster mostly tracks stable/X branches. This quarter, we started moving the stable/13 hosts to stable/14.

At the time of this writing, there are 133 physical machines in the cluster, 48 run current, and 64 have been upgraded to stable/14. The remaining machines are slated for upgrading or decommissioning in the near future. Of the 290 jails in the cluster, 206 run stable/14.

 12.x: Regular   2, Jails   8
 13.x: Regular  19, Jails  68
 14.x: Regular  64, Jails 206
>15.x: Regular  48, Jails   8
-----------------------------
Total: Regular 133, Jails 290
Total installations:      423
Running -RELEASE|{-p*}:    0
Total geographic sites:    16

Moving cluster services to Chicago

Earlier this year, we started building up our new site in Chicago. This quarter, we began decommissioning older machines in New Jersey and moving services to the newer machines in Chicago. Our long-term goal is for Chicago to become our primary location. This work will take several more months to complete.

FreeBSD Official Mirrors Overview

Current locations are Australia, Brazil, Germany, Japan (two full mirror sites), Malaysia, South Africa, Sweden, Taiwan, United Kingdom (full mirror site), United States of America — California, Chicago, New Jersey (primary site), and Washington.

The hardware and network connection have been generously provided by:

New official mirrors are always welcome. We have noted the benefits of hosting single mirrors at Internet Exchange Points globally, as evidenced by our existing mirrors in Australia, Brazil, and South Africa. If you are affiliated with or know of any organizations willing to sponsor a single mirror server, please contact us. We are particularly interested in locations on the United States West Coast and throughout Europe.

See generic mirrored layout for full mirror site specs and tiny-mirror for a single mirror site.

Sponsors: The FreeBSD Foundation and Netzkommune


Continuous Integration

Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet

In the second quarter of 2024, we worked with the project contributors and developers to address their testing requirements. Concurrently, we collaborated with external projects and companies to enhance their products by testing more on FreeBSD.

Important completed tasks:

Work in progress tasks:

  • Merging Pre-commit CI with CIRRUS-CI

  • Designing and implementing pre-commit CI building and testing and pull/merge-request based system (to support the workflow working group)

  • Proof of concept system is in progress.

  • Designing and implementing use of CI cluster to build release artifacts as release engineering does, starting with snapshot builds

  • Simplifying CI/test environment setting up for contributors and developers

  • Setting up the CI stage environment and putting the experimental jobs on it

  • Redesigning the hardware test lab and adding more hardware for testing

Open or queued tasks:

  • Collecting and sorting CI tasks and ideas

  • Setting up public network access for the VM guest running tests

  • Implementing use of bare-metal hardware to run test suites

  • Adding drm ports building tests against -CURRENT

  • Planning to run ztest tests

  • Helping more software get FreeBSD support in its CI pipeline (Wiki pages: 3rdPartySoftwareCI, HostedCI)

  • Working with hosted CI providers to have better FreeBSD support

Please see freebsd-testing@ related tickets for more WIP information, and do not hesitate to join the effort!

Sponsor: The FreeBSD Foundation


Ports Collection

Contact: Tobias C. Berner <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter.

According to INDEX, there are currently 32,471 ports in the Ports Collection. There are currently ~3,497 open ports PRs. The last quarter saw 10,525 commits by 160 committers on the main branch and 1771 commits by 107 committers on the 2024Q2 branch. Compared to last quarter, this means a large decrease in the number of commits on the main branch (down from 12,991) and backports to the quarterly branch (compared to 888). The number of ports also increased (up from 32,244).

The most active committers to main were:

  • sunpoet 3739

  • yuri 1450

  • jbeich 491

  • eduardo 220

  • bofh 200

  • diizzy 197

  • rene 188

  • fernape 156

  • jhale 133

  • arrowd 129

A lot has happened in the ports tree in the last three quarter, an excerpt of the major software upgrades is:

  • pkg 1.21.3

  • Default version of lazarus switched to 3.4.0

  • Default version of fpc switched to 3.2.3

  • Default version of python switched to 3.11

  • chromium updated from 123.0.6312.86 to 126.0.6478.126

  • firefox updated from 124.0.1 to 127.0.2

  • firefox-esr updated from 115.9.0 to 115.12.1

  • rust updated from 1.77.0 to 1.79.0

  • sdl2 updated from 2.6.3 to 2.8.2

  • wlroots updated from 0.17.2 to 0.17.4

  • wine updated from 8.0.2 to 9.0

  • wine-devel updated from 9.4 to 9.11

  • xorg-server updated from 21.1.11 to 21.1.13

  • qt5 updated from 5.15.13 to 5.15.14

  • qt6 updated from 6.6.3 to 6.7.2

  • kf5 updated from 5.115.0 to 5.116.0

  • kf6 updated from 6.0.0 to 6.3.0

  • plasma6 updated from 6.0.2 to 6.1.1

During the last quarter, pkgmgr@ ran 24 exp-runs to test various ports upgrades, updates to default versions of ports, subpackage support and base system changes.


Projects

Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.

Audio Stack Improvements

Contact: Christos Margiolis <christos@FreeBSD.org>

The FreeBSD audio stack is one of those fields that does not attract the same attention and development as others do, since it has been left largely unmaintained, and, although high in quality, there is still room for improvement — from lack of audio development frameworks, to missing userland utilities and kernel driver-related bugs. This project is meant to touch on all those areas, and as such, is more of a general improvement project, than an implementation of a specific feature.

Important work since last report:

  • Asynchronous audio device detach is now possible. This functionality already ships with FreeBSD 14.1-RELEASE, as well as 14-STABLE.

  • Got rid of the "snd_clone" device cloning framework used in sound(4) and replaced it with DEVFS_CDEVPRIV(9). More info about behavior changes in the commit description. Also ships with 14.1-RELEASE and 14-STABLE.

  • Several sound(4) crash and bug fixes.

  • More out of the box support for snd_hda(4) laptop sound.

  • Series of commits that clean up and simplify parts of sound(4).

  • Several fixes regarding the OSS API, with the most notable so far being a proper implementation of the SNDCTL_AUDIOINFO and SNDCTL_ENGINEINFO IOCTLs.

  • Started implementing audio(3), an OSS audio and MIDI library.

  • Took over maintenance of virtual_oss(8).

Future work includes:

  • Implementation of an audio(8) utility, in similar fashion to mixer(8).

  • Implementation of a bluetooth device management utility.

  • Improve mixer(3) and mixer(8).

  • Improve documentation and test suite where needed.

  • Attempt to find a better (ideally automatic) way to handle snd_hda(4) pin-patching. This is an experimental attempt and is not guaranteed to actually yield a working result.

You can also follow the development process in freebsd-multimedia@, where I post regular reports.

Sponsor: The FreeBSD Foundation


FreeBSD GitHub Pull Request Report

Contact: Warner Losh <imp@FreeBSD.org>

The FreeBSD Project has been trying an experiment to accept contributions via GitHub pull requests. We have learned a lot in the last year that we’ve been doing this. We have created a number of rules relating to the pull requests. In general, pull requests should only be for things that are user-visible, add value to the project and are ready to go, modulo final review.

Current status:

  • We are able to keep up with the pull requests doing everything by hand, but only if we do it at least weekly.

  • We have discovered the by-hand process is tedious and error-prone.

  • We can stage multiple pull requests in a testing branch so we can batch-up testing.

  • We can automatically land the result so merged pull requests show up as merged in GitHub infrastructure.

We need help with automating the process:

  • Add automated testing that is context specific (for example, run igor over man pages).

  • Add build/install tests that test-boot the resulting image.

  • Automate common tasks, like man page corrections, into staging process.

  • Add simple smoke testing for the staging branch for tier 1 architectures.

  • Investigate optionally integrating Jenkins testing to scale up the size of changes we can accept.

  • Integrate with Bugzilla problem reports and Phabricator reviews.

  • Improve the submitter experience on GitHub by automating common feedback to mistakes in the pull requests.

  • Create checklists for submitters to reduce errors.

  • Create better reporting about pull requests, especially the frequent contributors of good pull requests.

We are coordinating on FreeBSD’s Discord in the #github-hacking channel. Join us there to pitch in, or just chat about the project.

Once things are fine-tuned, we want to publicity to steer contributors, at least the base system, to GitHub pull requests. We need more developers looking at the FreeBSD GitHub pull requests. We will also need more developers to review and land pull requests once it is automated and the automation has matured. We sincerely hope that we can improve the FreeBSD contribution experience with this, as well as gain useful fixes from the community.


Userland

Changes affecting the base system and programs in it.

Capsicum-rs

Contact: Alan Somers <asomers@FreeBSD.org>

Capsicum is a lightweight OS capability and sandbox framework implementing a hybrid capability system model.

I have adopted the library providing Rust bindings for Capsicum, and extended it with support for libcasper(3) and cap_net(3). It is already being used by net-mgmt/nfs-exporter and by a TLS-enabled FTP server (the FTP server is closed-source, but all of the interesting bits reside in an open source library, and an example server can be found at https://github.com/bolcom/libunftp/tree/master/crates/unftp-sbe-fs/examples).

Sponsor: Axcient


Service jails — Automatic jailing of rc.d services

Contact: Alexander Leidinger <netchild@FreeBSD.org>

Service jails extend the rc(8) system to allow automatic jailing of rc.d services. A service jail inherits the filesystem of the parent host or jail, but uses all other limits of the jail (process visibility, restricted network access, filesystem mounting permissions, sysvipc, …​) by default. Additional configuration allows inheritance of the IPs of the parent, sysvipc, memory page locking, and use of the bhyve virtual machine monitor (vmm(4)).

The base system infrastructure and the basesystem rc.d services are committed to 15-current, and the handbook / rc article updates are committed to the documentation. Next steps are to extend services in the ports collection to be able to make use of it.

If you want to put e.g. nginx into a service jail and allow IPv4 and IPv6 access, simply change rc.conf(5) to have:

nginx_svcj_options=net_basic
nginx_svcj=YES

While this does not have the same security benefits as a manual jail setup with a separate filesystem and IP/VNET, it is much easier to set up, while providing some of the security benefits of a jail like hiding other processes of the same user.

Any testing and feedback (even as simple as "service X works in a service jail") is welcome.


Kernel

Updates to kernel subsystems/features, driver support, filesystems, and more.

Hierarchical rate limits for OpenZFS

Contact: Pawel Dawidek <pawel@dawidek.net>

Rate limits allow to limit bandwidth and number of metadata operations per dataset.

Hierarchical rate limits allow for granular resource control especially in shared environments, eg. when a single ZFS pool serves data to multiple, independent virtual machines or jails.

Because the limits are hierarchical, they are enforced like the quota property: the children datasets must obey the limits of the parent dataset.

The limits can be configured using six new properties: limit_bw_read, limit_bw_write, limit_bw_total, limit_op_read, limit_op_write, limit_op_total The limit_bw_* properties limit the read, write, or combined bandwidth, respectively, that a dataset and its descendants can consume. Those limits are in bytes per second. The limit_op_* properties limit the read, write, or both metadata operations, respectively, that a dataset and its descendants can generate. Those limits are in number of operations per second.

Limits are applied to file systems and ZFS volumes (and their snapshots).

The initial work is done and the pull request is up for review.

Sponsor: Klara Systems Sponsor: FreeBSD Foundation


A low-cost conditional execution mechanism

Contact: Marko Vlaić <mvlaic@freebsd.org> Contact: Bojan Novković (mentor) <bnovkov@freebsd.org>

This project aims to implement a low-cost conditional execution mechanism, similar to the static_key interface in Linux. The current working name is zcond, as in zero-cost conditional. The idea is to reduce the overhead of rarely used features in performance sensitive kernel code paths. Specifically, code blocks of the following, simple structure, are targeted:

if(some_global_flag == true) {
    do_some_additional_task();
}

A block like this can cause performance overhead in a frequently executed piece of code.

The motivating use case for the mechanism is tracing (as outlined here in the wiki), but the project will identify more areas where this mechanism could be applied.

The backbone of the mechanism is runtime instruction hot patching, making it highly architecture dependent. Great care has to be given to security, because kernel instructions are overwritten at runtime. More details and some implementation ideas can be found in the project proposal on the project wiki page. More documents describing the implementation and design decisions will be produced, as the project moves along.

The goal of the project is to produce a working implementation of the mechanism for the x86-64 architecture. It will then be applied to an existing piece of kernel code and benchmarked. If the benchmark results come out positive, it will be ported to other architectures.

This project is a part of the Google Summer of Code 2024.. The project is still in its early stages, but any feedback would be highly appreciated.

Help wanted

  • General feedback on the mechanism, API and implementation

  • More use cases besides tracing are welcome

  • Code review

  • Name suggestions

Sponsor: Google LLC (GSoC 2024)


Architectures

Updating platform-specific features and bringing in support for new hardware platforms.

FreeBSD/riscv64

Contact: Mitchell Horne <mhorne@FreeBSD.org>
Contact: Ruslan Bukin <br@FreeBSD.org>
Contact: Jari Sihvola <jsihv@gmx.com>

The FreeBSD/RISC-V project is providing support for running FreeBSD on the RISC-V Instruction Set Architecture.

BSDCan Devsummit Updates

Mitchell gave a presentation on the status of the FreeBSD/riscv64 platform at the June developers' summit, part of BSDCan 2024. The presentation discussed the challenges of supporting the evolving RISC-V ISA, and gave a brief overview of some available hardware targets. It is informal in nature, but available to watch on youtube.

StarFive JH7110 SoC / VisionFive v2

Work has been ongoing to support the JH7110 RISC-V SoC in FreeBSD. This SoC is present in several existing RISC-V SBCs. The primary support target is the VisionFive v2, but this work is likely to benefit similar hardware such as the Pine64 Star64 and the Milk-V Mars.

At present, the support in CURRENT is partially complete. What is working: * UART * clk/reset drivers * MMC/SD

The next target is working ethernet, which is supported with extensions to the if_eqos driver. This code is functional, but still in review.

Work on this has been performed by Jari, with review, testing, and integration from Mitchell.

T-HEAD/XuanTie CPU Support

As discussed in the devsummit presentation, these CPUs present several barriers to support. The problems come primarily from custom vendor extensions: non-coherent device I/O with custom cache management instructions, and a custom (spec-violating) implementation of page-based memory types. Combined, these quirks require difficult and invasive changes to pmap, busdma, and early boot code.

At the same time, we are seeing an increasing amount of available hardware based on this IP, and so support becomes a repeated question and incentive.

Work on page-based memory types is underway and expected to land soon in CURRENT. This feature allows individual virtual memory pages to be assigned specific properties, e.g. cacheability requirements. The riscv64 pmap has been extended to support the official RISC-V 'Svpbmt' extension, and the T-HEAD version of PBMT. These alternative encodings are incompatible, but provide similar functionality.

Work on the device coherence and cache management challenges will begin next quarter.

The hope is that this foundational work will (eventually) enable board support for available RISC-V hardware such as: * Allwinner D1 (Nehza) * Sipeed Lichee Pi 4A * Beagle-V Ahead * Milk-V Pioneer

RISC-V Hypervisor

Experimental support for the RISC-V hypervisor in bhyve/vmm(4) is under active development. Currently, single-core VMs are possible, and the guest can boot to multi-user.

Note: the "H" (hypervisor) extension is not implemented by any known existing or upcoming RISC-V hardware. It is fully supported by software simulators such as Spike or QEMU.

Sponsor: The FreeBSD Foundation (mhorne’s work) Sponsor: UKRI (br’s work)


Cloud

Updating cloud-specific features and bringing in support for new cloud platforms.

FreeBSD on Microsoft HyperV and Azure

Contact: Microsoft FreeBSD Integration Services Team <bsdic@microsoft.com>
Contact: freebsd-cloud Mailing List
Contact: The FreeBSD Azure Release Engineering Team <releng-azure@FreeBSD.org>
Contact: Wei Hu <whu@FreeBSD.org>
Contact: Souradeep Chakrabarti <schakrabarti@microsoft.com>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>

In this quarter, we have published the 14.1-RELEASE on Azure Marketplace. Souradeep Chakrabarti from Microsoft has improved the TLB flushing mechanism in Azure.

Work in progress tasks:

The above tasks are sponsored by The FreeBSD Foundation, with resources provided by Microsoft.

Open tasks:

Sponsor: Microsoft for people in Microsoft, and for resources for the rest
Sponsor: The FreeBSD Foundation for everything else


OpenStack on FreeBSD

Contact: Chih-Hsin Chang <starbops@hey.com>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>

The OpenStack on FreeBSD project aims to integrate OpenStack cloud infrastructure with the FreeBSD operating system, utilizing FreeBSD’s unique features while maintaining compatibility with OpenStack standards.

In the second quarter of 2024, we continued to advance the project significantly. One of our key activities was attending BSDCan 2024, where we presented on "Towards a Robust FreeBSD-based Cloud: Porting OpenStack Components". This presentation helped increase the project’s visibility and attracted interest from potential contributors.

We expanded the current single-node proof-of-concept (POC) site to a three-node setup. This involved detailed environment setup and network planning. Additionally, we began migrating the base to FreeBSD 15.0-CURRENT to ensure our project stays aligned with the latest advancements in FreeBSD.

We also started working on adapting the manual installation steps and code patches into FreeBSD ports, aiming to streamline the installation process for future users. Another major milestone was initiating work on making bare-metal provisioning function on both FreeBSD instances and service hosts for OpenStack Ironic.

Looking forward to the next quarter, our focus will be on refining these advancements and further enhancing the project’s robustness and ease of use. Specifically, we aim to upgrade our OpenStack components from the Xena version to a more recent release, as Xena has entered the "unmaintained" phase and will soon reach EOL. We welcome any suggestions and contributions from the community to help us achieve our goals.

Sponsor: The FreeBSD Foundation


Documentation

Noteworthy changes in the documentation tree, manual pages, or new external books/documents.

Documentation Engineering Team

Contact: FreeBSD Doceng Team <doceng@FreeBSD.org>

The doceng@ team is a body to handle some of the meta-project issues associated with the FreeBSD Documentation Project; for more information, see FreeBSD Doceng Team Charter.

Document changes

During this quarter multiple pieces of the documentation were updated including the removal of mentions to ports that were removed from the ports tree, typos, URL corrections, and asciidoc improvements.

These are three highlighted contributions in this quarter:

FreeBSD Translations on Weblate

Q2 2024 Status
  • 17 team languages

  • 214 registered users

9 new translators joined Weblate:

  • BFR (de_DE)

  • Elwood (de_DE)

  • Freebsd-Freund (de_DE)

  • MSantos (pt)

  • SINF-KEN (fr_FR, nl_NL)

  • aperechnev (ru)

  • edsonwolf

  • fdecunta (es_ES)

  • wheatfox (zh_CN, zh_HANS)

Languages
  • Chinese (Simplified) (zh-cn) (progress: 7%)

  • Chinese (Traditional) (zh-tw) (progress: 3%)

  • Dutch (nl) (progress: 1%)

  • French (fr) (progress: 1%)

  • German (de) (progress: 1%)

  • Greek (el) (progress: 1%)

  • Indonesian (id) (progress: 1%)

  • Italian (it) (progress: 5%)

  • Korean (ko) (progress: 32%)

  • Norwegian (nb-no) (progress: 1%)

  • Persian (fa-ir) (progress: 3%)

  • Polish (progress: 2%)

  • Portuguese (progress: 0%)

  • Portuguese (pt-br) (progress: 22%)

  • Spanish (es) (progress: 36%)

  • Turkish (tr) (progress: 2%)

We want to thank everyone that contributed, translating or reviewing documents.

And please, help promote this effort on your local user group, we always need more volunteers.

Packages maintained by DocEng

During this quarter the following work was done in packages maintained by doceng@:

Ports

Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.

GCC on FreeBSD

Contact: Lorenzo Salvadore <salvadore@FreeBSD.org>

Upstream has released GCC 14.1. Therefore port lang/gcc-14 has been created, as well as lang/gcc15-devel which tracks the new latest development GCC version. Previous major versions of GCC are being updated as well.

Only very little work has been done on existing bugs for GCC ports this quarter, contrary to the original plan stated in the precedent report. This is due to the fact that I noticed an increasing interest in upstream to support GCC on FreeBSD, which however gets blocked by the fact that GCC sources do not build out-of-the-box on our platform, but needs instead many patches which are in our ports tree framework. It is then necessary to focus on upstreaming all of those patches, possibly at once and not one at the time as it has been done until now, and to also regularly run the tests suite, fixing all tests that fail. Then I am working on a new setup of automation that allows me to build and test all supported GCC versions as efficiently as possible, including GCC from sources but without the ports tree framework support. All of this has significantly slowed down usual GCC ports maintainership (users of the -devel ports have probably noticed that they are being updated much less frequently), but I am confident that it will eventually pay off.

It should also be noted that Robert Clausecker has enabled lang/gcc11, lang/gcc12, and lang/gcc13 for riscv64 architecture as the ports build fine on 15-CURRENT.


KDE on FreeBSD

Contact: Adriaan de Groot <kde@FreeBSD.org>

The KDE on FreeBSD project packages CMake, Qt, and software from the KDE Community, for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma (for both X11 and Wayland) and hundreds of applications that can be used on any FreeBSD machine. The KDE team is part of desktop@, building the software stack to make FreeBSD beautiful and usable as a daily driver graphical desktop workstation.

We missed a quarter, while the broader KDE community celebrated the KDE Megarelease of KDE Frameworks 6, KDE Plasma 6 and KDE Gear. The latest software is still not available on FreeBSD, pending more testing and some integration work.

Infrastructure

CMake was updated several times and is now version 3.29.6, the latest upstream release. CMake in the ports framework now supports setting parallel-build for tests.

Qt5 is now on long-term support and updates only rarely. An update to 5.15.14 and WebEngine to 5.15.17 was landed in May.

Qt6 and the Python bindings for Qt, Pyside, were updated to version 6.7.2.

KDE Stack

KDE Gear, Plasma and Frameworks release happen very regularly. These updates arrive in the FreeBSD ports tree only piecemeal, due to lagging work on compatibility and testing.

  • KDE Frameworks reached version 6.3.0

  • KDE Plasma 6 Desktop was updated to version 6.0.4

  • KDE Gear 6 has not yet arrived in the ports tree

The KDE ecosystem includes a wide range of ports — most maintained by kde@, all building on a shared base of Qt and KDE Frameworks. The KDE team updates them all as needed. This quarter the KDE team would like to thank Tobias C. Berner, Gleb Popov and Jason E. Hale again for keeping things up-to-date.


FreeBSD Erlang Ecosystem Ports update

Contact: FreeBSD Erlang mailing list <erlang@FreeBSD.org>

The Erlang runtime system, commonly known as the BEAM, provides a runtime that is used by a number of programming languages and applications in the FreeBSD ports collection.

Notable changes in 2024 include:

  • adding OTP27, the latest Erlang runtime release, Elixir 1.17, and Gleam 1.20

  • more than 57 point release updates so far in 2024

  • improved inline documentation within Erlang ports

  • moved RabbitMQ port to the generic UNIX build, decoupling this from Elixir as a build-time dependency. This enables moving RabbitMQ to the latest supported release. Users of RabbitMQ need to update each quarter to avoid being stuck on an unsupported release of Erlang/OTP + RabbitMQ

Note that as the upstream Erlang OTP team only commit to supporting the two latest major releases, more and more point updates are arriving for OTP26-27, but not for the older Erlang runtime releases, which are now unlikely to get security and bug fixes.

During 2024Q3, the Erlang team is planning to:

  • migrate the base lang/erlang port to OTP26 and update related dependencies

Additional testing and community contributions are welcome, please reach out on the mailing list, especially if you are able to help testing of specific port updates.


Third Party Projects

Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.

Containers and FreeBSD: Pot, Potluck and Potman

Contact: Luca Pizzamiglio (Pot) <pizzamig@FreeBSD.org>
Contact: Bretton Vine (Potluck) <bv@honeyguide.eu>
Contact: Michael Gmelin (Potman) <grembo@FreeBSD.org>

Pot is a jail management tool that also supports orchestration through Nomad. Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker: a repository of Pot flavours and complete container images for usage with Pot and in many cases Nomad.

During this quarter, there were no new Pot releases.

Potluck images saw some updates again though. All images have been rebuilt for the new quarterly packages, a new Opensearch container has been added. Additional features, updates and fixes have been committed to containers like PostgreSQL-Patroni or Nomad-Server-TLS.

In total, there are 58 container images and templates available now.

As always, feedback and patches are welcome.

Sponsors: Nikulipe UAB, Honeyguide Group


Last modified on: August 18, 2024 by Lorenzo Salvadore