FreeBSD The Power to Serve

FreeBSD Quarterly Status Report 3rd Quarter 2021

This report covers FreeBSD related projects for the period between July and September, and is the third of four planned reports for 2021, and contains 42 entries.

The third quarter of 2021 was quite active in lots of different areas, so the report covers a bunch of interesting work including but not limited to boot performance, compile-time analysis, hole-punching support, various drivers, ZFS raidz expansion, an update to the sound mixer, and many more.

Regrettably, the status report got a bit delayed, but we hope that the aphorism better late than never can apply here.

Yours,
Daniel Ebdrup Jensen, with status hat on.



FreeBSD Team Reports

Entries from the various official and semi-official teams, as found in the Administration Page.

FreeBSD Foundation

Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Donations from individuals and corporations are used to fund and manage software development projects, conferences, and developer summits. We also provide travel grants to FreeBSD contributors, purchase and support hardware to improve and maintain FreeBSD infrastructure, and provide resources to improve security, quality assurance, and release engineering efforts. We publish marketing material to promote, educate, and advocate for the FreeBSD Project, facilitate collaboration between commercial vendors and FreeBSD developers, and finally, represent the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

Here are some highlights of what we did to help FreeBSD last quarter:

Fundraising Efforts

Fundraising last quarter wasn’t as spectacular as we were hoping. But, then again, people tend to take vacations during the summer months, which makes it that more difficult for our funding requests to go through the management chain for approvals.

So far this year we’ve raised $180,000 towards our $2,000,000 spending budget. Why do we need so much money? Well, last year we decided to make more significant software contributions to FreeBSD. In order to do that, we had to grow our team. We developed a technology roadmap based on input we were receiving from commercial users as well as market trends. Based on the roadmap, we identified positions we needed to fill.

This year we’ve hired three full-time software developers, one full-time ARM kernel developer, and one project manager. We also are funding wifi work full-time and some other projects to help with FreeBSD on the desktop. You can read about this effort to attract new users and contributors to the Project in individual entries elsewhere in this status report.

Our growth wasn’t just in our technology team, but in our advocacy team too. Here we hired a marketing coordinator and technical writer to provide more educational and informational content. You’ll see in the Advocacy and Education section below all the work we did to promote FreeBSD, provide community engagement, education opportunities, and informative content to help pave the path to getting started with FreeBSD.

You’ll find out how we used your donations here and in individual entries throughout this status report.

We’re passionate about supporting you, the FreeBSD community, but we can’t do it without your financial support.

Please consider making a donation to help us continue and increase our support for FreeBSD in 2021: https://www.FreeBSDfoundation.org/donate/.

We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies!

OS Improvements

During the third quarter, Foundation staff and grant recipients committed 420 src tree changes, 24 ports tree changes, and 11 doc tree changes. This represents 38%, 48%, and 16% of src, port, and doc commits which identify a sponsor.

You can read about Foundation-sponsored projects in individual quarterly report entries:

  • Base System OpenSSH Update

  • Fixes for msdosfs_rename VOP

  • Hole Punching

  • Improved amd64 UEFI boot

  • Intel Wireless driver support

  • LLDB Debugger Improvements

  • Microchip LAN743x mgb(4) Device Driver

  • OpenZFS RAIDZ Expansion update

  • Using OCF in WireGuard

  • syzkaller on FreeBSD

Foundation-sponsored arm64 work:

  • Support for booting FreeBSD on the Arm Armv8-A AEM simulator

  • sha256 instruction support to libmd(4) on arm64

  • Initial work to support RAS, PAC and BTI on arm64

Continuous Integration and Quality Assurance

The Foundation provides a full-time staff member and funds projects to improve continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project.

See the separate Continuous Integration entry for details.

Supporting FreeBSD Infrastructure

The Foundation provides hardware and support for the Project. Last quarter, we continued supporting the test cluster at Sentex, purchased a few needed drives and spam filtering software for project email. We also set up a new and more efficient hardware request/purchase process for clusteradm to use.

Partnerships and Commercial User Support

We met virtually with corporate users and started travelling again in late Q3 for some in-person meetings. The goals of the meetings are to facilitate collaboration between commercial users and FreeBSD developers. We also met with companies to discuss their needs and share that information with the Project.

FreeBSD Advocacy and Education

Much of our effort is dedicated to Project advocacy. This may involve highlighting interesting FreeBSD work, producing literature, attending events, or giving presentations. The goal of the literature we produce is to teach people FreeBSD basics and help make their path to adoption or contribution easier. Other than attending and presenting at events, we encourage and help community members run their own FreeBSD events, give presentations, or staff FreeBSD tables.

The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, working together on projects, and facilitating collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. We finally made it back to our first in-person meeting with the Open Source Summit in late September. We are also continuing to attend virtual events. In addition to attending and planning virtual events, we are continually working on new training initiatives and updating our selection of how-to guides to facilitate getting more folks to try out FreeBSD.

Check out some of the advocacy and education work we did last quarter:

We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/.

You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/.

Legal/FreeBSD IP

The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

Go to https://www.FreeBSDfoundation.org to find more about how we support FreeBSD and how we can help you!


FreeBSD Release Engineering Team

Contact: FreeBSD Release Engineering Team, <re@FreeBSD.org>

The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.

Throughout the third quarter, several development snapshots builds were released for the main, stable/13, and stable/12 branches. More work had been done on various release-related tooling following the conversion from Subversion to Git. Additionally, the team had drafted the proposed schedule for the upcoming 12.3-RELEASE.

Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD Foundation


Cluster Administration Team

Contact: Cluster Administration Team <clusteradm@FreeBSD.org>

FreeBSD Cluster Administration Team members are responsible for managing the machines the Project relies on to synchronise its distributed work and communications. In this quarter, the team has worked on the following:

  • Fixed www.FreeBSD.org mirror sync source

  • Improvements to GeoDNS for {download,ftp,pkg}.FreeBSD.org

  • Added IPv6 support

  • More optimal mirror selection in Asia

  • Finished installing a new mirror site in Japan, sponsored by Broadband Tower, Inc.

  • Full mirror site (ftp, pkg, git, doc, dns,…​)

  • The network and machine resources for this mirror are generously sponsored by the Cloud and SDN Laboratory at BroadBand Tower, Inc., one of the Internet data center service providers in Tokyo, Japan, with 300+ Gbps international IP transit bandwidth

  • Improved the retention script used for the artifact server of CI cluster to offer a mix of the latest artifacts but also a selection of older artifacts for comparison, space permitting

  • Ongoing day to day cluster administration

  • Replacing failed disks

  • Babysitting pkgsync

Work in progress:

  • Move pkg-master.nyi to new hardware

  • Improve the package building infrastructure

  • Review the service jails and service administrators operation

  • Setup powerpc pkgbuilder/ref/universal machines

  • Search for more providers that can fit the requirements for a generic mirrored layout or a tiny mirror

  • Upgrading public-facing cluster services from 12.2-STABLE to 13.0-STABLE

  • Working with doceng@ to improve https://www.freebsd.org and https://docs.freebsd.org

  • Improve the web service architecture

  • Improve the cluster backup plan


Continuous Integration

Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet

The FreeBSD CI team maintains the continuous integration system of the FreeBSD project. The CI system checks the committed changes can be successfully built, then performs various tests and analysis over the newly built results. The artifacts from those builds are archived in the artifact server for further testing and debugging needs. The CI team members examine the failing builds and unstable tests and work with the experts in that area to fix the code or adjust test infrastructure.

During the third quarter of 2021, we continued working with the contributors and developers in the project to fulfil their testing needs and also keep collaborating with external projects and companies to improve their products and FreeBSD.

Important changes:

New jobs:

Retired jobs:

Work in progress and open tasks:

  • Designing and implementing pre-commit CI building and testing (to support the workflow working group)

  • Designing and implementing use of CI cluster to build release artifacts as release engineering does

  • Collecting and sorting CI tasks and ideas here

  • Testing and merging pull requests in the FreeBSD-ci repo

  • Reducing the procedures of CI/test environment setting up for contributors and developers

  • Setting up the CI stage environment and putting the experimental jobs on it

  • Setting up public network access for the VM guest running tests

  • Implementing using bare metal hardware to run test suites

  • Adding drm ports building tests against -CURRENT

  • Planning to run ztest tests

  • Adding more external toolchain related jobs

  • Improving maturity of the hardware lab and adding more hardware under test

  • Helping more software get FreeBSD support in its CI pipeline (Wiki pages: 3rdPartySoftwareCI, HostedCI)

  • Working with hosted CI providers to have better FreeBSD support

Please see freebsd-testing@ related tickets for more WIP information, and don’t hesitate to join the effort!

Sponsor: The FreeBSD Foundation


Ports Collection

Contact: René Ladan <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter.

There are currently 46,500 ports in the Ports Tree according to FreshPorts. The open PR count for ports is currently 2,588, of which 608 are unassigned. The last quarter saw 9,833 commits on the "main" branch by 148 committers and 743 commits on the quarterly branch by 54 committers. Compared to 2021q2, activity mostly remained the same, the PR counts increased a bit but there were also more commits to the quarterly branch.

During 2021q3, we welcomed Daniel Engberg (diizzy@) and Yasuhiro Kimura (yasu@), and we said goodbye to culot@ and grog@ after their commit bits were taken in for safekeeping.

Two new uses were introduced: angr to help with testing Python-related ports and mlt to help depending on mlt, a multimedia framework for TV broadcasting. Ruby saw minor updates for the 2.6, 2.7, and 3.0 series.

The "big" packages were also updated: pkg to 1.17.2, Firefox to 93.0, and Chromium to 92.0.4515.159.

As usual, exp-runs were performed by antoine@ but only those of July were recorded. There were 5 exp-runs to test adding CLOCK_\*_COARSE compatibility definitions for Linux and to test ports updates.

Furthermore, rene@ gave a presentation "portmgr: behind the scenes" at EuroBSDCon 2021 about how portmgr came to be and its daily activities.


Documentation Engineering Team

Contact: FreeBSD Doceng Team <doceng@FreeBSD.org>

The doceng@ team is a body to handle some of the meta-project issues associated with the FreeBSD Documentation Project; for more information, see FreeBSD Doceng Team Charter.

During the last quarter, Philip Paeps (philip@) and Li-Wen Hsu (lwhsu@), already src and ports committers, were granted documentation commit bits, both now have all commit bit types. Ed Maste (emaste@) who is already a src committer was granted a documentation commit bit. We also said goodbye to Murray Stokely (murray@), Gábor Kövesdán (gabor@), Warren Block (wblock@), and Sevan Janiyan (sevan@). Gábor Kövesdán (gabor@) and Warren Block (wblock@) are now former members of the doceng@ team; we thank them for their years of service.

Implicit (blanket) approvals were documented in the Committers Guide. That does not cover all cases, but doceng@ encourages any FreeBSD committer from ports and src to contribute to the doc tree.

All ports/packages misc/freebsd-doc-* were updated. They have the entire documentation set from the FreeBSD Documentation Project, like Handbook, FAQ, articles, and more.


FreeBSD Website Revamp - WebApps working group

Contact: Sergio Carlavilla <carlavilla@FreeBSD.org>

Working group in charge of creating the new FreeBSD Documentation Portal and redesigning the FreeBSD main website and its components. FreeBSD developers can follow and join the working group on the FreeBSD Slack channel #wg-www21. The work will be divided into four phases:

  1. Redesign of the Documentation Portal

    Create a new design, responsive and with global search. (Work in progress, almost complete)

  2. Redesign of the Manual Pages on web

    Scripts to generate the HTML pages using mandoc. (Work in progress)

  3. Redesign of the Ports page on web

    Ports scripts to create an applications portal. (Not started)

  4. Redesign of the FreeBSD main website

    New design, responsive and dark theme. (Not started)


Projects

Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.

Boot Performance Improvements

Contact: Colin Percival <cperciva@FreeBSD.org>

Colin Percival is coordinating an effort to speed up the FreeBSD boot process. For benchmarking purposes, he is using an EC2 c5.xlarge instance as a reference platform and is measuring the time between when the virtual machine enters the EC2 "running" state and when it is possible to SSH into the instance.

This work started in 2017, leading to a conference presentation, "Profiling the FreeBSD kernel boot", and quickly yielded roughly 4850 ms of improvements (starting from a baseline of about 30 seconds).

Since June, another roughly 9790 ms of time has been shaved off the boot process, taking it down to approximately 15 seconds. There is still more work to be done; in particular, while the loader and kernel have been profiled, the TSLOG system Colin is using does not currently support userland profiling.

Issues are listed on the wiki page as they are identified; the wiki page also has instructions for performing profiling. Users are encouraged to profile the boot process on their own systems, in case they experience delays which don’t show up on the system Colin is using for testing.

This work is supported by Colin’s FreeBSD/EC2 Patreon.


Git Migration Working Group

Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: Warner Losh <imp@FreeBSD.org>
Contact: Ed Maste <emaste@FreeBSD.org>
Contact: FreeBSD-git mailing list
Contact: #gitcvt channel on EFnet

With the end of the third quarter of 2021, we are approaching the one-year anniversary of the transition of the doc and src repositories from Subversion to Git. The 2021Q4 branch of the ports tree has been created, the third quarterly branch created using Git.

During 2021Q3, repository hooks have been refined as problems were discovered and a few lingering references to Subversion were updated in the Committer’s Guide. The Working Group continues to track progress on two permissively-licensed Git-compatible tools: Gitup and Game of Trees. Gitup is a small, dependency-free tool to clone and update git repositories. It is used only to keep a local tree up-to-date, and has no support for local commits. Game of Trees is a version control client that is compatible with Git repositories. It provides a user interface and workflow that is distinct from that of Git. It is in no way intended to be a drop-in replacement for Git, but can be used to develop software maintained in a Git repository.

Gitup and Game of Trees are currently available as ports and packages. Future work will evaluate them as candidates for the base system.

Remaining work includes continuing with refinements to Git documentation in the Handbook, committing new and updated repository hooks for managing branch content and commit mail, and surveying pre-commit hooks to use the CI cluster to build release artifacts. Priorities have been set for the next working group tasked with refining our Git workflow. The first meeting will be in mid-October.

Sponsor: The FreeBSD Foundation (in part)


LLDB Debugger Improvements

Contact: Kamil Rytarowski <kamil@moritz.systems>
Contact: Michał Górny <mgorny@moritz.systems>

According to the upstream description, "LLDB is a next generation, high-performance debugger. It is built as a set of reusable components which highly leverage existing libraries in the larger LLVM Project, such as the Clang expression parser and LLVM disassembler."

FreeBSD includes LLDB in the base system. At present, it has some limitations compared to the GNU GDB debugger, and does not yet provide a complete replacement. This project spans from July 2021 to January 2022 and aims to make LLDB suitable for debugging FreeBSD kernels.

The work so far was focused on improving the compatibility between LLDB and other servers implementing the GDB Remote Protocol. Multiple bugs were fixed that limited LLDB’s functionality when interfacing with GNU GDB’s gdbserver and QEMU’s gdbserver implementations. Support for debugging executables running on gdbserver architectures other than amd64 was fixed, and was explicitly tested on arm64 and i386.

This effort also resulted in adjusting lldb-server’s implementation to conform better to the standard set by GDB. Thanks to these improvements, the LLDB client needs to provide less divergent code paths depending on the server used, and the single code path used is tested more thoroughly.

The work also involved improvements to the LLDB API and code deduplication, that result in reducing the maintenance effort and lowering the bar for future contributions. The test coverage was improved, increasing the likelihood that any future problems will be detected before they affect users.

The introduced changes are expected to be shipped with LLDB 14.0.

Sponsor: The FreeBSD Foundation


Linux compatibility layer update

Contact: Dmitry Chagin, <dchagin@FreeBSD.org>
Contact: Edward Tomasz Napierala, <trasz@FreeBSD.org>

The goal of this project is to improve FreeBSD’s ability to execute unmodified Linux binaries. The current support status of specific Linux applications is being tracked on the Linux app status Wiki page.

The vdso(7) implementation was reworked. The futex(2) support was overhauled to make use of FreeBSD’s native umtx mechanism. It now supports priority-inheritance futexes, in addition to fixing several bugs.

The rt_sigsuspend(2) and sigaltstack(2) syscalls are now supported on ARM64.

The faccessat2(2), clone3(2) system calls are now implemented. The CLONE_CLEAR_RESETHAND option is now supported.

The prctl(2) syscall now supports PR_SET_NO_NEW_PRIVS.

The ptrace(2) syscall now supports PTRACE_GET_SYSCALL_INFO, which is a prerequisite to support newer strace(1) versions.

There is ongoing work to make Linuxulator on arm64 on par with the amd64 one; right now it’s good enough for development work.

Sponsor: EPSRC (Edward’s work)


Sound mixer improvements

Contact: Christos Margiolis <christos@FreeBSD.org>
Contact: Hans Petter Selasky <hselasky@FreeBSD.org>

This project is an attempt to improve the capabilities of the OSS sound mixer on FreeBSD. This means a new mixer(3) library, a complete rewrite of mixer(8), and updates to sound(4).

Development started during Google Summer of Code 2021, but will likely continue independently.

Applied changes:

Patches, comments and discussion are all welcome.


Base System OpenSSH Update

Contact: Ed Maste <emaste@freebsd.org>

OpenSSH, a suite of remote login and file transfer tools, was updated from version 7.9p1 to 8.7p1 in the FreeBSD base system.

FreeBSD base OpenSSH includes a number of local bug fixes, configuration changes, and small features. As part of the work for this update, I submitted some of these upstream and am preparing to do the same with the remaining changes.

OpenSSH now supports FIDO/U2F devices, although additional work is required to enable this in the FreeBSD base system. This includes importing a pair of dependencies: libcbor, and libfido2. Within the next couple of months I expect to import these, enable FIDO/U2F support, and update to OpenSSH version 8.8p1.

NOTE: OpenSSH 8.8p1 will disable the ssh-rsa signature scheme by default, so some additional work is required for this next update. For more information please see the Important note for future FreeBSD base system OpenSSH update mailing list post.

Sponsor: The FreeBSD Foundation


Kernel

Updates to kernel subsystems/features, driver support, filesystems, and more.

Arm64 improvements

Contact: Andrew Turner <andrew@FreeBSD.org>

FreeBSD has been ported to the Arm Armv8-A Architecture Envelope Model (AEM), an Armv8 architecture simulator. AEM is useful for Armv8 development and testing, because it supports different configurations, including the ability to enable or disable different extensions. As part of this work, the virtio legacy pci driver was fixed and ACPI resource parsing code was updated to work with the ACPI tables that the model provides.

Libmd(4) has been updated to use the sha256 instructions when available. This can lead to a large performance improvement when these instructions are available. For example, on the Apple M1 under a hypervisor, the time to calculate the sha256 of a 1GB file has decreased from a median of 3.46s to 0.5s.

Using an ifunc (indirect function) in a static binary is now supported. This will allow different implementations of a function to be used depending on which hardware it is being run on. Using an ifunc in dynamic binaries was already supported.

The arm64 ID register definitions have been updated to match the Armv8.5 specification and other special registers have been updated to the Armv8.7 spec.

There have been numerous cleanups in decoding the arm64 ID registers in the kernel to ensure we provide a consistent view of the hardware to userspace if it reads the emulated ID register directly, or uses an ELF HWCAP value.

There has been early work on supporting the Arm Reliability, Availability, and Serviceability (RAS) extension. The external aborts it may use are now handled in the kernel.

Support for the Arm Pointer Authentication extension is under review. This extension allows programs to use new instructions that add a hash to unused bits in a code or data pointer. They can then later check the hash in a way that will either, depending on the CPU, create an invalid pointer or raise an exception. This can be used to protect the function return address from being overwritten, making Return Oriented Programming (ROP) attacks difficult. The change supports both userspace and kernel pointer authentication. It is waiting on debugger changes to be sent upstream so lldb can clear the hash when needed, e.g., in a stack trace.

Work has started on supporting the Branch Target Identification extension. This adds a flag to the page tables to disallow unintended targets from some branch instructions. When a CPU branches to a new location from a register, the CPU will check if the instruction is correct. This will protect, e.g., against a function pointer being called when it points to the middle of a function. The kernel has been built with this feature enabled and tested in the Arm AEM.

Sponsor: The FreeBSD Foundation


FreeBSD on Microsoft HyperV and Azure

Contact: Microsoft FreeBSD Integration Services Team <bsdic@microsoft.com>
Contact: freebsd-cloud Mailing List
Contact: The FreeBSD Azure Release Engineering Team <releng-azure@FreeBSD.org>
Contact: Wei Hu <whu@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>

The changes for building official images on Azure Marketplace, which fulfill the requirements of Azure and FreeBSD cloud images like disk layout and UEFI for Gen2 VM, along with some minor improvements like configurations to speed up booting, have been imported.

Above tasks are sponsored by The FreeBSD Foundation, with resources provided by Microsoft.

Microsoft Azure Network Adapter (MANA) is the new network adapter from Microsoft which will be available in the Azure public cloud. It provides SR-IOV NIC as a Virtual Function (VF) to guest OS running on Hyper-V. Wei has been working on its driver for a while and committed in this quarter.

This task is sponsored by Microsoft.

Work in progress tasks:

Open tasks:


Improved amd64 UEFI boot

Contact: Konstantin Belousov <kib@FreeBSD.org>

UEFI BIOS on PC provides a much richer and more streamlined environment for pre-OS programs, but also imposes some requirements on the programs that are executed there, OS loaders in particular. One such requirement is that the loader must coordinate its memory use with the BIOS, only using memory that was allocated for it. Even after the loader handoff to the operating system kernel, there are still some memory regions that are reserved by the BIOS for different reasons. Examples are runtime services code and data.

On the other hand, legacy/CSM BIOS boot works with memory completely differently; there are known memory regions that hold BIOS data and must be avoided. Otherwise, the memory is considered free for loader and OS to use. (Of course it is not that straightforward, the definition of known regions is up to the vendor and there are a lot of workarounds there.)

The BIOS boot puts the kernel and preloaded data (like modules, memory disk, CPU microcode update etc) at the contigous physical memory block starting at 2M. This algorithm goes back to how the i386 kernel boots.

Also, when preparing to pass control to the kernel, the loader creates very special temporary mappings, where the low 1G of physical address space is mapped 1:1 into virtual address space, and then repeated for each 1G until the virtual memory end. The kernel knows about its physical location and the temporary mapping, and constructs kernel page tables assuming that the physical address of the text is at 2M.

This mechanism of loader to kernel handoff was left unchanged when the loader gained support for the UEFI environment. The loader prepares kernel and auxiliary preload data in a so-called staging area while UEFI boot services are active, and after EFI_BOOT_SERVICES.ExitBootServices(), the temporary mapping is activated and the staging area is copied at 2M.

An advantage at that time was that no changes to the kernel were needed. But there are issues; the biggest is that memory at 2M might be not free for reuse. For instance, BIOS runtime code or data might be located there. Or there might be no memory at 2M at all. Or trampoline page table or code, or even some parts of the staging area overlapping with the 2M region where staging area is copied. The outcome was a hard to diagnose boot time failure, typically a hard hang when the loader started the kernel.

Another limitation is the 1G transient mapping, which due to copying means that the total size of preloaded data cannot exceed around 400M for everything, including kernel, memory disks, and anything else. Also the code to grow the staging area on demand was quite unflexible, only able to grow the staging area in place.

The work described in this report allows the UEFI loader on amd64 to start the kernel from the staging area without copying. Kernel assumptions about the hand-off were explicitly identified and documented. The kernel only requires the staging area to be located below 4G together with the trampoline page table (this is a consequence of CPU architecture requiring 32bit protected mode to enter long mode), be 2M aligned and the whole low 4G mapped 1:1 at hand-off. The kernel computes its physical address and builds kernel page tables accordingly.

Making the kernel boot with staging area in-place required identifying all places where the amd64 kernel had a dependency on its physical location. The most complicated part was application processors startup, which required rewriting initialization code, which we were able to streamline as result. In particular, when an AP enters paging mode, it does so straight into the correct kernel page table, without loading intermediate trampoline page table.

The updated loader automatically detects if the loaded kernel can handle in-place staging area ('non-copying mode'). If needed, this can be overridden with the loader’s copy_staging command. For instance, 'copy_staging enable' tells the loader to unconditionally copy the staging area to 2M regardless of kernel capabilities (default is 'copy_staging auto'). Also, the code to grow the staging area was made much more robust, allowing it to grow without hand-tuning and recompiling the loader.

Sponsor: The FreeBSD Foundation


ENA FreeBSD Driver Update

Contact: Michal Krawczyk <mk@semihalf.com>
Contact: Artur Rojek <ar@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>

ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used.

Completed since the last update:

  • Update ENA driver to v2.4.1

  • Introduce full kernel RSS API support

  • Allow reconfiguration of the RSS indirection table and hash key

  • Support netmap on the c6gn/m6i AWS instance types

  • Fix race between detach function and some of the procedural sysctl nodes

  • Add new statistics counters

  • Improve safety of the reset handling routine

  • Avoid mbuf collapse on Tx path for the LLQ condition

Work in progress:

  • Prototype the driver port to the iflib framework

  • MFC the ENA v2.4.1 driver to the FreeBSD 11/12/13-STABLE branches

  • Add IPv6 L4 checksum offload support to the driver

Sponsor: Amazon.com Inc


Hole-punching support on FreeBSD

Contact: Ka Ho Ng <khng@FreeBSD.org>

Hole-punching functionality allows turning a contiguous range of bytes into a hole for a given file. File systems supporting hole-punching may deallocate the file system space from the given file. One use case for the functionality is to convert TRIM/UNMAP/DEALLOCATE requests from a virtual machine guest to hole-punching calls on the host’s side, thus allows reclamation of file system space when unneeded by the guest.

A set of APIs and KPIs are added that developers can call to invoke hole-punching on a given file, if the underlying file system exposes that functionality. For file systems not supporting hole-punching there is a fallback implementation in the kernel which does zero-filling instead. Besides the APIs and KPIs addition, the truncate(1) utility is expanded by adding a -d flag to support invoking hole-punching as well.

At the time of writing this report, OpenZFS and tmpfs are the file systems that support hole-punching.

Sponsor: The FreeBSD Foundation


Intel Networking on FreeBSD

Contact: Intel <freebsd@intel.com> Contact: Kevin Bowling <kbowling@FreeBSD.org>

lem(4)/em(4)/igb(4) and ix(4) were updated with various common code improvements obtained from DPDK. There have also been several bug fixes from the community:

Intel has updated ixl(4) with various improvements:


Intel Wireless driver support

Contact: Bjoern A. Zeeb <bz@FreeBSD.ORG>

The Intel Wireless driver update project aims to bring support for newer chipsets along with mac80211 LinuxKPI compat code. The dual-licensed Intel driver code was ported in the past for the iwm(4) native driver; using the LinuxKPI compat framework allows us to use the driver directly, with only very minor modificationsi that we hope will be incorporated into the original driver.

After the initial snapshot at the end of June, another snapshot was released in early September. Thank you to everyone testing on CURRENT and reporting back.

While we keep updating the driver and all the compat code needed for that, the focus now is on stability and adding support for newer 802.11 standards. The driver is currently set to 11a/b/g and 11n will be next before we look at 11ac.

We are currently trying to get as much into FreeBSD as is possible and makes sense. Full integration into FreeBSD main is waiting for a policy update.

For the latest state of the development or code, please follow the referenced wiki page or the freebsd-wireless mailing list. We expect to release another snapshot soon and hope to also support stable/13 again with that one.

Sponsor: The FreeBSD Foundation


Microchip LAN743x mgb(4) Device Driver

Contact: Ed Maste <emaste@freebsd.org>

Microchip’s LAN7430 and LAN7431 are PCIe Gigabit Ethernet interface ICs, with an integrated PHY (LAN7430) or RGMII interface (LAN7431).

In 2019 Gerald ND Aryeetey, a FreeBSD Foundation intern, developed the mgb(4) driver for these devices. It was added to the tree but not yet connected to the build. Since it was committed a number of sweeping iflib changes were made, which included updates to mgb(4).

I have addressed some outstanding issues in the driver, and have now added the module to the build. It is available in -CURRENT snapshot images. The driver is functional, although some additional work is still needed. In particular, configuration of the Receive Filtering Engine, and VLAN and RX/TX checksum offload are required. Caveats and notes are described in the man page.

I am very interested in feedback and test results.

Sponsor: The FreeBSD Foundation


Fixes for msdosfs_rename VOP

Contact: Konstantin Belousov <kib@FreeBSD.org>
Contact: Peter Holm <pho@FreeBSD.org>

Our msdosfs(5) implementation is old code and has a relatively large legacy cost. In particular, even though it got fine-grained locking and miscellaneous bugfixes over time, sometimes a serious issue is found in it.

Recently trasz@ found that msdosfs rename can be easily deadlocked. Further examination of rename code revealed a lot of issues with locking, potential use after free, and filesystem structure corruption.

As part of the update, locking in the msdosfs rename code was reworked. We need to lock up to four vnodes, and check one path to ensure that rename does not create circular parent relations between directories. For that, the locking procedure was copied from UFS rename, where all vnodes except the first are locked in try-mode. Lockless relockup was added to msdosfs and the directory path checker was changed to non-blocking mode.

During this work, all known issues were fixed and msdosfs passes the enhanced stress2 suite of tests.

Sponsor: The FreeBSD Foundation (kib’s contributions)


OpenZFS RAIDZ Expansion update

Contact: Matthew Ahrens <matt@mahrens.org>

Project

This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn’t sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks).

FreeBSD’s ZFS implementation comes from the OpenZFS project. This work will be integrated into the OpenZFS repo, with support for FreeBSD and Linux.

Status

The work is functionally complete, and a pull request has been opened.

Remaining Work

Remaining work includes some code cleanup, design documentation, addressing test failures.

We also need to solicit code reviewers and respond to code review feedback.

Sponsor: The FreeBSD Foundation


RFC1191 support in IPSEC tunnels

Contact: Wojciech Macek <wma@semihalf.com>

The Semihalf team has been working on providing support for RFC1191 in IPSEC tunnels. This allows the PMTUD algorithm to dynamically discover the maximum path MTU the tunnel can handle and update tunnel parameters accordingly. As a result, a better performance can be observed as there is no need for packet fragmentation.

The newly introduced rework includes:

Sponsor: Stormshield


Realtek rtw88 support

Contact: Bjoern A. Zeeb <bz@FreeBSD.ORG>

This project aims to bring in support for Realtek’s rtw88 chips.

With the growing LinuxKPI compatibility code an initial port of the Realtek rtw88 driver was done. This gives us support for a second driver after Intel and helps to validate and grow the LinuxKPI code base. Changes and overall time needed to get the driver compiling were very little. At the moment we are seeing DMA issues which prevent most people from loading firmware or using the driver later on.

Thanks to everyone who was brave enough to give this initial code drop a try.

While this is a leasure time project we would love to better support Realtek wireless devices and will keep working on this.

For the latest state of the development or code, please follow the referenced wiki page or the freebsd-wireless mailing list.


Marvell SDHCI improvements and ACPI support

Contact: Bartlomiej Grzesik <bag@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>

The Semihalf team was working on the ACPI support for the Marvell Octeon TX2 CN913x and Armada 7k/8k SoCs' SD/MMC controller, as well as extending and improving the related generic parts of the FreeBSD OS.

Applied changes:

  • Add ACPI_GET_PROPERTY to access Device Specific Data (DSD) commit b91fc6c43a81

  • Introduce generic device_get_property/device_has_property, which allow to obtain DT/ACPI properties with the same generic code commit 3f9a00e3b577

  • Switch mmc_helper to device_* API, to access the controller description from DT/ACPI in a unified way commit 8a8166e5bcfb

  • Add ACPI support for the sdhci_xenon driver commit d78e464d2330 and commit adbce5ff747b

  • Apply other minor improvements and fixes related to properties parsing in core mmc code and sdhci_xenon driver.

Sponsor: Semihalf


Stack gap handling improvements

Contact: Dawid Gorecki <dgr@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>

Stack gap is a feature that randomizes the stack address by creating a random sized gap between the top of stack and strings area. The current implementation of this mitigation can cause issues for some applications e.g. Firefox (PR239873). Until now the only workaround for this problem was disabling the stack gap for those programs, as is done for ntpd.

Semihalf team has been working on fixing the root causes of stack gap related problems.

One of the issues could be observed when setting the stack limit to a low value with setrlimit(2). Since the stack gap size can be significant, and the process had no knowledge of how large the stack gap is, this caused programs to abort immediately after returning from a syscall due to the stack extending past the limit. The fix involves increasing the soft resource limit (rlim_cur) by the size of stack gap during the setrlimit(2) call. This fixes the issue with ntpd without disabling the stack gap entirely. The patch is currently under review.

The second identified problem is related to the way the thread stacks are handled. Thread stacks are calculated using the kern.usrstack sysctl, which should point to the beginning of the stack. This sysctl returned a constant value depending on the ABI and the presence of the stack gap was ignored. A new sysctl was introduced, and the thread library was updated to use it accordingly. Patches D31897 and D31898 are currently under discussion. These fix the issues with Firefox and Thunderbird not starting with the stack gap feature enabled.

Sponsor: Stormshield


syzkaller on FreeBSD

Contact: Mark Johnston <markj@FreeBSD.org> Contact: Michael Tuexen <tuexen@FreeBSD.org>

syzkaller is a coverage-guided operating system kernel fuzzer. See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller.

In the past quarter we made a concerted effort to shrink the backlog of reports from the public syzbot instance. A number of long-standing locking bugs in the socket subsystem have been fixed, and the SCTP protocol implementation has seen many bug fixes as well. Beyond that, many bugs in various other kernel subsystems have been fixed and the backlog has become substantially smaller over the past quarter. As a direct result of this effort, we have been able to identify regressions more easily and fix bugs closer to the time of introduction. Work is still ongoing to further shrink the backlog.

KASAN (Kernel Address SANitizer) was enabled in the default kernel configuration used by syzbot, which has greatly enhanced our ability to root-cause and fix kernel bugs. See the kernel-sanitizers entry in the 2021q2 quarterly report for an introduction to KASAN and KMSAN. KASAN helps ensure that memory safety bugs manifest more deterministically, improving syzkaller’s ability to find reproducers and simplifying debugging.

A KMSAN (Kernel Memory SANitizer) port was committed to FreeBSD’s main branch in August. Some initial work has been done to make it usable by syzkaller (mainly, kcov(4) required several small modifications to work in a KMSAN-enabled kernel), and a number of bugs were found and fixed using private syzkaller instances.

Goals for the next several months include:

  • Addition of a KMSAN target in syzbot.

  • Further reduction in the syzbot backlog.

  • Upstreaming syzkaller patches to support fuzzing of the Linuxulator.

  • Fuzzing using ZFS-based VM images.

Sponsor: The FreeBSD Foundation


Using OCF in WireGuard

Contact: John Baldwin <jhb@FreeBSD.org>

I have looked at updating the data path crypto in the upstream WireGuard driver to use the in-kernel OpenCrypto Framework for the data path. Data packets sent over a WireGuard tunnel are encrypted with the Chacha20-Poly1305 AEAD cipher. Unlike TLS and IPsec, Wireguard uses an 8 byte nonce rather than a 12 byte nonce with this cipher.

To date, most of the work has focused on updating OCF to better support multiple nonce (and tag/MAC) lengths for a given cipher. I resurrected an old branch I had previously started on for this aimed at supporting all of the AES-CCM NIST KAT vectors many of which use non-default nonce and tag lengths. I have refined the approach to better fit the existing OCF model where nonce and MAC lengths are properties of a session (similar to key lengths). (My earlier branch had made the nonce length a property of individual operations instead.) Mostly this entailed extending the /dev/crypto interface to permit setting these parameters for a session. Existing tests for OCF run in userland and use the /dev/crypto interface including both the cryptocheck utility and the NIST KAT vector tests.

Building on these framework changes, I have extended the existing Chacha20-Poly1305 cipher in OCF to support both 8 and 12 byte nonces including in the accelerated ossl(4) driver. I also have a patch against the upstream WireGuard FreeBSD driver to make use of this for the dataplane and have verified that it interoperates with the stock WireGuard driver.

Future work will focus on upstreaming the OCF changes as well as additional review of the upstream WireGuard driver.

Sponsor: The FreeBSD Foundation


Intel Volume Management Device driver update

Contact: Alexander Motin <mav@FreeBSD.org>

Intel VMD is used by Intel’s VROC (Virtual RAID on chip) to isolate parts of PCI tree, including NVMe devices, behind one or several VMD devices. Each VMD device has 3 memory windows to access config space and memory of its child devices, plus some number (I saw 19 and 33) of MSI-X interrupt vectors to forward their MSI and MSI-X interrupts through.

The vmd(4) driver was first introduced in FreeBSD 13.0, but was found to have a number of problems addressed in this update: - The VMD device was made to look more like a regular PCI-to-PCI bridge, implementing the regular pcib(4) interface and using the standard pci(4) bus driver on top. - Memory and bus numbers resource management was rewritten to properly handle resource requests using memory windows of the VMD device. - Interrupt handling was completely rewritten to distribute child interrupts among available VMD MSI-X vectors, setting them up to be handled via standard OS mechanisms with minimal overhead instead of custom dispatching via taskqueue. Due to the limited number of vectors and routing abilities of VMD, children are limited to only one MSI or three MSI-X vectors per device to avoid sharing. The limits can be tuned, depending on the number of VMD vectors and children. To improve the flexibility the nvme(4) driver was made to work with a limited number of vectors, starting from just one MSI/MSI-X if needed.

Sponsor: iXsystems, Inc.


Ports

Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.

Improve CPE Data in the Ports Collection

Contact: Bernhard Froehlich <decke@FreeBSD.org>

Common Platform Enumeration (CPE) is an open standard for naming hardware and software components. Originally developed by MITRE, it is now maintained by NIST under the aegis of the National Vulnerability Database. A CPE URI uniquely identifies a device or program by its vendor, product name, version, revision, etc. NIST maintains the official CPE dictionary which lists CPE names for all software versions that have ever been mentioned in an advisory.

In the FreeBSD Ports Collection it has been possible to annotate CPE data with USES=cpe since 2014 but only around 1000 ports out of 30.000 did use it. This is why a script called chkcpe was created to validate existing CPE data and find new possible matches automatically.

Why do we need it?

It allows comparing CPE URIs for installed packages against published vulnerability data and will give us a much better and more complete pkg audit. As a side effect we will also have a better overview of vulnerable ports in the FreeBSD Ports Collection that need to be patched or updated.

How can I help?

In this phase there is no easy possibility for port maintainers to help. Creating separate PRs only to add CPE data does not make sense because the overhead is very high. This is why I did spend some time on chkcpe to improve the review and commit workflow. Right now review and commit consumes around 3 minutes per port.

If you are a Ports Committer and want to help please get in touch!

Open Tasks

  • Review remaining reports (~1800) and update ports when appropriate

  • Improve matching quality to find more possible matches

  • Support using CPE data in pkg audit

  • Scan for vulnerable ports in the Ports Collection


FreeBSD Erlang Ecosystem Ports update

Contact: FreeBSD Erlang mailing list <erlang@FreeBSD.org>

The Erlang runtime system, commonly known as the BEAM, provides a runtime that is used by a number of programming languages and applications in the FreeBSD ports collection.

Earlier this year, both Elixir and Erlang runtimes were brought up to date, but as separate ports, to enable porters and users to test applications side by side.

In Q3, the current runtimes have been brought across as defaults - this means that lang/elixir and lang/erlang are running as the latest releases of these superb programming languages and runtimes.

Older releases of lang/erlang-runtime{21,22,23} are still available as ports. The very old releases prior to OTP20 have been removed from the ports tree, as they are no longer supported upstream either.

Only newer OTP releases include the updated SSL application that will correctly validate cross-signed certificates, as used in Let’s Encrypt’s upcoming root certificate deprecations.

Further details on these changes are well documented at Erlang/OTP impact of DST Root CA X3 expiration and DST Root CA X3 expiration update

All of the NIF driver related ports that pull in other FreeBSD ports tree dependencies have been updated to match the newer lang/erlang release, and a number of ports that are not being updated in their upstream community, have therefore been marked as broken.

The Erlang team is planning to:

  • remove the deprecated OTP20 and OTP21 runtimes in 2021Q4

  • remove ports directly dependent on erlang- and elixir- languages, where they are more commonly installed via mix and rebar3 tools, the standard community build tool chain.

Additional testing and community contributions are welcome; please reach out on the mailing list, especially if you are able to help testing of specific port updates.


KDE on FreeBSD

Contact: Adriaan de Groot <kde@FreeBSD.org>

The KDE on FreeBSD project aims to package almost all of the software from the KDE Community for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma (for both X11 and Wayland) and hundreds of applications that can be used on any FreeBSD machine.

The KDE team (kde@) is part of desktop@ and x11@ as well, building the software stack to make FreeBSD beautiful and usable as a daily-driver graphics-based desktop machine.

Q3 is summer in the northern hemisphere: bicycles were biked and mountains were hiked and the team was psyched to chase the software we like. And there were plenty of things to chase:

  • Three CMake releases landed, ending up with CMake 3.21.3 and libpkg support that once again works with CPack to produce FreeBSD packages.

  • Monthly releases of KDE Frameworks, KDE Plasma, and KDE Gear kept the exp-runs going. kde@ would like to thank Antoine for overseeing our many exp-run requests.

  • The KDE Plasma applications for monitoring the state of the system — ksysguard, Plasma system monitor, and supporting libraries — received a number of updates. FreeBSD support code has been merged upstream.

  • Across all of the Qt and KDE packages, an attempt was made to reduce how "heavy" the dependencies are. In general this means removing developer-only dependencies, avoiding the installation of test-code, etc. The underlying idea is to cut down the size of the installation when specific ports are installed, while preserving the "developer batteries included" character of the meta-ports.

  • A runtime-incompatibility was introduced by MySQL 5.7.34, which affected KDE’s personal-information-management applications and email. This was patched in the Qt ports.

  • The mixer application in KDE Plasma now defaults to using pulseaudio.

  • accountsservice was updated, and then patched with code from OpenBSD.

  • kdiff3 was updated to 1.9.3, now with upstream patches for some corner cases originally reported on FreeBSD.

  • kimageannotator and ksnip were updated, for nicer screenshots.

  • kraft, the small-business support application, was updated to version 0.97.

  • krita had an upstream release to address a performance issue, which then landed in FreeBSD.

  • kstars, the interactive planetarium and sky-watching application, was updated to 3.5.5.

  • latte-dock, used as an alternative launcher, updated to 0.10.2.

  • libphonenumber, Google’s library for dealing with all the ways phone numbers are represented around the world, received multiple updates.

  • maliit-framework, for on-screen keyboards, as added to the ports tree.

  • pipewire was kept up-to-date. This is a next-generation video-handling framework, and is being increasingly used in Wayland environments for efficient passing of video data between applications.

  • poppler was updated to version 21.09, with significant speed improvements.

  • sddm was made a little more robust when installed on its own, with xinitrc support.

  • math/eigen2 was removed.


OpenSearch on FreeBSD

Contact: OpenSearch Team <opensearch@FreeBSD.org>

OpenSearch is a community-driven, open-source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists of a search engine daemon, OpenSearch, and a visualization and user interface, OpenSearch Dashboards. OpenSearch enables people to easily ingest, secure, search, aggregate, view, and analyze data. These capabilities are popular for use cases such as application search, log analytics, and more. With OpenSearch people benefit from having an open-source product they can use, modify, extend, monetize, and resell how they want.

After the release of OpenSearch 1.0.0, a FreeBSD OpenSearch team was created to coordinate the efforts for porting it to FreeBSD. Because ElasticSearch 7 and Kibana 7 were already in the ports tree, we could rely on these ports to get started quickly (kudos to the elastic@ team) and could focus on the parts that already changed between the previous codebase and the fork. The ports have been committed to the FreeBSD ports tree as textproc/opensearch and textproc/opensearch-dashboards, and currently provide the latest 1.0.1 version of OpenSearch.

Contributing

The ports have been thoroughly tested in our testing environments and on some production workloads, but we are actively looking for feedback from users. Feel free to send us an email to report successes or failures.


Valgrind - Official Support for FreeBSD has Landed

Contact: Paul Floyd <pjfloyd@wanadoo.fr>

Valgrind is an instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.

With the release of version 3.18.0 of Valgrind, official FreeBSD support has landed upstream. The two supported architectures are amd64 and i386 (x86).

The next step will be to update the FreeBSD port. This will involve switching from code that was maintained out-of-tree for over 15 years to using the official upstream tarball.

As Valgrind is closely tied to operating system details, obtaining official FreeBSD support was the result of significant effort from dozens of developers.


Wine on FreeBSD

Contact: Gerald Pfeifer <gerald@FreeBSD.org> Contact: Damjan Jovanovic <damjan.jov@gmail.com>

Wine allows running Windows programs on FreeBSD.

This quarter we focused on the wine-devel port that tracks mainline development, followed half a dozen of version updates, simplified the port, removed three options (SDL, VKD3D, VULKAN) which are unconditionally active now, and fixed a number of portability issues.

These changes will make their way into the primary Wine port over the coming weeks.

After working on our Wine ports for more than 21 years, maintaining for more than 19 years, time has come to hand over the baton.

Luckily Damjan has stepped up and is going to look after wine-devel and associated ports - thanks! Help with the following is still very welcome:

  • WineHQ bug 50257

  • FreeBSD Bugzilla 257533

  • Maintain daily (or at least regular) test builds of upstream Wine. This quarter this triggered two dozen fixes in support of FreeBSD upstream, though usually it’s quite less effort.


Ztop

Links:
Repository
Port

Contact: Alan Somers <asomers@FreeBSD.org>

ztop is a new utility that displays ZFS datasets' I/O in real time, like top, but for ZFS. Unlike the existing zpool iostat, which only displays traffic at the level of a pool, ztop displays it at the level of individual datasets.

Previously, there was no tool that could display this information. The Prometheus Node Exporter can export it into Prometheus, from which it can be viewed after the fact with various other tools, but that requires a large stack of software, and isn’t real-time.

I started ztop this quarter, and it is now fully functional. However, it has revealed a performance problem within the kernel. In the presence of more than a few thousand datasets, the sysctl interface is too slow and ztop becomes sluggish. Future work, if I have the time, will address that problem.

Sponsor: Axcient


Miscellaneous

Objects that defy categorization.

-CURRENT compilation time analysis

Contact: Wolfram Schneider <wosch@FreeBSD.org>

Re-building FreeBSD from source takes a lot of CPU resources. Depending on your machine it takes between 20min and several hours. At the end of `make buildworld' we log the real time and which parameters are used for parallel build. E.g.

time make -j $(sysctl -n hw.ncpu) buildworld > buildworld.log 2>&1
tail buildworld.log
>>> World build completed on Sat Sep  4 20:58:00 UTC 2021
>>> World built in 7235 seconds, ncpu: 3, make -j3
     7235.61 real     20527.30 user       915.88 sys

The build process runs in several steps. It would be great to know which step takes most of the time, and what are the effects of special build parameters. With a small patch in Aug 2021 we get this information now:

egrep '>>> stage| real ' buildworld.log
>>> stage 1.1: legacy release compatibility shims
        0.28 real         0.18 user         0.10 sys
>>> stage 1.2: bootstrap tools
      165.99 real       472.58 user        11.56 sys
>>> stage 2.1: cleaning up the object tree
       21.47 real        36.96 user        14.14 sys
       15.87 real        29.14 user        11.87 sys
>>> stage 2.3: build tools
        2.42 real         3.79 user         0.62 sys
>>> stage 3: cross tools
        9.92 real        18.49 user         1.75 sys
>>> stage 3.1: recording build metadata
        0.07 real         0.01 user         0.06 sys
>>> stage 4.1: building includes
       16.62 real        36.46 user         9.48 sys
>>> stage 4.2: building libraries
     5440.89 real     15724.60 user       482.58 sys
>>> stage 4.3: building lib32 shim libraries
      615.91 real      1654.77 user       164.58 sys
>>> stage 4.4: building everything
      937.23 real      2540.06 user       205.47 sys

In this example, we spent most of the time in "stage 4.2: building libraries", 77% of the CPU time and 75% of the real time. Now running the buildworld with the parameter WITHOUT_TOOLCHAIN=yes we get a 3.3x faster build. Instead of 2h it will be done in 36 minutes!

time make -j $(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=yes buildworld > buildworld.log 2>&1
>>> World build completed on Fri Sep 17 12:31:41 UTC 2021
>>> World built in 2207 seconds, ncpu: 3, make -j3
     2207.44 real      5710.83 user       676.16 sys
egrep '>>> stage| real ' buildworld.log
>>> stage 1.1: legacy release compatibility shims
        0.35 real         0.20 user         0.16 sys
>>> stage 1.2: bootstrap tools
       20.47 real        51.98 user         5.12 sys
>>> stage 2.1: cleaning up the object tree
       20.92 real        34.45 user        13.57 sys
       16.33 real        28.59 user        12.33 sys
>>> stage 2.3: build tools
        2.59 real         3.90 user         0.86 sys
>>> stage 3: cross tools
       10.46 real        18.62 user         2.35 sys
>>> stage 3.1: recording build metadata
        0.07 real         0.03 user         0.05 sys
        0.08 real         0.03 user         0.05 sys
>>> stage 4.1: building includes
       15.50 real        33.03 user         9.29 sys
>>> stage 4.2: building libraries
      750.31 real      1962.43 user       218.60 sys
>>> stage 4.3: building lib32 shim libraries
      684.05 real      1780.35 user       213.22 sys
>>> stage 4.4: building everything
      677.87 real      1787.13 user       186.82 sys

Using WITHOUT_TOOLCHAIN=yes is fine as long as there are no major LLVM compiler changes.

If you dislike this feature or suspect it is causing trouble you can disable it with the variable TIME_ENV=""

Next task: get more detailed timing information for the long-running stages (4.2, 4.3, 4.4)


Third-Party Projects

Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.

Gitlab 14.3 available

Contact: Matthias Fechner <mfechner@FreeBSD.org>

GitLab is the DevOps Platform. Bring velocity with confidence, security without sacrifice, and visibility into DevOps success.

Version 14.3 now available on FreeBSD.


helloSystem

Contact: Simon Peter <probono@puredarwin.org>
Contact: #helloSystem on irc.libera.chat, mirrored to #helloSystem:matrix.org on Matrix

What is helloSystem?

helloSystem is FreeBSD preconfigured as a desktop operating system with a focus on simplicity, elegance, and usability. Its design follows the “Less, but better” philosophy.

Q3 2021 Status

  • Version 0.6.0 of helloSystem has been published including many contributed features and bugfixes

    • Improved window management with KWin as the window manager

    • Simplified Filer file manager

  • Contributors have started to port the helloSystem desktop environment, helloDesktop, to the FreeBSD Ports and Packages Collection (see changelog at https://github.com/helloSystem/ISO/releases/tag/r0.6.0 for details)

Installable Live ISO images and a full changelog are available at https://github.com/helloSystem/ISO/releases/tag/r0.6.0

Contributing

Help is wanted in a number of areas, especially in the areas of the FreeBSD core OS and kernel, and Qt/C++.


Containers & FreeBSD: Pot, Potluck & Potman

Contact: Luca Pizzamiglio (Pot) <pizzamig@freebsd.org>
Contact: Stephan Lichtenauer (Potluck) <sl@honeyguide.eu>
Contact: Michael Gmelin (Potman) <grembo@FreeBSD.org>

Pot is a jail management tool that also supports orchestration through Nomad.

In the last quarter, a new release 0.13.0 with a number of major new features was made available.
Layered images allow a split into a foundation image component that only changes seldom and that e.g. contains the basic jail and packages and a "leaf" image component on top with potentially small additions that might change more frequently, like software built in a CI pipeline. Since it is not the complete image that has to be rebuilt each time, image creation and distribution can be sped up immensely.
Also a copy-out command has been included where the container state that was initialised from within the container can be made persistent and reinjected into additional containers afterwards.

Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker: a repository of Pot flavours and complete container images for usage with Pot and in many cases Nomad.

Here we had a busy quarter. Not only did we commit a number of new images like Nextcloud which can be deployed via Nomad, but the images used to build the foundation of a virtual data center (Nomad, Consul and Vault) themselves have received major updates.

For these images, further improvements for even better security and reliability will likely be finished in the coming quarter.

Also, we are aware that right now the advantages of the container approach as it is described in Virtual DC Part I/Part II/Part III are not yet obvious and transparent enough and also no longer completely up to date, so we plan to provide additional documentation as soon as we find the time to do so.

Potman aims to simplify building Pot images with Vagrant and VirtualBox based on the Potluck approach, e.g. as part of a DevOps workflow for software development including testing and publishing them to a repository.

Here, not too much has happened over the last quarter but the current idea is to incorporate additional features in the medium to longer term to modularise and simplify the image build definitions and then utilise Potman in the Potluck library build process.

As always, feedback and patches are welcome.


WireGuard on FreeBSD Stabilizes, Eyes Upstreaming

Contact: Jason A. Donenfeld <Jason@zx2c4.com>

WireGuard is a secure tunneling protocol that lives in the kernel.

For the last quarter, the wireguard-freebsd codebase has been quite stable and complete. For a while, there were rapid-fire releases fixing issues, and a lot of effort was made to track down every bug report on bugs.freebsd.org, IRC, the mailing list, or elsewhere. But by now, the reports have dried up, and mostly users come to IRC with questions on usage and integration, the usual types of things associated with a stabler project. We also have automated CI now for each commit, compiling and running a small smoke test on wireguard-freebsd’s supported releases — 12.1, 12.2, and 13.0. At some point, hopefully that small smoke test will expand to include the larger battery of tests from Linux.

The wireguard-freebsd repository has been geared around being an out-of-tree kmod, which is distributed in ports. But it is also organized to be eventually upstreamed. To that end, the repository maintains two files: compat.h and support.h. compat.h contains polyfills of code that exists in FreeBSD’s main branch but does not exist in various older releases, with ifdefs for each of the various releases we support. On the other hand, support.h contains code that is not yet in FreeBSD’s main branch. The goal is to eventually move all the code from support.h into compat.h, at which point, the repository will be ready for upstreaming. As of writing, there’s basically only one real function left — sogetsockaddr — and then two convenience macros that need to be sent upstream for consideration by ConcurrencyKit.

A significant aspect that isn’t in support.h, though, is the cryptographic primitives that the code uses. The files crypto.c and crypto.h contain boring C "reference implementations" of ChaCha20Poly1305, XChaCha20Poly1305, Blake2s, and X25519 (which is formally verified via MIT’s fiat-crypto project). These four algorithms are used by the handshake path on very small inputs for WireGuard’s key exchange, and will hopefully be making their way to sys/crypto/ in the FreeBSD tree as just ordinary functions. On the flip side, the datapath uses an entry point of ChaCha20Poly1305 that works on mbufs (which might be rather large) and is performance critical. To that end, jhb@ has been improving OCF for WireGuard’s particulars. The next step will then be moving our current calls from chacha20poly1305_{en,de}crypt_mbuf in wg_noise.c to instead call out to OCF, submitting crypto_buffer`s of type `CRYPTO_BUF_MBUF. This will automatically benefit from Andy Polyakov’s optimized ChaPoly implementations that OCF has long since imported from OpenSSL.

When we make the move to OCF, it’s likely that the wireguard-freebsd repo as-is will become "wireguard-freebsd-compat", which will be explicitly aimed at backports to earlier FreeBSD releases for ports, while a new wireguard-freebsd repository will be a whole FreeBSD tree, where we can work directly on integration patches for upstream. That repository will also have an imported version of the wg(8) utility from wireguard-tools, which I’ll be relicensing as MIT.

I’m quite excited for the upcoming quarter and seeing how much of wireguard-freebsd we’re able to land upstream.