FreeBSD The Power to Serve

FreeBSD Status Report First Quarter 2023

Here is the first status report of 2023, including 25 reports: we have our usual team reports, some news about cloud projects, progress in the src, ports and doc trees and more.

We also provide some information about 13.2-RELEASE, which was postponed to the beginning of 2023Q2; but since this report is being published after the new version release, it is already available for installation. Users of RELEASE versions can now take advantage of many improvements such as better support for the iwlwifi(4) driver or the new rtw88(4) driver, topics that have been covered in past status reports.

Have a nice read.

Lorenzo Salvadore, on behalf of the status team.

FreeBSD Team Reports

Entries from the various official and semi-official teams, as found in the Administration Page.

FreeBSD Core Team

Contact: FreeBSD Core Team <core@FreeBSD.org>

The FreeBSD Core Team is the governing body of FreeBSD.

Items

Core Team Charter: Draft

At the first Core Team meeting of 2023, Team delegates from the December 2022 meeting in Boulder, US presented the delegation’s conclusions to the entire Team. The Team will continue to discuss the issues and work together with the FreeBSD Foundation.

FreeBSD annual developers survey

The Core Team together with the FreeBSD Foundation have decided that the FreeBSD Foundation will be in charge of conducting the annual developers survey.

Matrix IM solution

The Core Team continues to evaluate Matrix as an IM solution for FreeBSD developers. An instance has already been prepared and tests are underway.

Commit bits

  • Core approved the src commit bit for Cheng Cui (cc@)

  • Core approved the restore of the src commit bit for Joseph Koshy (jkoshy@).

FreeBSD Foundation

Links:
FreeBSD Foundation URL: https://www.freebsdfoundation.org
Technology Roadmap URL: https://freebsdfoundation.org/blog/technology-roadmap/
Donate URL: https://www.freebsdfoundation.org/donate/
Foundation Partnership Program URL: https://freebsdfoundation.org/our-donors/freebsd-foundation-partnership-program/
FreeBSD Journal URL: https://www.freebsdfoundation.org/journal/
Foundation News and Events URL: https://www.freebsdfoundation.org/news-and-events/

Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Donations from individuals and corporations are used to fund and manage software development projects, conferences, and developer summits. We also provide travel grants to FreeBSD contributors, purchase and support hardware to improve and maintain FreeBSD infrastructure, and provide resources to improve security, quality assurance, and release engineering efforts. We publish marketing material to promote, educate, and advocate for the FreeBSD Project, facilitate collaboration between commercial vendors and FreeBSD developers, and finally, represent the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

Fundraising Efforts

We finally have our 2022 fundraising numbers in and we raised a total of $1,231,096! We were short of our goal, which forced us to pull around $74,000 from our longer term investments.

Besides receiving a lot of donations from you our users and contributors, we received larger donations from Juniper, Meta, Arm, Netflix, Beckhoff, Tarsnap, Modirum, Koum Family Foundation, and Stormshield. I’d like to extend a heartfelt thank you on behalf of the Foundation to everyone, including individuals and corporations, for your financial contributions in 2022!

This year our budget is around $2,230,000, which includes increased spending towards FreeBSD advocacy and software development. More than half our budget is allocated towards work directly related to improving FreeBSD and keeping it secure. To fund the 2023 budget, we increased our fundraising goal and plan on using some of our investment money. When we received our first million dollar donation, the plan was to use up to 10% of it each year to increase our work to improve FreeBSD, so this has been part of our funding plan for a few years now.

The 2023 budget is in the process of being approved by the board of directors and will be published once it is approved.

This quarter we received donations from Juniper, Tarsnap, Microsoft, and Stormshield. So, we are already off to a great start! But, we definitely need more to support our planned efforts for 2023.

If you want to help us continue our efforts, please consider making a donation towards our 2023 fundraising campaign! https://www.freebsdfoundation.org/donate/

We also have a Partnership Program for larger commercial donors. You can read about it at https://freebsdfoundation.org/our-donors/freebsd-foundation-partnership-program/.

OS Improvements

During the first quarter of 2023, 226 src, 39 ports, and 12 doc tree commits identified the Foundation as a sponsor. Some of this sponsored work is described in separate report entries:

  • Continuous Integration

  • Enabling Snapshots on Filesystems Using Journaled Soft Updates

  • FreeBSD as a Tier 1 cloud-init Platform

  • FreeBSD Release Engineering Team

  • Improve the kinst DTrace provider

  • OpenStack on FreeBSD

Other Foundation-sponsored work included:

  • OpenSSH fixes and updates to versions 9.2p1 and 9.3p1

  • a vendor import and update of libpcap to version 1.10.3

  • improvements to tmpfs, msdosfs, and makefs

  • the addition of a new kqueue1 syscall

  • man page updates

  • dtrace and bhyve fixes

  • LinuxKPI work

Continuous Integration and Quality Assurance

The Foundation provides a full-time staff member and funds projects to improve continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project. You can read more about CI work in a dedicated report entry. A current project that is being funded by the FreeBSD Foundation is one to develop a set of scripts to help src developers conduct CI tests themselves. One of the main goals is to offer more visibility at the pre-commit stage. A review for the first milestone has been submitted.

FreeBSD Advocacy and Education

Much of our effort is dedicated to the FreeBSD Project advocacy. This may involve highlighting interesting FreeBSD work, producing literature and video tutorials, attending events, or giving presentations. The goal of the literature we produce is to teach people FreeBSD basics and help make their path to adoption or contribution easier. Other than attending and presenting at events, we encourage and help community members run their own FreeBSD events, give presentations, or staff FreeBSD tables.

The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, working together on projects, and facilitating collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. We are back to attending events mostly in person and began planning the in person May 2023 Developer Summit, co-located with BSDCan. In addition to attending and planning events, we are continually working on new training initiatives and updating our selection of how-to guides to facilitate getting more folks to try out FreeBSD.

Check out some of our advocacy and education work:

We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.freebsdfoundation.org/journal/.

You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/.

Legal/FreeBSD IP

The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

Go to https://www.freebsdfoundation.org to find more about how we support FreeBSD and how we can help you!

FreeBSD Release Engineering Team

Links:
FreeBSD 13.2-RELEASE schedule URL: https://www.freebsd.org/releases/13.2R/schedule/
FreeBSD 14.0-RELEASE schedule URL: https://www.freebsd.org/releases/14.0R/schedule/
FreeBSD releases URL: https://download.freebsd.org/releases/ISO-IMAGES/
FreeBSD development snapshots URL: https://download.freebsd.org/snapshots/ISO-IMAGES/

Contact: FreeBSD Release Engineering Team, <re@FreeBSD.org>

The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.

During the first quarter of 2023, the Release Engineering Team started work on the upcoming 13.2-RELEASE. As of this writing, the 13.2 cycle has followed the originally set schedule, with the addition of fourth, fifth and sixth RC builds, postponing the final release from the end of March to early April.

The Release Engineering Team continued providing weekly development snapshot builds for the main, stable/13, and stable/12 branches.

Sponsor: Rubicon Communications, LLC ("Netgate")
Sponsor: Tarsnap
Sponsor: The FreeBSD Foundation

Cluster Administration Team

Links:
Cluster Administration Team members URL: https://www.freebsd.org/administration/#t-clusteradm

Contact: Cluster Administration Team <clusteradm@FreeBSD.org>

FreeBSD Cluster Administration Team members are responsible for managing the machines the Project relies on to synchronise its distributed work and communications.

In this quarter, the team has worked on the following:

  • Regular support for FreeBSD.org user accounts.

  • Regular disk and parts support (and replacement) for all physical hosts and mirrors.

  • Improve the PowerPC package builders.

    • With new parts obtained through the FreeBSD Foundation, the builders now have new NVMEs with heatsinks and more memory. It helped to solve the heat issue, and they are building packages faster now.

  • Decouple dynamic resources from the main websites.

    • Work in coordination with doceng and webmaster to decouple dynamic resources from the websites, www.FreeBSD.org, and docs.FreeBSD.org.

Work in progress

  • Large-scale network upgrade at our primary site.

    • New Juniper switches arrived at our primary site to replace the former ones. We thank Juniper for the donation.

  • Replace old servers in our primary site and a few mirrors.

    • Besides the broken CI servers, we have a few old servers with broken disks and faulty PSUs. This task is in conjunction with the FreeBSD Foundation and donors/sponsors.

  • Deploy infrastructure to mirror the websites.

    • Since the FreeBSD website is now mostly static, we have begun deploying infrastructure to mirror www.FreeBSD.org and docs.FreeBSD.org around the world in FreeBSD project-managed mirrors.

  • Create a new search database engine for internal FreeBSD.org searching needs like mailing list and docs.

FreeBSD Official Mirrors Overview

Current locations are Australia, Brazil, Germany, Japan (two full mirror sites), Malaysia, South Africa, Taiwan, United Kingdom (full mirror site), United States of America — California, New Jersey (primary site), and Washington.

The hardware and network connection have been generously provided by:

The Frankfurt single server mirror is the primary Europe mirror in bandwidth and usage.

We are still looking for an additional full mirror site (five servers) in Europe to replace old servers in the United Kingdom full mirror site.

We see a good pattern in having single mirrors in Internet Exchange Points worldwide (Australia, Brazil, and South Africa); if you know or work for some of them that could sponsor a single mirror server, please get in touch. United States (West Coast) and Europe (anywhere) are preferable places.

See generic mirrored layout for full mirror site specs and tiny-mirror for a single mirror site.

Continuous Integration

Links:
FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org
FreeBSD Jenkins wiki URL: https://wiki.FreeBSD.org/Jenkins
Hosted CI wiki URL: https://wiki.FreeBSD.org/HostedCI
3rd Party Software CI URL: https://wiki.FreeBSD.org/3rdPartySoftwareCI
Tickets related to freebsd-testing@ URL: https://bugs.freebsd.org/bugzilla/buglist.cgi?bug_status=open&email1=testing%40FreeBSD.org&emailassigned_to1=1&emailcc1=1&emailtype1=equals
FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci
dev-ci Mailing List URL: https://lists.FreeBSD.org/subscription/dev-ci

Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet

In the first quarter of 2023, we worked with the project contributors and developers to address their testing requirements. Concurrently, we collaborated with external projects and companies to enhance their products by testing more on FreeBSD.

Important completed tasks:

Work in progress tasks:

  • Designing and implementing pre-commit CI building and testing (to support the workflow working group)

  • Designing and implementing use of CI cluster to build release artifacts as release engineering does

  • Simplifying CI/test environment setting up for contributors and developers

  • Setting up the CI stage environment and putting the experimental jobs on it

  • Organizing the scripts in freebsd-ci repository to prepare for merging to src repository

  • Improving the hardware test lab and adding more hardware for testing

  • Merge https://reviews.freebsd.org/D38815

  • Merge https://reviews.freebsd.org/D36257

Open or queued tasks:

  • Collecting and sorting CI tasks and ideas

  • Setting up public network access for the VM guest running tests

  • Implementing use of bare-metal hardware to run test suites

  • Adding drm ports building tests against -CURRENT

  • Planning to run ztest tests

  • Helping more software get FreeBSD support in its CI pipeline (Wiki pages: 3rdPartySoftwareCI, HostedCI)

  • Working with hosted CI providers to have better FreeBSD support

Please see freebsd-testing@ related tickets for more WIP information, and don’t hesitate to join the effort!

Sponsor: The FreeBSD Foundation

Ports Collection

Links:
About FreeBSD Ports URL: https://www.FreeBSD.org/ports/
Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributing/#ports-contributing
FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/
Ports Management Team URL: https://www.freebsd.org/portmgr/
Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/

Contact: René Ladan <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages (through its subsidiary pkgmgr), and personnel matters. Below is what happened in this quarter.

Currently we have around 33,500 ports in the tree. For these ports, there are 3,021 open problem reports, of which 764 are unassigned. The first three months of this year saw 9,021 commits by 163 committers for the main branch and 701 commits by 55 committers for the 2023Q1 branch. Compared to 2022Q4, this means a slight increase in the number of ports, port PRs, ports commits, and active port committers.

During this quarter, we welcomed Robert Clausecker (fuz@), Vladimir Druzenko (vvd@), Robert Nagy (rnagy@), welcomed back Norikatsu Shigemura (nork@), and said goodbye to Marius Strobl (marius@). Portgmr added Muhammad Moinur Rahman (bofh@) as a new member after a successful lurkership.

During the bi-weekly portmgr meetings, the following topics were discussed:

  • improving the situation of binary packages for kernel modules

  • ways to measure the impact of ports on their dependencies and how to maintain high-impact ports.

During this quarter, 32 exp-runs were run to test port updates, updating default versions (LLVM to 15, MySQL to 8.0, Ruby to 3.1), and updating byacc in base. Furthermore, the default version of Go switched to 1.20 and that of Lazarus to 2.2.6.

Four new USES were introduced:

  • budgie to support ports related to the Budgie Desktop

  • ldap to provide support for OpenLDAP, with a new default version of 26 (i.e. 2.6)

  • nextcloud to support Nextcloud applications

  • ruby to provide support for Ruby ports (formerly bsd.ruby.mk).

Status Team

Links:
FreeBSD status reports URL: https://www.freebsd.org/status/
FreeBSD Status Report Process URL: https://docs.freebsd.org/en/articles/freebsd-status-report-process/
Archived status reports GitHub repository URL: https://github.com/freebsd/freebsd-quarterly

Contact: <status@FreeBSD.org>

The new workflow has started

In the first quarter of this year, the status team has started implementing the new workflow that was announced at the end of 2022. Here are some details.

New email addresses

Last quarter we have announced the creation of new email addresses:

Unfortunately, the mailing list does not work as expected at the moment. The issue has been reported but no solution could be found yet. However, a work around allowed to send the second and the last reminder to the list.

Automation

Some automation has been introduced to ensure that no report submission is lost:

  • on Phabricator a herald rule automatically blocks any review touching the status reports directory: even if a report submitter forgets to add the status team as reviewer, salvadore@ (member of the status team) will block the patch anyway. The same rule will also block any review that includes the status team as reviewer, to ensure that at least one member of status has reviewed the patch before commit.

  • a GitHub action automatically adds the newly introduced status report label to any pull request touching the status reports directory. The GitHub action should be easily modifiable by anyone wanting to apply more labels automatically depending on the path of the modified files.

More automation is planned.

Documentation reorganization

The status report README and How To have been updated and merged in one unique document: the FreeBSD status report process. You can check it out to read details about reports submission and publication. It will be updated regularly as the status team proceeds with the implementation of its new workflow. In particular, new material about automation is coming soon.

Userland

Changes affecting the base system and programs in it.

daemon(8) improvements

Links:
daemon(8) URL: https://man.freebsd.org/daemon/8
Libera IRC URL: https://libera.chat/

Contact: Ihor Antonov <ihor@antonovs.family>
Contact: Kyle Evans <kevans@FreeBSD.org>

An ongoing effort to improve code quality and supervision capabilities of the daemon utility. Daemon is a tool that can daemonize (send to background) or supervise any running process, automatically restarting it if it crashes. Daemon is widely used in the ports tree and can be used more in base.

This quarter long_opts support was added and the codebase went through an initial refactoring phase to prepare it for further changes. There are no functional changes so far but more changes are coming. Please contact directly or on #freebsd-dev on Libera IRC if you encounter unexpected bugs.

Planned work items for the next quarter:

We are looking for feedback, bug reports (old and new) and feature requests.

Kernel

Updates to kernel subsystems/features, driver support, filesystems, and more.

Enabling Snapshots on Filesystems Using Journaled Soft Updates in 13.2

Contact: Marshall Kirk McKusick <mckusick@freebsd.org>

The ability to make UFS/FFS filesystem snapshots when running with journaled soft updates, and using them for doing background dumps on a live filesystem, was merged to releng/13.2 during the first quarter of 2023, and lands in FreeBSD 13.2-RELEASE.

Background dumps are requested by using the -L flag to dump(8).

The details of this project were described in the 2022 fourth quarter report.

Sponsored by: The FreeBSD Foundation

Improve the kinst DTrace provider

Links:
libdtrace: implement inline function tracing URL: https://reviews.freebsd.org/D38825
dtrace(1): add -d flag to dump D script post-dt_sugar URL: https://reviews.freebsd.org/D38732

Contact: Christos Margiolis <christos@FreeBSD.org>
Contact: Mark Johnston <markj@FreeBSD.org>

kinst is a new DTrace provider created by christos@ and markj@ that allows for arbitrary instruction tracing in a kernel function. kinst has been added to the base system in FreeBSD 14.0.

The 2022Q3 status report gives a brief introduction to kinst. We’re now working on inline function tracing (see review D38825 above) — a much-requested DTrace feature — by using kernel DWARF and ELF info to find the call sites of each inline copy and use that information to transform D syntax by turning kinst probes of the form:

   kinst::<inline_func>:<entry/return>
        /<pred>/
        {
                <acts>
        }

To:

   kinst::<caller_func1>:<offset>,
        kinst::<caller_func2>:<offset>,
        kinst::<caller_func3>:<offset>
        /<pred>/
        {
                <acts>
        }

For example:

   # dtrace -dn 'kinst::cam_iosched_has_more_trim:entry { printf("\t%d\t%s", pid, execname); }'
        kinst::cam_iosched_get_trim:13,
        kinst::cam_iosched_next_bio:13,
        kinst::cam_iosched_schedule:40
        {
                printf("\t%d\t%s", pid, execname);
        }

        dtrace: description 'kinst::cam_iosched_has_more_trim:entry ' matched 3 probes
        CPU     ID                    FUNCTION:NAME
          2  79315          cam_iosched_next_bio:13     0       kernel
          2  79316          cam_iosched_schedule:40     0       kernel
          0  79316          cam_iosched_schedule:40     12      intr
          2  79315          cam_iosched_next_bio:13     0       kernel
          2  79316          cam_iosched_schedule:40     0       kernel
          0  79316          cam_iosched_schedule:40     12      intr
        ^C

A new -d flag has also been added to dtrace(1) which dumps the D script after libdtrace has applied syntactic transformations.

Further goals include:

  • Implement a locals structure in D which stores the local variables of the traced function. For example with kinst::foo:<x>, we could print the local variable bar by doing print(locals→bar) inside a D script.

  • Port kinst to riscv and/or arm64.

Sponsor: The FreeBSD Foundation

Native Linux timerfd

Links:
Differential revision URL: https://reviews.freebsd.org/D38459

Contact: Jake Freeland <jfree@FreeBSD.org>

The timerfd facility is a set of Linux-standard system calls that operate on interval timers. These timers are analogous to per-process timers but are represented by a file descriptor, rather than a process. These file descriptors may be passed to other processes, are preserved across fork(2), and may be monitored via kevent(2), poll(2), or select(2).

A timerfd implementation in FreeBSD already exists for Linux compatibility, but this differential revision makes the interface native. The goal behind this change is to ease the FreeBSD porting process for programs that include timerfd.

This specific implementation avoids adding new names to the system call table. Instead, timerfd_create() is wrapped by the specialfd() system call. The timerfd_gettime() and `timerfd_settime() calls are wrapped ioctl() s.

Developers that wish to support FreeBSD should avoid using timerfd. The kqueue() EVFILT_TIMER filter is preferred for establishing arbitrary timers.

Architectures

Updating platform-specific features and bringing in support for new hardware platforms.

Kernel Address Sanitizer on AArch64

Contact: Kyle Evans <kevans@FreeBSD.org>

Sanitizers are bug detection facilities which use a combination of instrumentation inserted by the compiler (LLVM in this case) and runtime state tracking to detect bugs in C code. They can automatically detect many types of C programming bugs, such as use-after-frees and uses of uninitialized variables, which may otherwise require substantial effort to identify. They are particularly effective in combination with regression testing suites or fuzzing tools such as syzkaller. Unlike tools such as Valgrind, software must be recompiled to enable a given sanitizer, but sanitizers can be used in the kernel. Kernels with sanitizers enabled incur a significant performance overhead from the runtime, in both CPU utilization and memory usage.

As of 89c52f9d59fa, the kernel address sanitizer that was previously exclusive to amd64 is ported to arm64.

Prior testing has been done on a decent variety of machines, including:

  • Various Ampere Altra machines

  • QEMU

  • Microsoft’s "Volterra" Devkit

  • bhyve (WIP).

Further testing on other hardware would be both welcomed and appreciated.

Sponsor: Juniper Networks, Inc.
Sponsor: Klara, Inc.

bsd-user: Upstreaming and Status Report

Links:
QEMU Project URL: https://qemu.org
FreeBSD bsd-user qemu fork URL: https://github.com/qemu-bsd-user/qemu-bsd-user
QEMU Project’s gitlab mirror URL: https://gitlab.com/qemu-project/qemu

Contact: Warner Losh <imp@FreeBSD.org>

In this quarter, Warner upstreamed two patch sets in the qemu-project repo (with a third pending). Doug Rabson submitted some optimizations to save a handle to the qemu-user emulator in the kernel for future exec. Contact has been made with some folks interested in getting bsd-user working on NetBSD. Summer of Code project to upstream shows some interest.

Upstreaming Efforts

The sysctl system call was upstreamed this quarter. Doug’s changes were also upstreamed (see below). Some cleanups around NetBSD and OpenBSD and to generate syscalls on the fly are pending.

Doug Rabson’s Changes

As part of his container work, Doug submitted changes that allows the kernel to cache the emulator used to run programs. This allows the kernel to directly exec the new binary with that cached emulator. This simplifies bsd-user and removes one source of difference between it and linux-user. Doug also provided an important fix that prevented aarch64 from running.

Bug Fixes and Improvements

In addition to Doug’s fixes, Warner cleaned up things a bit this quarter.

  • Warner removed the final bits of 'run on any BSD code' that was present in the emulator.

  • While the basic system calls could be emulated between all the BSDs, their system call interface has diverged too much, and it is too feature rich for this to be feasible any time soon.

  • Warner had planned to just remove the NetBSD and OpenBSD bits, but there is some interest from at least the NetBSD folks for making things build.

  • Now that the NetBSD folks have contact information, and know direction, Warner hopes they will submit a pull request to build bsd-user on NetBSD for NetBSD.

  • Warner added SIGSYS support so that we can catch unimplemented system calls sooner, and improved reporting of them to get more data about what fails.

  • Warner cleaned up some code in the blitz branch.

  • We’re merged up to 8.0rc1 in upstream in the blitz branch we’re using to stay current.

Summer of Code Projects

There’s much interest in the bsd-user upstreaming task Warner added to Qemu’s project list. With luck, we’ll have a student to fund to do the job of upstreaming all the system calls needed to run simple programs. With a lot of luck, we’ll be able to run any program that does the same thing(s) that clang does (one goal is to have it compile hello world). Future quarterly reports will provide details, should we be fortunate enough to get a slot for this.

Help Needed

We can always use help with bsd-user.

  • Pull requests for new system calls are welcome.

  • Automation in generating many of the things we do by hand would be helpful (like system call argument tracing).

  • Enthusiastic volunteers who want to help me with the upstreaming (many tasks are easy and quick if you don’t want to commit).

  • Coordination with the NetBSD folks and cleanup they come up with.

  • Bug fixes (especially thread bugs) are needed.

Cloud

Updating cloud-specific features and bringing in support for new cloud platforms.

FreeBSD as a Tier 1 cloud-init Platform

Links:
cloud-init Website URL: https://cloud-init.io/
cloud-init Documentation URL: https://cloudinit.readthedocs.io/en/latest/
cloud-init ongoing refactorization URL: https://github.com/canonical/cloud-init/blob/main/WIP-ONGOING-REFACTORIZATION.rst

Contact: Mina Galić <freebsd@igalic.co>

cloud-init is the standard way of provisioning servers in the cloud. Unfortunately, cloud-init support for operating systems other than Linux is rather poor, and the lack of cloud-init support on FreeBSD is a hindrance to cloud providers who want to offer FreeBSD as a Tier 1 platform. To remedy the situation, this project aims to bring FreeBSD cloud-init support on par with Linux support. The broader plan is to lift support across all BSDs.

This quarter has been going very, very slowly, for personal reasons — also for lack of access to the right resources. I have been trying to port the Infiniband functions. This has proven difficult, because it falsified my thesis that ifconfig(8) is all that is needed to figure out network interfaces on FreeBSD.

While waiting for resources, I debugged a boot panic and got it fixed: 499171a98c88. This now makes it possible to boot FreeBSD on LXD — cloud-init’s CI platform. We still need to fix the high CPU usage problem, but there is already an accepted review: D38898

A cloud-init colleague who works for Azure managed to give me access to an HPC VM on Azure. Unfortunately, it was only for a limited time, and that was not enough to figure out how to get Infiniband up and running on FreeBSD — a task handled by Azure Agent on Linux, but FreeBSD’s sysutils/azure-agent is rather lacking.

People interested in helping with this project could provide ifconfig(8), ibstat(8), ibv_devinfo(1), etc… pastes from their Infiniband systems. I would also be very happy about getting access to hardware with Infiniband NICs, or hearing from people who have successfully used FreeBSD on Azure HPC with Infiniband.

If there is interest in that platform, I will direct some energy to fixing Azure Agent.

Sponsor: The FreeBSD Foundation

OpenStack on FreeBSD

Links:
OpenStack URL: https://www.openstack.org/
OpenStack on FreeBSD URL: https://github.com/openstack-on-freebsd

Contact: Chih-Hsin Chang <starbops@hey.com>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>

This project aims to port key OpenStack components so that FreeBSD can function as an OpenStack host.

In 2023 Q1, the big news is that we’re able to spawn FreeBSD instances with bhyve(8) on the OpenStack platform. But there are still some major limitations regarding the capabilities of the spawned instances that need to be resolved:

  • No self-service networks (only support the flat network)

  • No network connectivity inside the instance

  • Only support FreeBSD raw images (FreeBSD-13.1-RELEASE-amd64.raw tested)

  • No disk resize

  • No console integration (need to use cu(1) command manually)

The step-by-step documents for constructing a POC site can be found in the docs repository. The patched version of each OpenStack component is under the same GitHub organization.

Also, we attended AsiaBSDCon 2023 at the end of March and gave a short talk about the current project status at the developer summit. We got precious feedback at the event and will focus on the following for the next quarter:

  • Resolve the Open vSwitch networking issue

  • Convert each OpenStack component into FreeBSD ports

People interested in helping with the project can first help check the documentation by following the installation guide. And here is an open task for the project:

  • FreeBSD-specific implementation for the oslo.privsep library

Feedback and help are always welcome.

Sponsor: The FreeBSD Foundation

Documentation

Noteworthy changes in the documentation tree, manual pages, or new external books/documents.

Documentation Engineering Team

Links:
FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/
FreeBSD Documentation Project Primer for New Contributors URL: https://docs.freebsd.org/en/books/fdp-primer/
Documentation Engineering Team URL: https://www.freebsd.org/administration/#t-doceng

Contact: FreeBSD Doceng Team <doceng@FreeBSD.org>

The doceng@ team is a body to handle some of the meta-project issues associated with the FreeBSD Documentation Project; for more information, see FreeBSD Doceng Team Charter.

During the last quarter:

  • The doc commit bit for Pau Amma was taken in.

  • Lorenzo Salvadore has been proposed as doc committer. carlavilla@ and dbaio@ will mentor him.

  • Ryusuke SUZUKI steps down from doceng. doceng would like to thank ryusuke@ for his service.

Items pending and in the discussion:

Porter’s Handbook:

Three new Uses knobs have been added to the Handbook:

Also:

FreeBSD Translations on Weblate

Link: Translate FreeBSD on Weblate
Link: FreeBSD Weblate Instance

Q4 2022 Status

  • 12 languages

  • 150 registered users

Languages

  • Chinese (Simplified) (zh-cn) (progress: 14%)

  • Chinese (Traditional) (zh-tw) (progress: 11%)

  • Dutch (nl) (progress: 1%)

  • French (fr) (progress: 1%)

  • German (de) (progress: 1%)

  • Indonesian (id) (progress: 1%)

  • Italian (it) (progress: 10%)

  • Korean (ko) (progress: 11%)

  • Norwegian (nb-no) (progress: 1%)

  • Persian (fa-ir) (progress: 6%)

  • Portuguese (pt-br) (progress: 29%)

  • Sinhala (si) (progress: 1%)

  • Spanish (es) (progress: 37%)

  • Turkish (tr) (progress: 5%)

We want to thank everyone that contributed, translating or reviewing documents.

And please, help promote this effort on your local user group, we always need more volunteers.

FreeBSD Handbook working group

Contact: Sergio Carlavilla <carlavilla@FreeBSD.org>

Chapters 1 to 6 have been updated. Chapter 7 is work in progress.

FreeBSD Website Revamp - WebApps working group

Contact: Sergio Carlavilla <carlavilla@FreeBSD.org>

Working group in charge of creating the new FreeBSD Documentation Portal and redesigning the FreeBSD main website and its components. FreeBSD developers can follow and join the working group on the FreeBSD Slack channel #wg-www21. The work will be divided into four phases:

  1. Redesign of the Documentation Portal

    Create a new design, responsive and with global search. (Complete)

  2. Redesign of the Manual Pages on web

    Scripts to generate the HTML pages using mandoc. (Complete) Public instance on https://man-dev.FreeBSD.org

  3. Redesign of the Ports page on web

    Ports scripts to create an applications portal. (Work in progress)

  4. Redesign of the FreeBSD main website

    New design, responsive and dark theme. (Work in progress)

The FreeBSD Russian Documentation Project

Links:
FAQ URL: https://docs.freebsd.org/ru/books/faq/
Web URL: https://www.freebsd.org/ru/

Contact: Andrey Zakhvatov <andrey.zakhvatov@gmail.com>

The FreeBSD Russian Documentation Project current goal is to provide up-to-date Russian translations of the most significant parts of FreeBSD documentation (FAQ, Handbook, Web). It is important to support Russian-speaking persons with high-quality official technical materials and increase acceptance of the operating system around the globe. We hope that this activity will receive some support within the Russian-speaking FreeBSD community and lead to an increased number of translated materials.

FAQ translation was updated and synched with the latest original version. There is also a very slight progress with web pages updates.

Check the official translation guide in case you are willing to help. We will appreciate your help with translation of the following materials:

  • Web pages (easy)

  • Handbook sections (only the X11 section is in progress right now)

  • Articles

Ports

Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.

Freshports: SQL Injection Attack and Help Request

Links:
FreshPorts URL: freshports.org
FreshPorts blog URL: https://news.freshports.org/

Contact: Dan Langille <dvl@FreeBSD.org>

FreshPorts and FreshSource have reported upon FreeBSD commits for 20 years. They cover all commits, not just ports.

FreshPorts tracks the commits and extracts data from the port Makefiles to create a database of information useful to both port maintainers and port users.

For example, https://www.freshports.org/security/acme.sh/ shows the history of the security/acme.sh port, back to its creation in May 2017. Also available are dependencies, flavors, configuration options, and available packages. All of this is useful for both users and developers of ports.

SQL Injection Attack

In March, an SQL injection attack was noticed and the website was patched. Notices were sent out via our Twitter account, our status page, and a notice on the top of each page of the website. The immediate attack vector was shutdown and soon patched. Additional preventative patches were added across the website. Everything we know about has been fixed. Users were notified and advised to change their passwords.

Details at:

Help Needed

It has been over 22 years since FreshPorts started. Others must take over eventually. I’d like to start that process now. There are several aspects to FreshPorts:

  • FreeBSD admin (updating the OS and packages)

  • front end code (website - mostly PHP)

  • back end code (commit processing - Perl, Python, shell)

  • database design (PostgreSQL).

The database does not change very often and requires little maintenance compared to the applications and OS. The website pretty much runs itself. From time to time, a change to the FreeBSD ports infrastructure breaks something or requires a modification, but there is rarely any urgency to fix that. This is not a huge time commitment. There is a lot of learning. While not a complex application, FreshPorts is also not trivial.

DRM drivers (i.e. GPU drivers)

Links:
Git repository on GitHub URL: https://github.com/freebsd/drm-kmod

Contact: Emmanuel Vadot <manu@FreeBSD.org>
Contact: Jean-Sébastien Pédron <dumbbell@FreeBSD.org>
Contact: The Graphics team <freebsd-x11@FreeBSD.org>

GPUs are driven by DRM drivers. They are developed specifically for Linux using a permissive license. Our mission is to port those drivers to FreeBSD to make sure modern GPUs are fully supported.

We didn’t publish a report to share our progress for a long time. Therefore this status report entry will cover more than just the last quarter.

Update to Linux 5.15 LTS and Linux 5.16

As of this status report, the graphics/drm-kmod meta port still installs the DRM drivers from Linux 5.10 (released on December 13, 2020) on FreeBSD 13.1 and greater. This version of the driver lacks support for recent GPUs, in particular Intel 12th gen Alder Lake ones. In the past months, we worked to update the DRM drivers to bring support for more modern AMD and Intel GPUs.

The drm-kmod Git repository master branch was first updated to Linux 5.15 (released on October 31, 2021). This is an LTS branch in Linux and we wanted to take advantage of that. Thus at that point, we followed two paths:

  • A 5.15-lts branch was created to backport all bug fixes from Linux 5.15.x patch releases. This work is now available in the drm-515-kmod port.

  • The porting effort from subsequent Linux versions continued. The master branch is now at Linux 5.16 (release on January 9, 2022).

The Intel driver from Linux 5.15 LTS supports 12th gen GPUs (Alder Lake). It looks to work on FreeBSD but we only tested it lightly so far. We still need more of that, that’s why graphics/drm-kmod still installs graphics/drm-510-kmod instead of graphics/drm-515-kmod. At last, FreeBSD should run as a desktop on this GPU generation and several new AMD GPUs, though problems will surely appear through real test and use.

In the process, we updated firmwares to linux-firmware 20230210.

Linux 5.17 and future work

DRM drivers from Linux 5.17 (released on March 20, 2022) were already ported but this work still sits in its own branch.

A couple of issues block further testing and the merge into the master branch:

  • Our current integration with vt(4), the console/terminal driver, is quite far from the DRM drivers expectations which are based on Linux' fbdev KPI. Something changed in both the Intel and AMD drivers, meaning that vt(4) breaks with the 5.17 update.

  • The initial Linux 5.17 release does not contain the fixes backported to Linux 5.15 LTS. It seems quite unstable with the Intel 12th gen GPU mentioned earlier.

To address the issue with our vt(4) integration layer, we started to write a new vt backend specifically to use the fbdev callbacks exposed by the DRM drivers. This backend will be provided with the DRM drivers, not the FreeBSD kernel, to make it easier to maintain as the drivers evolve. This is still a work in progress and locking in particular is tricky to get right.

Regarding the bad support of Intel 12th gen in the 5.17 update, bug fixes backported to Linux 5.17.x patch releases will probably not be ported as part of this work. Instead we will focus on Linux 5.18 (released on May 22, 2022) and following.

KDE on FreeBSD

Links:
KDE FreeBSD URL: https://freebsd.kde.org/
KDE Community FreeBSD URL:https://community.kde.org/FreeBSD

Contact: Adriaan de Groot <kde@FreeBSD.org>

The KDE on FreeBSD project packages CMake, Qt, and software from the KDE Community, for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma (for both X11 and Wayland) and hundreds of applications that can be used on any FreeBSD machine.

The KDE team (kde@) is part of desktop@ and x11@, building the software stack to make FreeBSD beautiful and usable as a daily-driver graphics-based desktop machine. The notes below describe mostly ports for KDE, but also include items that are important for the entire desktop stack.

Infrastructure

  • The Qt5 ports were updated to the KDE patch collection release 5.15.8.

  • The Qt6 ports — these are not used by KDE yet, but there are many ports that can use Qt6 and have Qt6 flavors — were updated to release 6.4.2. Python bindings for the Qt6 release of WebEngine were added.

  • The cmake ports were updated to release 3.25.1 and the CPack generator for FreeBSD packages was repaired.

  • The graphics/poppler port — used by many PDF-viewers — was updated to release 23.01.

  • The sysutils/bsdisks port — used as a shim for applications that expect Linux udisks, which means most desktop environments — was updated to release 0.29.

KDE Stack

KDE Gear releases happen every quarter, KDE Plasma updates once a month, and KDE Frameworks have a new release every month as well. These (large) updates land shortly after their upstream release and are not listed separately.

  • KDE Frameworks updated to 5.104.

  • KDE Gear updated to 22.12.3.

  • KDE Plasma Desktop was updated to version 5.27. This was a long delayed update, due to unresolved issues in the support stack and a misplaced patch from an earlier release of KDE Plasma. Thanks to arrowd@ and Serenity Cybersecurity, LLC for sorting that out.

  • New port devel/ktextaddons was added to the tree. This is part of the KDE PIM suite, and slated to become a new KDE Framework in some future release.

Related Ports

  • audio/amarok, one of the most popular KDE audio players of the early 2000’s, has been marked deprecated in the ports tree. It is no longer maintained upstream.

  • astro/kstars, an interactive planetarium, was updated to release 3.6.3.

  • devel/gitqlient, a graphical user interface for git, was updated to release 1.6.1 with support for new git commands.

  • devel/okteta, a hex viewer and editor for binary files, was updated to release 0.26.10.

  • devel/qcoro, C++ coroutines with Qt support, was updated to release 0.8.0.

  • graphics/krita, an application for painting and graphical work, was updated to release 5.1.5.

  • graphics/quickqanava, a graph visualization library, got a real release and an update in the ports tree.

  • irc/kvirc, an IRC client, was updated to the latest commit; there is no real release but there are bugfixes.

  • multimedia/haruna, a video and audio player, was updated to release 0.10.3.

  • net-im/neochat, one of a handful of Matrix clients, was updated to chase a new release of net-im/libquotient. There are continuing troubles with compatibility with older FreeBSD releases, leading to the KDE-FreeBSD team to declare FreeBSD 12 releases "effectively unsupported".

  • net-im/ruqola, a Rocket Chat client, was updated to release 1.9.1.

  • security/keysmith, a two-factor-authentication support application, was updated to release 23.01.0.

FSX

Links:
GitHub URL: https://github.com/asomers/fsx-rs FreshPorts URL: https://www.freshports.org/devel/fsx/

Contact: Alan Somers <asomers@freebsd.org>

The venerable FSX (File System eXerciser) tool, first written at Apple Computer in the nineties, has been a part of FreeBSD since 5.0. It stress tests file systems with a stream of randomly generated operations, verifying file data after every read. However, it has never been installed as part of the OS; it only exists in the source tree. That makes it difficult to use in CI pipelines. It has some other limitations, too.

So this quarter I rewrote the entire tool in Rust. The rewrite is byte-for-byte compatible with the original, given identical seed values. Future versions will break backwards-compatibility, however, in order to add new features like fspacectl and copy_file_range. The new version can be found in the ports tree, and in time I’ll remove the original.

GCC on FreeBSD

Links:
GCC Project URL: https://gcc.gnu.org
GCC 11 release series URL: https://gcc.gnu.org/gcc-11/
GCC 12 release series URL: https://gcc.gnu.org/gcc-12/

Contact: Lorenzo Salvadore <salvadore@FreeBSD.org>
Contact: Gerald Pfeifer <gerald@pfeifer.com>

The main news this quarter is the cleaning of old GCC versions from the ports tree: this will allow for a more efficient approach to bugs.

Deprecation of old GCC ports

The ports tree still contains several ports related to old and unsupported GCC versions. They are usually needed as dependencies for a few old ports, that it would be better to either update to use a supported GCC release, or deprecate. Bug reports have been created to track the issue and work has already started towards its resolution. Thanks to all ports contributors who are helping.

Deprecation of USE_GCC=X+

Gerald, who maintained the GCC ports for many years until recently, still contributes to the GCC maintenance on FreeBSD by helping simplify the GCC infrastructure in the ports tree, for example by removing special cases that deal with old unsupported GCC versions.

This quarter the most significant of his changes is probably the removal of support for the USE_GCC=X+ construct: any port depending on GCC should set USE_GCC=yes if GCC_DEFAULT works; if not, it should require a specific version (e.g. USE_GCC=11); it cannot ask for a minimal version anymore (e.g. USE_GCC=11+).

Valgrind - Preparing for Valgrind 3.21

Links:
Valgrind Home Page URL: https://www.valgrind.org/
Valgrind News URL: https://www.valgrind.org/docs/manual/dist.news.html

Contact: Paul Floyd <pjfloyd@wanadoo.fr>

The devel/valgrind-devel port had an intermediate update which was submitted on 2023-02-20. This contains most of what will be in the official release of Valgrind 3.21 which is due out shortly after this status report.

There is a nice improvement to the vgdb interface. It’s now much easier to see which bits of memory are initialized or not. There are a couple of fixes to the thread checks done by Helgrind.

For FreeBSD specifically, the address space limit has been raised to be the same as Linux and Solaris on amd64. It was 32Gbytes and now it is 128Gbytes. The kern.proc.pathname.PID sysctl(3) has been fixed so that it returns the path of the guest exe and not that of the Valgrind host. At the same time I fixed some _umtx_op false positives and corrected auxv AT_EXECPATH in a way similar to kern.proc.pathname.PID. Syscall wrappers have been added for sctp_generic_sendmsg(2) and sctp_generic_recvmsg(2).

Not yet available in the ports versions of Valgrind, there is a workaround for the use of rfork(2). Previously, since it is not supported, it would cause Valgrind to abort. Now it fails gracefully setting either EINVAL or ENOSYS. The main use of this system call is in posix_spawn(3), which will fall back to using vfork(2).

The mknodat(2) syscall wrapper was incorrectly implemented on i386 and has now been fixed.

There is a reworking of all of the aligned allocation functions so that they behave less like Linux glibc and more like the Valgrind build platform.

Third Party Projects

Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.

PkgBase.live

Links:
Website (archive.org) URL: https://web.archive.org/web/20221220222828/https://alpha.pkgbase.live/
Website source URL: https://codeberg.org/pkgbase/website

Contact: Mina Galić <freebsd@igalic.co>

PkgBase.live was an unofficial repository for the FreeBSD PkgBase project. As a service, PkgBase.live was inspired by https://up.bsd.lv/, which provides freebsd-update(8) for STABLE and CURRENT branches.

Hardware for PkgBase was kindly sponsored by a member of the FreeBSD community. However, as life and projects moved on, they had to decommission the hardware, giving me three months' notice. In that time, my own life was rather turbulent after a recent move to a different country so I haven’t been able to find a replacement.

For the time being, PkgBase.live is dead.

The website, and with it the How Did She Do it?! are still available in Git. I highly encourage copy-cats.

I will also happily accept a new hardware sponsor!

Please note that I have contacted the FreeBSD Project, and they are working on integrating PkgBase into release engineering. However, they are not yet ready, they also cannot "simply" take over PkgBase.live because it uses a completely different process.

Containers and FreeBSD: Pot, Potluck and Potman

Links:
Pot organization on GitHub URL: https://github.com/bsdpot

Contact: Luca Pizzamiglio (Pot) <pizzamig@freebsd.org>
Contact: Bretton Vine (Potluck) <bv@honeyguide.eu>
Contact: Michael Gmelin (Potman) <grembo@freebsd.org>

Pot is a jail management tool that also supports orchestration through Nomad.

During the last quarter, pot received a number of minor fixes but no new version has been released yet.

Potluck aims to be to FreeBSD and pot what Dockerhub is to Linux and Docker: a repository of pot flavours and complete container images for usage with pot and in many cases Nomad.

All Potluck images have been rebuilt to include the latest FreeBSD security advisories, a new Smokeping network latency monitoring image has been added, again a lot of work went into the Jitsi image, which unfortunately still seems to have some reliability issues.

Also, two new blog posts are available showing how easy it is to use Potluck images, one explaining how to set up Nextcloud with Minio as object storage and Prometheus for monitoring, one showing how to run your own Matrix Synapse server using OpenLDAP for access management.

As always, feedback and patches are welcome.

Last modified on: April 19, 2023 by Lorenzo Salvadore