This report covers FreeBSD related projects for the period between April and June, and is the second of four planned reports for 2021.
Some of this reports highlights include but are not limited to work on an experimental installer, changes to pf, additional work on the Linuxulator, updates on the state of kernel sanitizers, coverage of the raidz expansion feature for ZFS, and some news about resource accounting.
Daniel Ebdrup Jensen, with status hat on.
- FreeBSD Team Reports
- NXP DPAA driver
- ENA FreeBSD Driver Update
- Graphics Driver Update from Linux 5.7
- Intel Wireless driver support
- Kernel Sanitizers
- Linux compatibility layer update
- NXP LS1028A SoC support
- Marvell ARM64 SoCs support
- Multicast routing rework
- VFS path descriptors API
- Dummynet support for pf
- Ethernet support for pf
- pf syncookie support
- Microchip PolarFire SoC support
- Racct (Resource Accounting) Bug Fixes and Improvements
- OpenZFS RAIDZ Expansion update
- Microchip Switchtec support
- Third-Party Projects
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in the Administration Page.
Contact: Deb Goodkin <deb@FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security, quality assurance, and software engineering efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
COVID-19 Impact to the Foundation
Like most organizations, our team continued to work from home. Our temporary ban on travel for staff members remains in effect, but continues to not affect our output too much, since most conferences are still virtual. As the world continues to open up we will re-evaluate the travel ban. We continued supporting the community and Project through our regular channels.
Partnerships and Commercial User Support
We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. Our temporary travel ban stayed in effect during Q2, so we continued meeting with corporate users virtually. If things look good for in-person meetings in the fall, then we’ll start incorporating those into an in-person and virtual meeting mix.
First, we’d like to say thank you to everyone who has given us a financial contribution this year! Last quarter we raised $70,410, which includes donations from organizations like Verisign, VMware, and Stormshield, as well as many individuals.
We depend on these donations to fund our work supporting FreeBSD. Late last year we decided to put more funding into helping to improve FreeBSD. We hired a Sr. Software Developer to work on arm64 and a Project Coordinator to manage our projects and interface with the community. We also hired two of our part-time software developers full-time. The purpose of this was to provide more resources to step in to implement and improve major features in FreeBSD, review patches and bug reports, implement bug fixes, and support the security efforts. This ensures FreeBSD remains the innovative, secure, and reliable operating system that you rely on.
You’ll find out how we used your donations for Q2 in our report, as well as individual reports throughout this status report.
We are excited about our plans for 2021, which include more FreeBSD online advocacy and training, operating system course content, and the software development work mentioned above. While we are still in this pandemic, we’re working hard to help connect folks within the community with more virtual opportunities.
Please consider making a donation to help us continue and increase our support for FreeBSD in 2021: https://www.freebsdfoundation.org/donate/.
We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information and share with your companies! https://www.freebsdfoundation.org/FreeBSD-foundation-partnership-program/
During the second quarter Foundation staff and grant recipients committed 348 src tree changes, 19 ports tree changes, and 11 doc tree changes. This represents 40% of src commits which identify a sponsor. For ports commits it’s 15%, and 18% of doc commits. Foundation staff and grant recipients also contributed to a number of 3rd party projects. Two notable examples are the LLVM project’s LLDB debugger and the Syzkaller code-coverage-guided system call fuzzer.
You can read about a number of Foundation projects in individual quarterly reports. Smaller projects and improvements include:
Implement on-demand coredump generation by the kernel via
General kernel debugging improvements
Remove obsolete kernel mcount profiling
Nullfs and tmpfs bug fixes
libc cleanup and improvements
rtld dlerror and thread local variable fixes (reported by Julia developers)
kqueue and POSIX timer fixes
UFS bug fixes
Capsicum socket operation improvements
hwpmc (hardware performance profiling) maintenance and CPU support
Cirrus-CI boot smoke test
sndstat(4) schema updates
Virtio 1.0 modern support in bhyve(8)
As usual Foundation staff also supported the project with significant effort on code reviews and general bug triage and fixes. Also, Ka Ho added an article titled Use Language Servers for Development in the FreeBSD Src Tree.
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member and funds projects on improving continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project.
During the second quarter of 2021, work on pre-commit tests and building release artifacts in the CI environment continued. A project using the netperf cluster to do network-related CI jobs is being planned.
See the FreeBSD CI section of this report for completed work items and detailed information.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we supported the test cluster at Sentex, purchased a few needed parts for infrastructure in general, and started working with the clusteradm team on a more efficient and improved hardware request/purchase process.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. While we were still unable to attend in-person meetings due to Covid-19, we were able to attend virtual events and help organize the June 2021 FreeBSD Developer Summit. In addition to attending and planning virtual events, we are continually working on new training initiatives and updating our selection of how-to guides to facilitate getting more folks to try out FreeBSD.
Check out some of the advocacy and education work we did last quarter:
Participated as an Industry Partner for USENIX LISA21
Organized content and promoted FreeBSD Day on June 18-19, 2021
Helped organize and run the June 2021 FreeBSD Developer Summit - videos are now posted on the wiki.
Presented at the The 16th Open Source China Open Source World Summit on June 18
New blog posts on the Linxulator work we funded and What’s new in FreeBSD 13.0
New How To Guide on Git
Continued to promote the FreeBSD Office Hours series Videos from the one hour sessions can be found on the Project’s YouTube Channel. See the Office Hours section of this report for more information.
Committed to be a Silver Sponsor for EuroBSDcon
Keep up to date with our latest work in our newsletters: https://www.freebsdfoundation.org/news-and-events/newsletter/
We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/.
You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/.
The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.
Go to https://www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you!
FreeBSD Release Engineering Team
FreeBSD 13.0-RELEASE schedule URL: https://www.freebsd.org/releases/13.0R/schedule/
FreeBSD 13.0-RELEASE announcement URL: https://www.freebsd.org/releases/13.0R/announce/
FreeBSD releases URL: https://download.freebsd.org/ftp/releases/ISO-IMAGES/
FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/
Contact: FreeBSD Release Engineering Team, <re@FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.
During the second quarter of 2021, the Release Engineering Team completed work on the 13.0-RELEASE cycle, the first release from the stable/13 branch. During the release cycle, one additional BETA build and two additional RC builds were added to the schedule, however the release cycle went smoothly, overall.
Additionally throughout the quarter, several development snapshots builds were released for the main, stable/13, and stable/12 branches. Development snapshot builds for stable/11 will no longer be available.
Thank you to all that have helped test the 13.0 builds up until this point and have reported issues. As always, we strive for quality over quantity.
Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD Foundation
Cluster Administration Team
Contact: Cluster Administration Team <clusteradm@FreeBSD.org>
The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following:
Moved Phabricator (reviews.freebsd.org) to a faster machine
Moved https://www.freebsd.org CGI backend to a faster machine
Improved the network design of our primary cluster site
Removed public IPv4 connectivity from VLANs not hosting public-facing services
Tidied up the pkgbuild and pkgexp VLANs where the production and experimental package builders live.
Moved developer reference machines and universe building machines to a different VLAN to better accommodate aarch64 and ppc64 machines
Upgraded the machines running important internal FreeBSD.org services
DNS, Kerberos, LDAP, LetsEncrypt.org certbot, etc
FTP, Subversion, Git mirror seed
Upgraded the developer reference machines and universe builders to a newer FreeBSD 14-CURRENT
Upgraded package building machines to newer versions of FreeBSD 14-CURRENT
Assisted postmaster with migrating the mailing lists from Mailman to mlmmj
Recycled a particularly troublesome PowerPC64 package building machine with extreme prejudice (rest in pieces)
Installed a new production PowerPC64 package builder
Worked with Git migration working group for ports tree migration
Ongoing day to day cluster management activity
Putting out fires
Work in progress:
Move pkg-master.nyi to new hardware
Improve to the package building infrastructure
Review the service jails and service administrators operation
Setup powerpc pkgbuilder/ref/universal machines
Upgrading public-facing cluster services from 12.2-STABLE to 13.0-STABLE
Installing a new cluster site in Japan
Full mirror site (ftp, pkg, git, doc, dns,…)
The network and machine resources for this mirror are generously sponsored by the Cloud and SDN Laboratory at BroadBand Tower, Inc., one of the Internet data center service providers in Tokyo, Japan, with 300+ Gbps international IP transit bandwidth
Improvements to GeoDNS routing, particularly in Asia
Improve the web service architecture
Improve the cluster backup plan
FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
FreeBSD Hardware Testing Lab URL: https://ci.FreeBSD.org/hwlab
FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org
FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI
FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins
Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI
3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI
Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg
FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci
The FreeBSD CI team maintains the continuous integration system of the FreeBSD project. The CI system firstly checks the committed changes can be successfully built, then performs various tests and analysis over the newly built results. The artifacts from those builds are archived in the artifact server for further testing and debugging needs. The CI team members examine the failing builds and unstable tests and work with the experts in that area to fix the code or adjust test infrastructure. The details of these efforts are available in the weekly CI reports.
During the second quarter of 2021, we continued working with the contributors and developers in the project to fulfil their testing needs and also keep collaborating with external projects and companies to improve their products and FreeBSD.
The build environment of main (-CURRENT) and stable/13 branches are changed to 13.0-RELEASE
Work in progress and open tasks:
Designing and implementing pre-commit CI building and testing
Designing and implementing use of CI cluster to build release artifacts as release engineering does
Collecting and sorting CI tasks and ideas here
Testing and merging pull requests in the FreeBSD-ci repo
Reducing the procedures of CI/test environment setting up for contributors and developers
Setting up the CI stage environment and putting the experimental jobs on it
Setting up public network access for the VM guest running tests
Implementing using bare metal hardware to run test suites
Adding drm ports building tests against -CURRENT
Planning to run ztest and network stack tests
Adding more external toolchain related jobs
Improving the hardware lab to be more mature and adding more hardware
Working with hosted CI providers to have better FreeBSD support
Please see freebsd-testing@ related tickets for more WIP information, and don’t hesitate to join the effort!
Sponsor: The FreeBSD Foundation
About FreeBSD Ports URL:https://www.FreeBSD.org/ports/
Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributing/ports-contributing/
FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/
Ports Management Team URL: https://www.freebsd.org/portmgr/
Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/
The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter.
According to FreshPorts, we currently enjoy a little over 44,200 ports in the Ports Collection. These are accompanied by 2,532 PRs of which 526 are unassigned. During the last quarter, there were 10,428 commits on main by 164 committers and 618 commits on 2021Q2 by 59 committers. Compared to 2021Q1, the number of ports and PRs remained roughly the same and the main tree saw 10% more commits this quarter.
During the last quarter, we welcomed Charlie Li (vishwin@) and Guangyuan Yang (ygy@) and said goodbye to jlaffaye@, jpaetzel@, kmoore@, lifanov@, miwi@ and shurd@.
On the infrastructure side, a new USES, ansible, was added so that Ansible-related ports can more easily define plugins and modules. Several default versions were updated:
Default version of PYTHON and PYTHON3 switched to 3.8
Default version of librsvg2 on PowerPC switched to rust
Regarding the license framework, the badly maintained LEGAL was removed in favor of per-port LICENSEs and the BSD0CLAUSE license was added. In the options framework, pre-defined shared version control OPTIONS and descriptions were added.
Some notable port updates:
Xorg server 1.20.11
Finally, antoine@ ran 18 exp-runs to test updates for cmake, expat2, KDE, meson, poppler, rust, Xorg and the lapack family, to switch the default version of Python to 3.8, and to remove the outdated qtchooser.
Documentation Engineering Team
FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/
FreeBSD Documentation Project Primer for New Contributors URL: https://docs.freebsd.org/en/books/fdp-primer/
Documentation Engineering Team URL: https://www.freebsd.org/administration/#t-doceng
Contact: FreeBSD Doceng Team <doceng@FreeBSD.org>
The doceng@ team is a body to handle some of the meta-project issues associated with the FreeBSD Documentation Project, more infomation on FreeBSD Doceng Team Charter.
During the last quarter, we welcomed back Ceri Davies (ceri@) as a doc committer. Sergio Carlavilla (carlavilla@) and Danilo G. Baio (dbaio@) joined the doceng@ team.
A new article was included in the FreeBSD Documentation by Ka Ho Ng (khng@), Use Language Servers for Development in the FreeBSD Src Tree.
Much work was done regarding the transition to Hugo/Asciidoctor, but there are still pending issues in the FreeBSD Documentation Project after the migration. We thank everyone who is helping to find and fixing them. The TODO items wiki list was refreshed, and everyone is welcome to contribute to it.
Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.
BSDDialog - TUI Widgets
Contact: Alfonso Sabato Siciliano <email@example.com>
The purpose of this project is to provide an utility and a library to build scripts and tools with UI Widgets in a terminal. The project is inspired by Dialog; however BSDDialog is released under the terms of the BSD-2-Clause License while Dialog is licensed as LGPL so a link of the project has been added to the "GPL Software in the Base System" wiki page.
The aim is to provide full compatibility with the dialog utility (the challenge is to implement over seventy options and almost thirty widgets). The API compatibility with the library is not a priority because libbsddialog should meet FreeBSD’s needs, for example it provides the new mixedlist() function, it can take as argument a set of: checklists, radiolists and separators, so it could be useful for building a dialog4ports clone: easier to implement and not depending on non-permissive dependencies.
BSDDialog is currently under development, no widget is really completed (autosizing, resizing and scrolling are missing), nevertheless runnable examples for the utility and the library are inside the example folders and described in the README.
Finally, a curiosity: BSDDialog started from the MixerTUI code base, however the original code has been almost completely rewritten.
Contact: Yang Zhong <firstname.lastname@example.org>
bsdinstall is FreeBSD’s current installer. It is a terminal application with an unwieldy interface, and it asks some very obscure questions that are hard to understand for ordinary users. So, the purpose of this project is to create a graphical installer for FreeBSD that has a more streamlined interface, and to improve other aspects of the FreeBSD installation process.
The experimental installer uses a web front-end: a web server runs locally from the installation media, and the user configures their install by filling out web forms on a browser also running on the installation media. A Web interface is flexible and accessible, so it suits an installer well. This interface can also support remote installs, where the server runs on the target and the install is configured through some other machine, though I have not done much work here.
The installer also aims to have a modular design, where the user configuration options are written to an installation config file, that is then used for the actual installation. While bsdinstall already supports scripted installations, its config file format is very free-form. A more rigid config file design would make it easier to write other installation front-ends in the future.
The installer is currently a rough proof-of-concept, but it can handle a basic installation with limited configuration. Help with testing would be appreciated; you can try the installer by downloading one of the releases in the ISO repository. Also, please email me with any thoughts on the design of the installer, or on useful features it should have.
Sponsor: The FreeBSD Foundation
Git Migration Working Group
Git for FreeBSD development wiki URL: https://wiki.freebsd.org/Git
Git transition wiki URL: https://wiki.freebsd.org/GitTransition
doc git repo web URL: https://cgit.FreeBSD.org/doc
ports git repo web URL: https://cgit.FreeBSD.org/ports
src (base system) git repo web URL: https://cgit.FreeBSD.org/src
Committers guide Git primer URL: https://docs.freebsd.org/en/articles/committers-guide/#git-primer
Handbook Using Git appendix URL: https://docs.freebsd.org/en/books/handbook/mirrors/#git
Game of Trees URL: http://gameoftrees.org/
gitup URL: https://github.com/johnmehr/gitup
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: Warner Losh <imp@FreeBSD.org>
Contact: Ed Maste <emaste@FreeBSD.org>
Contact: Ulrich Spörlein <uqs@FreeBSD.org>
Contact: FreeBSD-git mailing list
Contact IRC #gitcvt channel on EFnet
The Git Working Group has been working on ports repository migration to Git, this task started at the end of the March, beginning with a final Subversion commit on March 31st to indicate that the conversion started. The whole migration completed on April 6th. Since 2021Q2, the ports quarterly branch is created in Git repository only. We continued working on portsnap and other ports infrastructure to accommodate git.
We continued working on implementing and updating commit hooks. The work including helped change FreeBSD 13.0 release process to use Git. And we are sorting and making the infrastructure available to the public, as well as the documents.
On June 8th, we worked with our ZFS developers to have better tracking of upstream OpenZFS development. The vendor/openzfs branch was renamed to vendor/openzfs/legacy. Two new branches were imported directly from upstream, vendor/openzfs/master and vendor/openzfs/zfs-2.1-release, and merged to main and stable/13. The details and the required action to correct the errors might result for the people tracking the old branch is available at https://lists.freebsd.org/archives/freebsd-current/2021-June/000153.html
The Git Working Group continues to track progress on two permissively-licensed git compatible tools: Gitup and Game of Trees. Gitup is a small, dependency-free tool to clone and update git repositories. It is used only to keep a local tree up-to-date, and has no support for local commits.
Game of Trees is a version control client that is compatible with Git repositories. It provides a user interface and workflow that is distinct from that of Git. It is in no way intended to be a drop-in replacement for git, but can be used to develop software maintained in a Git repository.
The core team began a new effort to investigate and evaluate new workflow changes in the June 2021 DevSummit. In the third quarter of 2021 we expect to complete the remaining migration tasks and create a new working group to help with workflow refresh. We’ve wound up our regular meetings, and the remaining migration tasks are being done by individuals (Li-Wen Hsu is mainly working on this). The new working group(s) will have people that participated in this working group as well as new people who will bring new perspectives to the process.
Sponsor: The FreeBSD Foundation (in part)
LLDB Debugger Improvements
Moritz Systems Project Description URL: https://www.moritz.systems/blog/lldb-freebsd-cpu-target-support-and-userland-debugging-improvements/
Progress Report 1 URL: https://www.moritz.systems/blog/freebsd-remote-process-plugin-on-non-x86-architectures/
Progress Report 2 URL: https://www.moritz.systems/blog/freebsd-legacy-process-plugin-removed/
Progress Report 3 URL: https://www.moritz.systems/blog/lldb-support-for-fork-and-vfork/
Progress Report 4 URL: link:https://www.moritz.systems/blog/lldb-core-dump-support-improvements/
Git Repository URL: https://github.com/moritz-systems/llvm-project
Contact: Kamil Rytarowski <email@example.com>
Contact: Michał Górny <firstname.lastname@example.org>
The LLDB project builds on libraries provided by LLVM and Clang to provide a great modern debugger. It uses the Clang ASTs and the expression parser, LLVM JIT, LLVM disassembler, etc so that it provides an experience that “just works”. It is also blazingly fast and more permissively licensed than GDB, the GNU Debugger.
FreeBSD includes LLDB in the base system. At present, it has some limitations in comparison with the GNU GDB debugger, and does not yet provide a complete replacement. This project spanned from January 2021 to April 2021 and aimed to improve the compatibility of LLDB with FreeBSD and extend it with new features.
The initial work was focused on finishing the port of the
FreeBSD process plugin to a new client-server model. After porting
all previously supported architectures we were able to remove the
legacy plugin. Afterwards, we have implemented support for tracing
vfork(2) syscalls using a
model that is compatible with the
setting from GDB. Finally, we have worked on improving support for
core dumps in LLDB. We have used the newly introduced
ptrace(2) request to support
creating a core dump of a stopped program without crashing it.
During our work, we have fixed many bugs and improved the state of
the test suite on amd64 and arm64 architectures.
The introduced changes are expected to be shipped with LLDB 13.0.
The overall experience of FreeBSD/LLDB developers and advanced users on this rock solid Operating System reached the state known from other environments. Furthermore, the FreeBSD-focused work also resulted in generic improvements, enhancing the LLDB support for Linux and NetBSD.
Sponsor: The FreeBSD Foundation
Performance Monitoring Counters
Contact: Mitchell Horne <mhorne@FreeBSD.org>
This quarter, some improvements were made to the
pmc(3) library and
Background: Traditionally, PMC event codes have been defined
manually in the
sys/dev/hwpmc/pmc_events.h header and
compiled statically into the kernel. These definitions provide a
translation between an event’s name and its encoding and are
leveraged by PMC tools like
pmcstat(8) via the
pmc(3) library. In 2018, a new source of truth for
event definitions was introduced for the amd64 and i386
architectures, which is the pmu-events tables. These are a set of
json files, maintained by the Linux kernel, which are compiled
directly into libpmc and provide a more complete set of event
The main result of this work was porting the use of the
pmu-events definitions to arm64. Some minor refactoring to libpmc
was performed, making porting to other platforms in the future
easier. A few small bugs were found and fixed along the way;
notably, the "instructions" alias for
should work again on Intel CPUs. The ability was added to query the
legacy event tables when pmu-events does not contain the desired
event, restoring the ability to use the
events on x86. Finally, event definitions for newer AMD Zen3 CPUs
were imported (credits: Greg V).
An larger update of the PMU event definitions is planned for Q3 of 2021. This will bring us in sync with 3 years of updates for x86 CPUs and greatly increase the number of events available on arm64 platforms.
Those who make regular or occasional use of FreeBSD’s PMC tools are encouraged to test their typical workflow and report any regressions.
Sponsor: The FreeBSD Foundation
syzkaller on FreeBSD
See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller.
Steady progress has been made on fixing bugs found by syzkaller, both by private instances and by the public syzbot instance. Bugs have been fixed in TCP, ktrace(2), kqueue(2), POSIX timers, fork(2), crypto(4), and in the VFS and UFS code. Work has also progressed on adding additional definition to syzkaller, so unfortunately the size of backlog remains more or less steady. Simran Kathpalia, a student participating in this year’s Google Summer Of Code (GSOC) program for FreeBSD, has made excellent progress in extending syzkaller to cover more of the FreeBSD kernel. She has added definitions for a number of system calls and has also been working on automating the generation of such definitions.
Some work has been done towards making it easier to easier to set up syzkaller instances with minimal effort. For example, much of the work can be automated using a BastilleBSD template, which uses a jail to simplify configuration. While this example still requires some manual configuration and is not very flexible, it is much simpler than setting up an instance manually, and helps motivate some planned UI improvements and bug fixes for FreeBSD jails.
Future work includes:
Further effort to try and reduce the size of the syzkaller backlog.
Completion of some proof-of-concept work to extend syzkaller to fuzz the Linuxulator subsystem.
Additional system call and ioctl definitions.
Automated fuzzing with kernel sanitizers enabled.
Sponsor: The FreeBSD Foundation
Updates to kernel subsystems/features, driver support, filesystems, and more.
NXP DPAA driver
Contact: Jakub Klama <email@example.com>
The Conclusive Engineering team is working on support for the NXP Data Path Acceleration Architecture (DPAA) present in the NXP QorIQ Layerscape ARMv8 SoCs as well as PowerPC-based QorIQ SoCs.
This work is focused on providing a clean-room, native implementation of DPAA driver and all of its associated components such as BMan, QMan, FMan and mEMAC for ARM targets such as LS1046A.
First functional version of the driver with support for the 1Gbps and 10Gbps MACs should be completed in the 2021 Q3 timeframe.
ENA FreeBSD Driver Update
ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used.
Completed since the last update:
Update ENA driver to v2.4.0
Update ENA man page
Restructure the logging system
Hide sysctl nodes for unused IO queues
Add support for the large LLQ headers
Update HAL version
Work in progress:
Introduce full kernel RSS API support
Allow reconfiguration of the RSS indirection table and hash key
Prototype the driver port to the iflib framework
MFC the ENA v2.4.0 driver to the FreeBSD 11/12/13-STABLE branches
Support netmap on the c6gn/m6i AWS instance types
Fix race between detach function and some of the procedural sysctl nodes
Add new statistics counters
Sponsor: Amazon.com Inc
Graphics Driver Update from Linux 5.7
We are attempting to update the drm-kmod graphics drivers to the Linux 5.7 code, based on the existing drm-kmod 5.5-wip branch.
Right now, the current version of drm-kmod do not support newer GPU such as Intel’s Tiger Lake/Iris Xe, used on laptops such as the 2020 HP Spectre x360. This prevents us from supporting accelerated graphics on newer hardware containing Intel or AMD GPUs.
Some tasks have already been completed, including:
i915kms console bring-up
Some tasks need to be completed, including:
i915kms Xorg bring-up (currently pagefaults on
amdgpu VA-API bring-up
We welcome help for the incomplete tasks, especially from users wanting to use newer hardware (or support FreeBSD-as-a-desktop in general).
Intel Wireless driver support
The Intel Wireless driver update project aims to bring support for newer chipsets along with mac80211 LinuxKPI compat code. The dual-licensed Intel driver code was ported in the past for the iwm(4) native driver and using the LinuxKPI compat framework allows us to use the driver directly with only very minor modifications. Multiple updates during development in the last year have shown that pulling in newer versions can be done in under 1-2 hours.
After a break, part-time work resumed and the last LinuxKPI conflicts got resolved and most of the other LinuxKPI changes were committed to FreeBSD main.
During the update process firmware crashes were unxpectedly encountered which got solved and the 802.11 compat code improved. The iwlwifi driver and firmware got updated from the iwlwifi-next git branch and the linux-firmware repository.
Integration into FreeBSD main is pending for a policy reason. For the latest state of the development or code, please follow the referenced wiki page or the freebsd-wireless mailing list.
I would love to thank everyone who has reviewed changes, did initial testing, helped with drm-kmod, helped in various other ways, answered questions, encouraged, or patiently waited for any results or running code.
Sponsor: The FreeBSD Foundation
Contact: Mark Johnston <markj@FreeBSD.org>
Sanitizers are bug detection facilities which use a combination of instrumentation inserted by the compiler (LLVM in this case) and runtime state tracking to detect bugs in C code. They can automatically detect many types of C programming bugs, such as use-after-frees and uses of uninitialized variables, which may otherwise require substantial effort to identify. They are particularly effective in combination with regression testing suites or fuzzing tools such as syzkaller. Unlike tools such as Valgrind, software must be recompiled to enable a given sanitizer, but sanitizers can be used in the kernel. Kernels with sanitizers enabled incur a significant performance overhead from the runtime, in both CPU utilization and memory usage.
Work has been ongoing to port a pair of kernel sanitizers from NetBSD to FreeBSD. These are the Kernel Address SANitizer (KASAN) and Kernel Memory SANitizer (KMSAN). The KASAN port is complete and has been committed to FreeBSD’s development branch, and a small LLVM patch to enable KASAN and KMSAN on FreeBSD has also been committed. KMSAN intercepts all memory accesses and verifies that they are valid using some hidden state saved for each memory allocation in the kernel; see kasan(9) for further details. It can be enabled in amd64 kernels by compiling the GENERIC-KASAN kernel configuration. The FreeBSD regression test suite currently passes with KASAN enabled.
The KMSAN port is currently in progress and is nearing completion. KMSAN detects uses of uninitialized memory using the algorithms described in the original MemorySanitizer paper. In particular, it can detect instances of uninitialized kernel memory being copied out to userspace. Kernel subsystems may additionally call into the KMSAN runtime to verify the state of a given buffer. This can be used, for example, to add checks in the network stack for uninitialized bytes in egress packets. A number of bugs have been found using KMASN and the FreeBSD regression test suite; many have already been fixed (search for "KMSAN" in src commit logs for examples), and patches have been written for all others found so far.
Future work includes:
Finishing the KMSAN port and committing it to the FreeBSD main branch.
Enabling CI jobs to run the test suite with KASAN and KMSAN enabled.
Adding syzbot configurations with KASAN and KMSAN enabled, and fixing bugs found it.
Reducing the scope of memory accesses which cannot be validated using KASAN or KMSAN today; this consists primarily of making use of the amd64 direct map optional in various parts of the kernel, and eliminating uses of UMA_ZONE_NOFREE.
Sponsor: The FreeBSD Foundation
Linux compatibility layer update
The goal of this project is to improve FreeBSD’s ability to execute unmodified Linux binaries. Current support status of specific Linux applications is being tracked at the Linux app status Wiki page.
The 32-bit syscalls which accept a
parameter now have their 64-bit
O_PATH flag to
open(2) system call
is now supported, as well as the
AT_EMPTY_PATH flag to
fstatat(2) (required for Qt5 applications in Ubuntu
fcntl(2) is now
ppoll(2) system call was reworked to fix
kernel build options got removed; they were confusing and not quite
functional. This doesn’t affect the usual setups which use
Linuxulator loaded as a kernel module.
The FreeBSD kernel can now generate coredumps for Linux
processes, on both amd64 and arm64. The cores can be analyzed with
There were a number of fixes to
implementation; as a result, the Linux
doesn’t get confused with multithreaded programs.
The kernel version was bumped to 4.4.0, as required for new Arch Linux binaries.
There was a number of fixes to Linuxulator on arm64, ranging from ELF auxilliary vector (auxv) and Thread-Local Storage fixes to documentation improvements. There is ongoing work to make Linuxulator on arm64 on par with the amd64 one. There is also ongoing work on improving the vDSO functionality.
sysutils/debootstrap port, and its
debootstrap package, now also work on
arm64, not just x86. It is also much faster now. There have been
some further improvements to the startup scripts.
NXP LS1028A SoC support
The Semihalf team has been working on adding the FreeBSD support for the NXP LS1028A SoC.
NXP LS1028A SoC is a dual-core 64-bit ARMv8 Cortex-A72 application processor with high-speed peripherals such as 2 Time-Sensitive Networking-capable (TSN) Ethernet controllers, quad-port TSN-enabled switch, PCIE, SD/MMC, USB3.0 and others.
The newly introduced support includes:
Upstream quad-port TSN switch support
Sponsor: Alstom Group
Marvell ARM64 SoCs support
Marvell Armada 7k8k and Octeon TX2 CN913x SoC families are quad-core 64-bit ARMv8 Cortex-A72 processors with high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications.
Although the mentioned SoCs are mostly supported in FreeBSD HEAD, some pieces required improvements.
Upstream PCIE improvements
Improve and merge ICU support rework (D28803)
Multicast routing rework
Contact: Wojciech Macek <firstname.lastname@example.org>
The Semihalf team has been working on reworking the existing ip_mroute module. The previous implementation was over 20 years old and didn’t use modern kernel features.
A complete rework of locking mechanism was done to eliminate taking multiple locks for a single job. Some portions of code were modified to use lockless constructs, the BW-meter API was refactored to work in a single context and routing hot path was identified and made more efficient. All these fixes helped achieve over a 5x speed boost in multicast routing.
The newly introduced rework includes:
VFS path descriptors API
Contact: Konstantin Belousov <kib@FreeBSD.org>
Unix VFS API, from a very high point of view, provides two kinds of interfaces. One is the path-based operations, typically manipulating metadata, like remove, rename, change permissions and similar. The second is the file-descriptor operations, typical are read, write, truncate, and other. The well-known open(2) syscall returns a file descriptor from the path.
The file descriptor is a reliable handle for the file targeted by operations. Whatever other action was performed on the file meanwhile, rename, even delete, file descriptor still references the same file, and any operation done on it, affects that file.
Until recently, there was no similar reliable way to take a handle on the file path, and then reliably use it for the operations of the first kind. Now, Linux introduced some facilities that make it possible, and FreeBSD tried to provide a compatible API.
First, there is a way to get the file descriptor for a path. It is created with the open(2) syscall, when the O_PATH flag is specified. The returned descriptor cannot be used for normal file operations, it only designates a place in the filesystem namespace. Attempts to e.g. read or write using it results in EBADF. But the system also does not check the access rights on the target file while opening. It is enough for the process to read the content of the directory with the file name for open(O_PATH) to succeed.
To actually use such descriptor, there is a new flag AT_EMPTY_PATH, understood by the \*at(2) set of syscalls. When supplied, and the path argument set to "" (empty path), \*at(2) syscalls operate on the file designated by the dirfd argument.
It is not too exciting for e.g. fstatat(2), where already available fstat(2) operates on any kind of file descriptor. But consider linkat(2), which in combination with AT_EMPTY_PATH now allows to re-link any opened file descriptor to a named file (this requires root privilege). Other examples of available operations are chownat(2), chmodat(2), and others; see man pages. When a file descriptor and AT_EMPTY_PATH are supplied, the program knows for sure that the corresponding operation was done on the specific file, not subject to a race when our file was renamed, and some other file replaced it, providing a different file on the same path. With O_PATH, we get the same access rules as for normal chmod, not needing to have read or write permission for the file content.
A way to translate the path descriptor to a normal file descriptor that can be read from or written to is to use openat(2) with O_EMPTY_PATH flag. The syscall checks the current access permissions and grants the requested mode by returning a normal descriptor.
In Linux, there is no equivalent of the O_EMPTY_PATH flag. It seems, instead, that its implementation of our equivalent of fdescfs opens the underlying file on open(2) of /dev/fd/N name. This behavior is incompatible with the existing FreeBSD fdescfs exposed operation, so it cannot be changed for the default mount of fdescfs on /dev/fd. A new mount flag "nodup" was added for fdescfs which emulates Linux for descriptors backed by vnodes. See the fdescfs(5) man page for details.
The described new Unix VFS API is used by Samba. Adding it to FreeBSD should allow our system to be still the first-grade platform for Samba, and was requested by Samba porters.
Sponsor: The FreeBSD Foundation
Dummynet support for pf
Contact: Kristof Provost <kp@FreeBSD.org>
The pf firewall currently relies on ALTQ for traffic shaping. ALTQ is not enabled in default kernel builds, and is not compatible with all network drivers (only drivers which implement if_start()).
Work is in progress to add support for dummynet traffic shaping to pf. Dummynet is already used by ipfw for traffic shaping.
As part of this work, automated tests are being added to dummynet, for both ipfw and pf. This requires extending dummynet to add vnet support, which was contributed by Tom Jones <thj@FreeBSD.org>.
While this work is incomplete feedback on architecture and functionality is welcomed.
Additional test cases
Debug failure of the IPv6 queue test case
Fix panic on vnet shutdown if there are still IPv6 packets queued. (These eventually get sent to ip6_input() with a now freed rcvif pointer. Badness ensues.)
Sponsor: Rubicon Communications, LLC ("Netgate")
Ethernet support for pf
Work is ongoing to add basic support for Ethernet filtering to pf.
This will allow layer 2 addresses to be used to tag packets for subsequent filtering or shaping in the existing pf code. The layer 2 code is strictly stateless.
The intended use case for this is to improve pf’s capabilities in captive portal setups (i.e. allow/deny internet access based on client MAC addresses).
(optional) anchor support
move nvlist interface code into libpfctl
audit nvlist code for bugs (several bugs were found in the recent nvlist alternatives to existing ioctl calls)
(optional) VLAN ID filtering
(optional) MAC address table support
While this work is incomplete, feedback on architecture and functionality is welcomed.
Sponsor: Rubicon Communications, LLC ("Netgate")
pf syncookie support
Contact: Kristof Provost <kp@FreeBSD.org>
SYN cookies are a mechanism to resist SYN flood denial of service attacks. The FreeBSD network stack has supported this since 2001. However, this does not prevent such an attack from flooding the pf state table.
OpenBSD ported this mechanism from FreeBSD into their pf in 2018. This code is now being ported to FreeBSD’s pf.
The work is still in progress, but the current version demonstrates the basic capability. Next steps include additional testing, extra automated test cases and implementing the adaptive mode.
The adaptive mode requires extra care in FreeBSD’s pf, above what was done in OpenBSD, because of different locking strategies.
Sponsor: Modirum MDPay
Microchip PolarFire SoC support
The Conclusive Engineering team is working on support for the Microchip PolarFire FPGA SoC chip family. PolarFire SoC is based on five 64-bit hard RISC-V cores, out of which four are equipped with MMU and capable of running FreeBSD. The SoC also features an FPGA and various peripherals including Gigabit Ethernet, PCIe and multi-function 12.3Gbps SERDES transceivers.
At the time of writing, the following tasks have been completed:
Initial platform bring-up with SMP support
Add support for creating "booti"-style images for RISC-V
Work in progress:
Clock and reset domain support
PCI Express driver
Racct (Resource Accounting) Bug Fixes and Improvements
Contact: Cyril Zhang <email@example.com>
racct is a resource accounting system in the FreeBSD kernel. It keeps track of resource usage, such as memory use and CPU runtime. Additionally, it provides an easy interface for limiting the resource usage of entities such as users and, importantly, jails. For example, it may be of interest to set a limit on the number of CPUs that a jail can use.
I have been discussing with other FreeBSD developers the prospect of adding racct support as an experimental feature to an OCI-compliant container runtime, runj by Samuel Karp. This would mimic Linux’s cgroups functionality in the OCI specification, which allows Linux containers to have limits on memory, CPU usage, etc. It also allows us to consider the possibility of adding FreeBSD-specific configuration to the OCI specification. As part of this work, I have been improving racct so that it is more functional for use with jails.
A new, more accurate scheme for calculating total CPU usage of all processes in a jail
Fixing a bug that incorrectly counted the runtime of all child processes in the parent’s runtime
Fixing a bug that incorrectly decremented the count of persistent resources, such as shared memory, when a process exits
Accounting for POSIX features like shared memory and semaphores, where previously only SysV features were accounted for
Feel free to get in touch.
Sponsor: The FreeBSD Foundation
OpenZFS RAIDZ Expansion update
Contact: Matthew Ahrens <firstname.lastname@example.org>
OpenZFS Pull Request URL: https://github.com/openzfs/zfs/pull/12225
video from 2021 FreeBSD Developer Summit URL: https://youtu.be/3SUKJye54aI?t=6166
slides from 2021 FreeBSD Developer Summit URL: https://docs.google.com/presentation/d/1FeQgEwChrtNQBHfWSNsPK3Y53O5BnPh3Cz5nRa5GAQY/view
news article from Ars Technica URL: https://arstechnica.com/gadgets/2021/06/raidz-expansion-code-lands-in-openzfs-master/
This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn’t sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks).
FreeBSD’s ZFS implementation comes from the OpenZFS project. This work will be integrated into the OpenZFS repo, with support for FreeBSD and Linux.
The work is functionally complete, and a pull request has been opened.
Remaining work includes some code cleanup, design documentation, addressing test failures.
We also need to solicit code reviewers and respond to code review feedback.
Sponsor: The FreeBSD Foundation
Microchip Switchtec support
The Conclusive Engineering team is working on support for the Microchip Switchtec PCI Express storage switch family.
Switchtec devices are PCI Express Gen3/Gen4/Gen5 packet switches which offer built-in NTB support for up to 48 NT partitions, extensive diagnostics and programmable firmware which can be used to implement additional functionality such as NVMe enclosure management.
At the time of writing, following features are completed:
Firmware management and diagnostics
Work in progress:
NVMe enclosure management
This work in co-sponsored by AGILESTORAGE Europe GmbH.
Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.
Contact: Emacs Ports Team <emacs@FreeBSD.org>
The Emacs development port, editors/emacs-devel, continues to be updated twice per month to synchronize with the HEAD of upstream’s master branch. A noteworthy change from the first June 2021 update was the addition of a NATIVECOMP port option. NATIVECOMP adds support for compiling Emacs lisp to native code using libgccjit, which was first enabled in lang/gcc11-devel and is now in lang/gcc11.
For more information about native compilation of Emacs lisp, see Gcc Emacs Wiki.
FreeBSD Erlang Ecosystem Ports update
Contact: FreeBSD Erlang mailing list <erlang@FreeBSD.org>
The Erlang runtime system, commonly known as the BEAM, provides a runtime that is used by a number of programming languages and applications in the FreeBSD ports collection.
Notable changes in 2021 include:
adding OTP24, including support on amd64 architecture for JIT compilation, and dropping the previous NATIVE and HIPE options
security fixes for the
adding a new language runtime for Gleam language
more than 40 point release updates in the last quarter alone for the Erlang runtimes
As the upstream Erlang OTP team have committed to supporting the 2 current releases, more and more point updates are arriving for OTP22-24, but not for the older Erlang runtime releases, which are now unlikely to get security and bug fixes.
The Erlang team is planning to:
deprecate in 2021Q3 any ports that are not compatible with OTP releases in the last 2 years
remove the deprecated runtimes in 2021Q4
remove ports directly dependent on erlang- and elixir- languages, where they are more commonly installed via
rebar3tools, the standard community build tool chain.
update RabbitMQ to the next major release, which requires OTP23 minimum, and introduces support for the JIT
bump the main
lang/erlangruntime to OTP24 because JIT is awesome
Additional testing and community contributions are welcomed, please reach out on the mailing list, especially if you are able to help testing of specific port updates.
GCC in the Ports Tree (lang/gcc*)
Contact: Gerald Pfeifer <email@example.com>
With the great help of linimon@ GCC 10 became the default version of GCC in the Ports Collection bringing many improvements and fixes.
Looking one step ahead, GCC 11 is now available as a port and even for use as GCC_DEFAULT via Mk/bsd.default-versions.mk .
Modern GCC ports like this now feature support for powerpcle, and most related changes also made it it upstream.
On the infrastructure side, USE_GCC now allows for a build time-only dependency, e.g. USE_GCC=yes:build .
And in case you are developing other ports, USE_GCC=any is a thing of the past and USE_GCC no longer tries to use the old 4.2-based system compiler (/usr/bin/gcc) even if present.
Finally, after some two decades of maintaining FreeBSD’s lang/gcc* ports, the time has come to hand over the baton and largely step back. Please reach out if you are want to help - we’d hate to simply drop maintainership and would be happy to collaborate and transition.
KDE on FreeBSD
Contact: Adriaan de Groot <kde@FreeBSD.org>
The KDE on FreeBSD project aims to package all of the software produced by the KDE Community for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma, graphics applications, instant messengers, a video-editing suite, as well as a tea timer and hundreds of other applications that can be used on any FreeBSD machine.
The KDE team (kde@) is part of desktop@ and x11@ as well, building the software stack to make FreeBSD beautiful and usable as a daily-driver graphics-based desktop machine.
Why the rename? We’ve been
#kde-freebsd since 2003
or so. However, changes in the IRC Network landscape have caused us
to migrate to Libera.Chat and join up with the rest of the desktop
folk — we have a lot of shared concerns, after all.
The new KDE Gear release — all the KDE software products that are not specifically KDE Frameworks or KDE Plasma, which release in a coordinated fashion on a regular schedule — arrived in ports the day it was released. Albert Astals Cid has a good blog post explaining the rationale behind KDE Gear.
Wayland support has arrived for KDE Plasma. It is now possible to run a KDE Plasma Wayland environment on FreeBSD machines with a DRM-enabled graphics card (Intel iGPU or AMDGPU).
There is slow-but-steady progress in reducing the dependencies of KDE Applications. Most FreeBSD ports are "batteries included" and that includes all the build-tools to create the application, but really, an IDE should not be a dependency of your photo-gallery application.
X11 was made optional in some of the Qt ports; this opens up possibilities such as "Wayland-Only" Qt, or "Framebuffer-Only". X11 remains enabled by default.
Qt6 is stuck in the "it builds, and parts of it run, but not enough" stage and has not landed in ports; it will probably wait until after the next quarterly is cut.
Poppler, CMake, polkit .. these underpinnings of the Free Desktop on FreeBSD all received their regular updates.
extproc/kdiff3had some serious issues with 3-way-merges. This was reported by Ed Maste, and turned out to have fixes upstream.
x11/konsole, the terminal emulator, would not accept
/bin/shas a shell; this is more of an issue on FreeBSD than on Linux where
bashis a very common choice. This was resolved upstream and landed in ports.
The regular KDE Frameworks releases (monthly) and KDE Plasma releases (monthly) and KDE Gear all were announced upstream and landed in FreeBSD ports without a fuss. Upstream CI continues to be updated with the latest FreeBSD ports so that CI reflects well what the KDE-on-FreeBSD team delivers to FreeBSD users.
libglvnd on FreeBSD
Contact: Kevin Bowling <kbowling@FreeBSD.org>
libglvnd (GL Vendor-Neutral Dispatch) "is a vendor-neutral dispatch layer for arbitrating OpenGL API calls between multiple vendors. It allows multiple drivers from different vendors to coexist on the same filesystem, and determines which vendor to dispatch each API call to at runtime. Both GLX and EGL are supported, in any combination with OpenGL and OpenGL ES."
Making this the default GL implementation in FreeBSD Ports unblocks a number of efforts and aligns us with future graphics stacks.
It cleans up the ability to have multiple GL implementations on one system or image, regardless of what hardware is in use, by eliminating the need for one-or-the-other hard links for these libraries. This can in turn be used by Nvidia Optimus setups, or using your fancy PCIe interconnect to have multiple vendor GPUs for any reason. It will support future Mesa releases where a concurrent LTS (Long Term Support) branch install can be used to fill in drivers that will be removed from newer Mesa releases.
Software like KDE and Firefox can use EGL contexts under X11 (with OpenGL) and Wayland, and in the case of upcoming Firefox releases it may match the speed of Google Chrome when doing so.
The library uses various implementation details to minimize any performance impact from indirection, with platform specific support for aarch64, amd64, armv7, i386, and powerpc64 (ELFv2).
Finally, it helps reduces over-linking with a new libOpenGL to provide OpenGL without X11 for headless servers or kms or Wayland environments.
If there is any remaining fallout from this change please add kbowling@ to the PR watchers.
Thank you to Jan Beich <jbeich@FreeBSD.org> for doing the ports infrastructure work and Matt Turner (Gentoo/freedesktop) for helping me understand the architecture and how to handle this change in a source-based distribution, as well as merging a change in libGLU upstream to support building without X11 dependency.
FreeBSD in Science
Meta-port for Biostar Handbook tools URL: https://www.freshports.org/biology/biostar-tools/
Simple, Portable Cluster Manager URL: https://acadix.biz/spcm.php
Desktop-installer URL: https://acadix.biz/desktop-installer.php
FreeBSD has long provided a solid foundation for scientific computing, with its remarkable reliability, network and storage performance, and the FreeBSD ports system which not only provides an extensive collection of scientific software, but also facilitates optimized installation by allowing the user to easily install from source with additional compiler flags.
Last quarter saw the introduction of sysutils/spcm (Simple, Portable Cluster Manager), an integrated tool set for building and managing HPC (High Performance Computing) clusters based on FreeBSD.
This quarter we saw rapid growth in the biology category, much of which culminated in the completion of the biology/biostar-tools metaport. This metaport installs virtually all of the software needed by bioinformatician in training. It is now easier for bioinformatics students to install and run the software they need on FreeBSD than on any other platform. Bioinformatics is the analysis of biological data such as gene and protein sequences.
We also hit a milestone of 200 ports in the biology category, thanks to contributions from 21 maintainers. Software installation has always been and still is a major hurdle for many researchers. Far too often, researchers attempt "cave man" installations, following primitive instructions from the developers' site for manually running configure scripts and make. In many other cases, they struggle with low-quality application containers or third-party on package managers that are prone to failures and difficult to use. Problems installing the software they need often cause major delays in their research and often end in failure. The rapid growth of scientific software in the FreeBSD ports collection, coupled with the introduction of SPCM and numerous improvements to sysutils/desktop-installer, have removed many hurdles that could delay or prevent scientific discovery.
Noteworthy changes in the documentation tree, in manpages, or in external books/documents.
FreeBSD Translations on Weblate
Contact: Danilo G. Baio <dbaio@FreeBSD.org>
The Weblate project Documentation (Books and Articles) is open to the public.
The Automatic Translation feature was executed in all languages, re-using translations from the former project (Docbook).
There are still pending issues about the new translation process, and the status can be seen on IdeaList#Translation.
Website translations weren’t released through Weblate yet. So we decided to hold it until we have more details about the working group that will redesign the Website and the Documentation Portal, to avoid re-work.
FreeBSD Website Revamp - WebApps working group
Contact: Sergio Carlavilla <carlavilla@FreeBSD.org>
Working group in charge of creating the new FreeBSD Documentation Portal; and redesigning the FreeBSD main Website and its components. FreeBSD developers can follow and join the working group on the FreeBSD Slack channel #wg-www21. The work will be divided into four phases:
Redesign of the Documentation Portal
Create a new design, responsive and with global search. (Work in progress)
Redesign of the Manual Pages on Web
Scripts to generate the HTML pages using mandoc. (Not started)
Redesign of the Ports page on Web
Ports scripts to create an applications portal. (Not started)
Redesign of the FreeBSD main Website
New design, responsive and dark theme. (Not started)
Objects that defy categorization.
Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.
Contact: Dan Langille <dvl@FreeBSD.org>
FreshPorts, and its sister site, FreshSource, have reported upon FreeBSD commits for 20 years. They cover all commits, but its primary focus is ports.
FreshPorts tracks the commits and extracts data from the port Makefiles to create a database of information useful to both port developers and port users.
For example, https://www.freshports.org/security/acme.sh/ shows the history of this port, back to its creation in May 2017.
The transition from subversion to git went superbly. Now the work is concentrating on branches. We are close to incorporating all branches (on src, doc, and ports) into the website.
The website now queries the main repo every three minutes and pulls in updates. Commit processing under git is faster and more reliable. Creating the XML directly from git and not from parsing commit emails has its benefits.
Since the last report, Jethro Nederhof has been doing fantastic work bringing the website up to HTML5 and into the modern world. Nothing dramatic, from what the users see, as it has been mostly behind-the-scenes.
We can always use more helpers. The website mostly runs itself and requires very little on-going maintenance (pkg upgrades, OS patches, etc). Most of the work is designing new features, fixing bugs, identifying issues, and discussion with users. Changes to the ports tree usually don’t affect the website much.
There is a long list of issues for the website.
The git_proc_commit project also has a set of issues, mostly in Python, some perhaps related to the website.
Documentation outlining how the various projects fit together and how they work is required.
I have some subversion repos which should be converted to git and uploaded to GitHub. This would allow people to work on the backend code (commit ingress and processing).
The FreshSource website could be updated to modern standards and the repo converted to git.
What is helloSystem?
helloSystem is FreeBSD preconfigured as a desktop operating system with a focus on simplicity, elegance, and usability. Its design follows the “Less, but better” philosophy.
Q2 2021 Status
Version 0.5.0 of helloSystem has been published. Installable Live ISO images are available. Download and changelog are at https://github.com/helloSystem/ISO/releases/tag/r0.5.0
helloSystem was part of the Desktop panel at the FreeBSD Developer Summit 2021. A video is available at https://www.youtube.com/watch?v=3SUKJye54aI&t=11410s
Work has started towards 0.6.0. Thanks for contributed features and bugfixes
Improved spatial file manager
Switch to KWin (if its dependencies can be reduced significantly for use outside of KDE Plasma)
Experimental and release builds of the Live ISO are available at https://github.com/helloSystem/ISO/releases.
Help is wanted in a number of areas, especially in the areas of the FreeBSD core OS and kernel, and Qt/C++.
Containers & FreeBSD: Pot, Potluck & Potman
Pot on github URL: https://github.com/pizzamig/pot
Potluck Repository & Project URL: https://potluck.honeyguide.net/
Potluck on github URL: https://github.com/hny-gd/potluck
Potman on github URL: https://github.com/grembo/potman
Pot is a jail management tool that also supports orchestration through Nomad.
In the last quarter, v0.12.0 was released and many new improvements have been worked on since then. The main new feature under development at the moment is the possibility to create layered pot images for e.g. quickly adding applications in a build pipeline to a bigger base image which can then be deployed via Ansible or Nomad.
Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker: A repository of Pot flavours and complete images for usage with Pot and in many cases Nomad.
Here, all images have been built for FreeBSD 13.0 for the first time, FreeBSD 11.4 support has been removed and 12.2 images have been upgraded to latest package versions. Furthermore, new images like a PostgreSQL Patroni cluster have been added or the Nomad and Consul images have been extended to support clustering, encryption and a new Vault image.
Potman aims to simplify building Pot images with Vagrant and VirtualBox based on the Potluck approach, e.g. as part of a DevOps workflow for software development including testing and publishing them to a repository.
Here, in the last quarter, usage and documentation has been improved.
Future plans include finishing the layered Pot images, further Potman workflow features and more new and improved Potluck images.
As always, feedback and patches are welcome.