Testing Guide for 4.7-RELEASE

Goals

As part of our on-going effort to improve the release engineering process, we have identified several areas that need significant quality assurance testing during the release candidate phase. Below, we've listed the changes in 4.7-RELEASE that we feel merit the most attention due to their involving substantial changes to the system, or having arrived late in the development cycle leading up to the release. In general, our goal in the QA process is to attempt to check a number of things:

• The system has not regressed with respects to stability, correctness, interoperability, or performance of features present in prior releases.
  
  
• New features result in the desired improvement in stability, correctness, interoperability, or performance.

To effectively determine this, it's desirable to test the system in a diverse set of environments, applying a wide set of workloads, forcing the system to operate both within and outside its normal specification. Particular focus should often be placed on the continuing (or new) capability of the system to perform correctly when used in concert with systems from other vendors.

Features to explore carefully:

• PPP. A number of significant PPP changes were merged to -STABLE before the code freeze. In particular, the following changes were made:

  • IPv6 support
  • Better RADIUS support (including MS-CHAP)
  • Improved handling of bad negotiation options
  • Improved negotiation diagnostics
  • Improved sockaddr (RTAX) parsing
  • More consistent compile time options (NOXXX to compile something out)
  • More available macros
  • Firewall punching
  • Improved ``proxy'' and ``proxyall'' handling
  • Handle connected UDP sockets on descriptor 0 in -direct mode
  • Fix a load of typos
  • Update to internal version 3.1
• IPFilter has recently been updated to version 3.4.29 and has not yet been thoroughly tested in -STABLE.
• ahc and ahd Adaptec SCSI drivers: There has been a major update to the ahc and ahd device drivers. These drivers have not yet been thoroughly tested in -STABLE.
• ATA. There were some problems with tagged queuing and some CDROM read operations with the ATA stack in FreeBSD 4.6. These problems were fixed in 4.6.2, but more testing on different hardware is still needed.
• Sendmail has been updated to 8.12.6. This should be a very stable bugfix release, but sendmail users are encouraged to help test this before 4.7 is released.
• XFree86 has been updated to 4.2.1. This release fixes some security issues and other bugs in 4.2.0, but should have almost identical functionality.
• Heimdal (Kerberos 5 support) has been updated. Kerberos 5 users are encouraged to test this before 4.7 is released.

The release notes will always be a good place to look for things to test.

Known Issues

• The 4.7-RC1 snapshots were built without packages due to some problems (which only recently came to light) in the bzip2 package support. Resolution: The RE team decided to return to gzip packages for 4.7-RC2 (as well as any subsequent RC snapshots and the final release), thus allowing this snapshot to have its normal package set.
• Partially as a result of the above package problems, the ports tree on the 4.7-RC1/i386 ISO image is not exactly the same as the 4.7-RC1/i386 FTP directory. Both will be eventually updated for subsequent RC snapshots and the final release. Resolution: Not a factor for subsequent snapshots.
• Loading kernel modules on 4.7-RC1/alpha is broken. Resolution: A fix has been committed and will be present in 4.7-RC2/alpha.
• When booting from the install media (e.g. a CDROM), sysinstall tries to load a set of modules from the mfsroot image. For some reason, sysinstall cannot load the module containing the aac driver; this results in an error dialog when starting sysinstall. Access to aac devices from within sysinstall is, understandably, broken by this error. This appears to be due to a dependency on the linux module. Resolution: The aac driver was brought back into the install kernels, and other modules were moved to modules.

---