As part of our continued support of the FreeBSD Project, we have a full-time staff member dedicated to improving the Project’s continuous integration system and the test infrastructure. In 2023, we added more testing jobs for ARM64 architectures like testing with Kernel Address Sanitizer and building test for non-standard compilers like GCC 12 and 13.

The post 2023 in Review: Continuous Integration and Workflow Improvement first appeared on FreeBSD Foundation.

The post Security, Performance, and Interoperability; Introducing FreeBSD 14 first appeared on FreeBSD Foundation.

The FreeBSD Foundation is pleased to announce that it has awarded Bjoern Zeeb a grant to improve the maturity of IPv6 support in FreeBSD and PC-BSD. This project is jointly sponsored with iXsystems.

FreeBSD’s KAME-based reference implementation of IPv6 first appeared in FreeBSD 4.0, and can be found in a broad range of FreeBSD-derived commercial products. To date, IPv6 has been an optionally configured feature present in the default FreeBSD kernel; however, IPv6 configuration has implied configuration of IPv4. With much “IPv6-ready” application software relying on dual-stack behavior, broken IPv6 applications go unnoticed. Adding support for an IPv6 kernel without IPv4 will make FreeBSD and PC-BSD an ideal test and development platform for both open source and proprietary IPv6-aware application software.

“Narrowing down the code base to not rely on legacy IP will help us to identify OS and application components requiring improvement to work well in an IPv6 environment. This project will help to ensure a bright IPv6 future, as FreeBSD is used throughout the Internet: root name servers, storage appliances, routers, firewalls, TVs, desktop and mobile systems, and many of the world’s busiest web sites,” said Mr. Zeeb. FreeBSD Foundation director and FreeBSD core team member Robert Watson described the project as critical to the future of FreeBSD, “Bjoern’s work will not only improve the maturity of our IPv6 implementation, but also motivate improvement of applications used in million of deployed FreeBSD and FreeBSD-derived systems.” The project will also improve the quality and performance of FreeBSD’s IPv6 stack.

Bjoern Zeeb is a consultant based in Germany and has been an active FreeBSD committer since 2004. He is currently also a member of the FreeBSD Security and Release Engineering teams, and was recently awarded the Itojun Service Award for his work on IPv6 in FreeBSD.

The post New Funded Project: Improved IPv6 Support first appeared on FreeBSD Foundation.

Grenville Armitage has provided a summary of the completed TCP Congestion Control Algorithms project:

Background

TCP is a crucial part of any modern operating system. FreeBSD’s standard “NewReno” congestion control (CC) is not able to fully utilize the high capacity links available today. A range of newer CC algorithms have emerged (and continue to emerge) from the networking research community over the past 15+ years. These include traditional loss-based algorithms (where packet losses indicate network congestion) and delay-based algorithms (where changes in Round Trip Time, RTT, are used to infer network congestion).

However, to date FreeBSD’s TCP stack has not had an easy-to-use mechanism for introducing new CC algorithms. In recent years the Centre for Advanced Internet Architectures (CAIA) at Swinburne University of Technology has (with the support of the Cisco University Research Program Fund at Community Foundation Silicon Valley) been developing a range of extensions to the FreeBSD TCP stack. These included a modular framework for adding new CC algorithms and new modular implementations of the existing NewReno algorithm, four other algorithms from the literature (H-TCP, CUBIC, Vegas and HD) and a novel algorithm developed at CAIA (CHD). In mid-2010 the FreeBSD Foundation funded CAIA to complete, tidy up and commit a number of these key enhancements to the FreeBSD TCP stack.

Delivered

Our project, “Five New TCP Congestion Control Algorithms for FreeBSD”, has delivered the following enhancements to FreeBSD’s TCP stack:

• Modular congestion control framework.
• Khelp (Kernel Helper) and Hhook (Helper Hook) frameworks.
• Basic Khelp/Hhook (Kernel help/hook) integration with the TCP stack.
• ERTT (Enhanced Round Trip Time) Khelp module for delay-based TCP algorithms.
• Modularised implementations of NewReno, CUBIC and HTCP loss-based TCP CC algorithms.
• Modularised implementations of Vegas, “HD” and “CHD” delay-based TCP CC algorithms.
• Technical report comparing the computational overhead associated with TCP before and after integrating the new frameworks and modularised NewReno algorithm

Benefits

Each congestion control algorithm is implemented as a loadable kernel module. Algorithms can be selected to suit the application/network characteristics and requirements of the host’s installation. The modular CC framework also makes it much easier for developers to implement new algorithms, allowing FreeBSD’s TCP to be at the forefront of advancements in this area, while still maintaining the stability of its network stack.

CUBIC and HTCP are variants of TCP that provide significant performance improvements (relative to NewReno) over high bandwidth, high latency paths. Vegas, HD, and CHD utilise RTT fluctuations to provide a more timely indication of network congestion — by not forcing network queues to overflow, delay-based CC algorithms can help to keep queuing delays low along a network path. CHD is also tolerant of packet losses that are unrelated to congestion (such as can occur over wireless links).

In addition, the Khelp/Hhook frameworks provide useful kernel infrastructure which are not specific to the TCP stack and we anticipate they will be used elsewhere in the kernel in the future to provide other unrelated enhancements to FreeBSD.

Participants

Code development, testing, and documentation: David Hayes and Lawrence Stewart

Editorial review of code and documentation: Rui Paulo and Bjoern Zeeb

Project supervision: Grenville Armitage

Project URL: http://caia.swin.edu.au/freebsd/5cc/

The post Summary of Five New TCP Congestion Control Algorithms Project first appeared on FreeBSD Foundation.

The FreeBSD Foundation is pleased to announce that Julien Ridoux and Darryl Veitch at the University of Melbourne have been awarded a grant to implement support of feed-forward clock synchronization algorithms.

The Network Time Protocol (NTP) is widely used for synchronization over the network and the ntpd daemon is the current reference synchronization algorithm. The system clock in FreeBSD is currently designed with ntpd in mind, leading to strong feedback coupling between the kernel and the synchronization daemon.

The RADclock is an example of an alternative class of synchronization algorithms based on feed-forward principles. This project will provide the core support for feed-forward algorithms, so that alternatives to ntpd can be developed and tested. The central motivation for this is the strong potential of such approaches for highly robust and accurate synchronization.

Beyond this, virtualization is one of the next major challenges faced by time keeping systems. The current feedback synchronization model is complex and introduces its own dynamics, an approach that is not suited to the requirements of virtualization. Feed-forward based synchronization offers a cleaner and simpler approach, which is capable of providing accurate time keeping over live migration of virtual machines.

This project will conclude in March 2011.

The post Feed-Forward Clock Synchronization Algorithms Project first appeared on FreeBSD Foundation.

The FreeBSD Foundation is pleased to announce that Swinburne University’s Technology’s Centre for Advanced Internet Architectures has been awarded a grant to implement five new TCP congestion control algorithms in FreeBSD.

Correctly functioning congestion control (CC) is crucial to the efficient operation of the Internet and IP networks in general. CC dynamically balances a flow’s throughput against the inferred impact on the network, lowering throughput to protect the network as required.

The FreeBSD operating system’s TCP stack currently utilizes the defacto standard NewReno loss-based CC algorithm, which has known problems coping with many aspects of modern data networks like lossy or large bandwidth/delay paths. There is significant and ongoing work both in the research community and industry to address CC related problems, with a particular focus on TCP because of its ubiquitous deployment and use.

Swinburne University of Technology’s ongoing work with FreeBSD’s TCP stack and congestion control implementation has progressively matured. This project aims to refine their prototypes and integrate them into FreeBSD.

The project will conclude in January 2011.

Addendum:

The five protocols are:

• H-TCP for high bandwidth-delay product paths
• CUBIC for fast long-distance networks
• Vegas for end to end congestion avoidance
• and 2 new algorithms Hamilton Delay and CAIA-Hamilton Delay

The post Five new TCP Congestion Control Algorithms Project first appeared on FreeBSD Foundation.

Max Khon has completed the DAHDI Project and provides the following report:

I am pleased to announce the completion of the DAHDI/FreeBSD project.

DAHDI (Digium/Asterisk Hardware Device Interface) is an open-source device driver framework and a set of HW drivers for E1/T1, ISDN digital and FXO/FXS analog cards.

The main goal of this funded project was to make it possible to use FreeBSD as a base system for software PBX solutions.

Currently, most of the DAHDI bits have been ported, including the DAHDI framework itself, HW drivers, TDMoE drivers, drivers for software and HW echo cancellation (Octasic, VPMADT032) and HW transcoding (TC400B). The project is hosted in the official DAHDI SVN repository.

Misc/dahdi in the FreeBSD ports collection now contains the most recent bits of DAHDI/FreeBSD and also some stuff that is not available in DAHDI/FreeBSD SVN due to licensing and copyright restrictions. These include the OSLEC echo canceller and the experimental zaphfc driver.

I will continue periodic merges from DAHDI/Linux SVN on a regular basis and roll out new DAHDI/FreeBSD releases. These will most likely be synchronized with DAHDI/Linux releases.

The post Update on DAHDI Project first appeared on FreeBSD Foundation.

Bjoern Zeeb has provided a summary regarding the completion of the funded portion of the FreeBSD Jail Based Virtualization Project:

I am happy to report that the funded parts of the FreeBSD Jail Based Virtualization project are completed. Some of the results have been shipping with 8.1-RELEASE while others are ready to be merged to HEAD.

Jails have been the well known operating system level virtualization technique in FreeBSD for over a decade. The import of Marko Zec’s network stack virtualization has introduced a new way for abstracting subsystems. As part of this project, the abstraction framework has been generalized. Together with Jamie Gritton’s flexible jail configuration syscalls, this will provide the infrastructure for, and will ease the virtualization of, further subsystems without much code duplication. The next subsystems to be virtualized will likely be SYSV/Posix IPC to help, for example, PostgreSQL users. This will probably be followed by the process namespace.

Along with the framework, debugging facilities, such as the interactive kernel debugger, have been enhanced so that every new subsystem will be able to immediately make use of these improvements without modifying a single line of code. Libjail and jls can now work on core dumps and netstat is able to query individual live network stacks attached to jails.

For the virtual network stack, work was focused on network stack teardown, a concept introduced with the network stack virtualization. The primary goal was to prototype a shutdown of the (virtual) network stacks from top to bottom, which means letting interfaces go last rather than first and still being able to cleanly shutdown TCP connections. Good progress was made, but a lot of code over the last two decades was never written in a way to be cleanly stopped. Work on this will have to continue, along with virtualizing the remaining network subsystems to allow long term stability and a leak and panic free shutdown. As a side effect, users of non-virtualized network stacks will also benefit, as other general network stack problems are identified and fixed along the way.

I am happy to see more early adopters, former OpenSolaris users, and people contributing code or reporting problems and would like to encourage people to further support this project.

My special thanks go the FreeBSD Foundation and CK Software GmbH for having sponsored this project, as well as to John Baldwin and Philip Paeps for helping with review and excellent suggestions.

The post Update on FreeBSD Jail Based Virtualization Project first appeared on FreeBSD Foundation.

On July 3, Lawrence Stewart committed SIFTR (Statistical Information For TCP Research) to HEAD. SIFTR was part of the Improvements to the FreeBSD TCP Stack project that the Foundation funded last year. SIFTR is a kernel module that logs a range of statistics on active TCP connections to a log file. It provides the ability to make highly granular measurements of TCP connection state, aimed at system administrators, developers and researchers.

The post SIFTR Committed first appeared on FreeBSD Foundation.

We are pleased to announce that Edward Tomasz Napierala has been awarded a grant to implement resource containers and a simple per-jail resource limits mechanism.

Unlike Solaris zones, the current implementation of FreeBSD Jails does not provide per-jail resource limits. As a result, users are often forced to replace jails with other virtualization mechanisms. The goal of this project is to create a single, unified framework for controlling resource utilisation, and to use that framework to implement per-jail resource limits. In the future, the same framework might be used to implement more sophisticated resource controls, such as Hierarchical Resource Limits, or to implement mechanisms similar to AIX WLM. It could also be used to provide precise resource usage accounting for administrative or billing purposes.

“It’s great that the Foundation decided to fund this project”, Edward noted. “It will make jail-based virtualization a much better choice in many scenarios, for example for Virtual Private Server providers.”

This project will be completed December, 2010.

The post Resource Containers Project first appeared on FreeBSD Foundation.