NetBSD Code-In 2012/2013 wiki

NetBSD now offers full support for running file systems in userspace. Major components are “puffs”, which is the kernel subsystem that realizes the pass-to-userspace framework file system, as well as the userland libraries that support constructing file system implementations. These are libpuffs and the FUSE compatibility library, librefuse.

This page describes the situation in NetBSD 5.0 and later.

There are a number of components interacting to provide routines for userland file systems: puffs as the kernel part, and libpuffs and librefuse to provide functions for userland file systems to call. The following image gives an overview of the components and their connections:

The components themselves are:

• puffs is the core component inside the kernel. It exposes a file system interface towards the userland programs (accessible via /dev/puffs and libpuffs), and communicates with the kernel's own idea of files, which are represented as vnodes. The puffs kernel component and how to interface it is described further in the MAN.PUFFS.4 manpage.
• The “libpuffs” library is the interface between userland file systems and the kernel component. It provides a number of convenience routines that userland file systems can use, and is described in the MAN.PUFFS.3 manpage.
• To facilitate running the huge amount of file systems already available for the FUSE interface, but not dictate the capabilities of puffs by it, it was decided that FUSE support should be provided as a compatibility layer on top of the native puffs interface, which is offered by “librefuse”. The re-implementation of the FUSE functionality is designed to be source code compatible with FUSE, and it is further described in the MAN.REFUSE.3 manpage.

Examples for puffs and refuse filesystems can be found in the following directory of the NetBSD source tree in src/share/examples/{puffs,refuse}.

puffs is enabled by default in GENERIC on major architectures starting from NetBSD 5.0. For a custom kernel, the following are required in the kernel configuration.

  file-system     PUFFS
  pseudo-device   putter
        

Alternatively, modules can be used.

This section presents usage examples for file systems shipped with the NetBSD base system.

mount_psshfs(8)

  # mount_psshfs host.name.tld:/directory /puffs
  # ls /puffs
  AdobeFnt.lst    OS                  bin     public_html
  Desktop         OpenOffice.org1.1.0 in      tmp
  ...
  # cd /puffs
  # ls -l .cshrc
  -rw-r--r--  1 39068  2000  4706 Jun 16 01:01 .cshrc
  # head -2 .cshrc
  # Default .cshrc for Solaris, Irix, ...
  #
  # md5 .cshrc
  MD5 (.cshrc) = 2ad1d2606a5678f312709a388376c2e5
  # ls -l test
  ls: test: No such file or directory
  # date >test
  # ls -l test
  -rw-r--r--  1 39068  2000  29 Nov 23 01:19 test
  # cat test
  Thu Nov 23 01:19:36 MET 2006
  # umount /puffs
        

9P

mount_9p(8)

  # mount_9p nobody@192.168.1.2:/tmp /puffs
  # cd /puffs
  # echo 9puffs in action > msg_from_earth
  # ls -l msg_from_earth
  -rw-r--r--  1 nobody  wheel  17 Apr 25 23:24 msg_from_earth
  # rsh 192.168.1.2 cat /tmp/msg_from_earth
  9puffs in action
  # 

Since there is currently no support in the implementation for access control or support for authentication, the account nobody was picked just to prove a point. The NFS nobody user really does not have anything to do with 9P. Support for access control and use of pre-established secure connections will be added to mount_9p on a later date.

FUSE compatibility was added within pkgsrc, and besides the required infrastructure work a number of FUSE packages were added to pkgsrc in the new “filesystem” category. Example packages that are currently available include:

• fuse : Filesystem in Userspace (compat headers, pkg-config files, etc.), needed for pkgsrc on Linux
• fuse-archivemount : FUSE gateway to libarchive
• fuse-cddfs : FUSE filesystem that uses libparanoia for audio CDs
• fuse-ntfs-3g : a NTFS driver with read and write support
• …

More packages are currently being worked on, please see pkgsrc/filesystems for a full list.

Here is an example of installing and using the FUSE-enabled NTFS-3g implementation:

  # cd pkgsrc/filesystems/fuse-ntfs-3g
  # make install
  # ntfs-3g ntfs.img /ntfs
      

Using FUSE file systems on NetBSD 4.0 is possible, but in addition to adding puffs support support to the kernel, it requires fetching and manually installing a backport of the ReFUSE library. The library is available here and further instructions are available here.

An in-depth technical description of puffs was presented at AsiaBSDCon 2007 in a paper entitled “puffs

• Pass-to-Userspace Framework File System”. The paper and slides are available.

The ReFUSE emulation layer for FUSE file systems is described in ”ReFUSE: Userspace FUSE Reimplementation Using puffs” presented at EuroBSDCon 2007.

A paper discussing the implementation of distributed file systems on top of puffs was presented at AsiaBSDCon 2008.

The puffs manual pages provide further information from a development perspective.

The source code in browsable form:

• kernel code
• libpuffs
• librefuse
• puffs file systems in the base system
• source code examples of other puffs file systems
• source code examples for refuse

This chapter describes the symbiosis of background knowledge on them, see the respective web pages.

The integration of the puffs and rump technologies enables mounting kernel file systems as userspace servers. Equivalent functionality to the in-kernel option is maintained, and an application accessing files is unable to tell the difference between the two.

A useful scenario for mounting a kernel file system as a userspace server is accessing untrusted media. As file system code is generally authored assuming a trusted and correct file system image, a corrupt or deliberately modified image can easily cause a kernel crash or security compromise. By using a userspace server, the compromise is limited to the userspace server program, not the whole system.

It also enables the mounting of kernel file systems which are not included as part of the kernel and loading a module is not desired. Additionally, using kernel file systems from different operating system versions is possible, since interfacing with the kernel is done through the puffs protocol, not the kernel binary interface. This is desired for example when wanting to take advantage of new functionality without upgrading the operating system or wanting to do a partial rollback to avoid a regression in a newer version.

The use of kernel file systems as userspace servers requires the support for puffs in the kernel.

To mount the file system, the relevant rump_fs command must be run. These commands share the same suffix as the normal in-kernel mount_fs commands. For instance, the equivalent of mount_msdos is rump_msdos. The usage is also equal, so the manual page of the mount equivalent should be consulted. The only exception is that rump file systems do not require the use of vnconfig to mount regular files. Rather, such images can be mounted directly by using the image as the device file path.

To unmount, umount should be run as normal. While it is possible to violently kill the server process, this does not give the file system an opportunity to flush its caches and cleanly unmount and may result in data loss. In case the server process is violently killed, puffs automatically performs a forced unmount and no further action is necessary.

To mount a file system image as a read-only mount:

  pain-rustique> rump_ffs -o ro ~/img/ffs.img /mnt
  OR
  pain-rustique> mount -t ffs -o ro,rump ~/img/ffs.img /mnt

If you want to mount a disk device instead, the procedure is the same. This example also instructs the file system to use journalling. Note: for reasons beyond the scope of this document, it is highly recommended you use the raw device instead of the usual block device.

  pain-rustique> rump_ffs -o log /dev/rwd0e /mnt2

Mounting nfs works in a similar fashion. The command line flag -p makes sure a non-root mount is succesful if the server allows them:

  pain-rustique> rump_nfs -p host:/export /mnt3

Finally, the option rump in fstab signals that the file system should be mounted using rump instead of the kernel file service. The following example is for specifying an nfs share mount from a laptop over wireless:

  server:/m/server    /m/server    nfs    rw,noauto,-r=8192,-w=8192,rump

After this the file system will show up on the mountlist and you can access it through the mountpath like any other mounted file system.

Internally, kernel file systems are implemented against the kernel virtual file system layer. This layer defines a protocol which kernel file systems convert to their backend storage protocol to satisfy the operation. The result is then converted back to a format the virtual file system layer understands and passed back to caller (most likely an application, although it can be for example the NFS server).

Analogously, puffs file systems must interpret and convert the puffs file system interface protocol. While the puffs interface is very similar to the kernel virtual file system interface, there are differences in some parameters. For example, the virtual file system passes information about I/O as a reference to struct uiomove. In contrast, puffs passes this as explicit parameters.

Since a userspace kernel file system server attaches itself as a puffs file system, protocol conversion from puffs to the kernel virtual file system must happen, if the whole protocol stack is to work. This conversion of protocols is done by the p2k, or puffs-to-kernel, component. It is installed as libp2k and linked with all kernel file system servers.