Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
quail-vnet-howto(8)	  BSD System Manager's Manual	   quail-vnet-howto(8)

     qjail-vnet-howto -- Details explaining how	to use qjail to	build a	VIMAGE
     vnet jail system.

     This howto	will strive to explain how the qjail utility is	used in	a real
     world application Utilizing a documentation approach that fills in	the
     blanks of how all the separate network functions fit together.

     VIMAGE (virtualized network stack)	is a highly experimental feature.  Af-
     ter you compile vimage into your kernel, every boot from there on will
     issue this	warning	message	as part	of the boot console log	messages.
     This is FreeBSD's way of saying, USE IT AT	YOUR OWN RISK.

     It's assumed that you know	how to compile your kernel. That's not going
     to	be covered here. With FreeBSD RELEASE 11.0, Vimage still has to	be
     compiled into your	host's kernel before the jail(8) vnet function will
     work. The "option VIMAGE" is the only statement you have to add to	your
     kernel definition source. Compiling in the	bridge,	epair, and the
     firewall you plan to use is NOT necessary.	The starting of	the following
     discussion	is based on a running vimage kernel already being booted and
     having network access to the public internet verified by issueing the
     "ping -c 2" command on the	host.

     The following sequence of qjail commands were issued on a desktop type of
     computer running RELEASE 11.0. This computer is a node on a small LAN
     hard cabled to the	gateway	computer who's firewall	provides the NAT
     services for public internet access. The gateway is also running 11.0 and
     uses the ipfilter firewall. All "ifconfig command"	output shown below has
     had the lo0 interface details removed for brevity.

     Step 1.  Enter pkg	install	qjail to Install the qjail software.

     Step 2.  Enter qjail install to install the qjail environment.

     Step 3.  Enter qjail create -4 v10  To create	a
	      directory	tree type jail.	This is	the network ip address used
	      to communicate with the vnet jail. The name of the jail is v10.

     Step 4.  Enter qjail config -w fxp0 -v none v10  To change	the
	      non-vnet jail into a vnet	jail. fxp0 is the interface device
	      name connecting this computer to the LAN.	You could use vtnet0
	      or vlan0 interface devices if so desired.

	      The "-v none" means this vnet jail will not be using any
	      firewall.	The "-v	value"	options	are none, ipfw,	pf, ipf. At
	      this time, IE: 11.0, the ipfw firewall is	the only one that
	      functions. The host must be using	IPFW also for things to	work.
	      Read "man	qjail" config -v section for more details about
	      firewall selection.

     Step 5.  Enter qjail start	v10  Start this	jail.

     Step 6.  Enter ifconfig -a	to see what the	host's network looks like.

	   ether 00:0c:f1:cd:55:ea
	   inet netmask 0xfffffff0 broadcast
	   media: Ethernet autoselect (100baseTX <full-duplex>)
	   status: active
	   ether 02:0d:73:5e:c9:0a
	   md6 options=1<PERFORMNUD>
	   groups: bridge
	   id 00:00:00:00:00:00	priority 32768 hellotime 2 fwddwlay
	   maxage 20 holdcut 5 proto rstp maxaddr 2000 timeout 1200
	   root	id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	   member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE>
		   ifmaxaddr 0 port 4 priority 128 path	cost 2000
	   member: fxp0	flags=143<LEARNING,DISCOVER,AUTOEDGE>
		   ifmaxaddr 0 port 1 priority 128 path	cost 200000
	   ether 02:c1:00:00:04:0a
	   inet6 fe80::c1:ff:fe00:40a%epair1a prefixlen	64 scopeid
	   media: Ethernet 10Gbase-T (10Gbase-T	<full-duplex>)
	   status: active
	   groups: epair

     The fxp0 network device has been assigned an IP address of by
     the host system because of	this ifconfig_fxp0="DHCP" statement being in
     the host's	rc.conf	file.
     qjail defines it's	bridge as bridge10 so hopefully	it's name is unique
     enough not	to get steped on by some other host function.
     The bridge	has 2 members, one for the host's network device and one for
     the epair.	epair1a	is on the host and its matching	pair epair1b is	in the
     vnet jail.	The number 1 is	incremented by 1 for each additional vnet jail
     created. So if there were two vnet	jails, there also would	be a epair2a
     member shown.

     Step 7.  Enter qjail list	List status of all defined jails.
     STATUS JID	 NIC	IP		Jailname
     ------ ---- ------	---------------	-------------------------------
     DRV 0  7	 fxp0	v10

     D means this is a directory tree type of jail
     R means this jail is running
     V means this is a vnet jail
     0 means this vnet jail has	no firewall
     7 means this is the seventh time a	jail has been started

     Step 8.  Enter qjail console v10  Auto login to this jails	console.
     FreeBSD 11.0-RELEASE-p7 (Vimage) #0 r313172: Fri Feb  3 17:02:05 EST 2017
     Welcome to	your FreeBSD jail.
     v10 /root >

     Step 9. Enter ifconfig -a	to see what the	Vnet jail network looks	like.
     epair1b: flags=8843<UP.BTOADCAST,RUNNING,SIMPLEX,MULTICAST> metric	0
	      ether 02:C1:00:00:05:0B
	      inet netmask 0xff000000 broadcast
	      inet6 fe80::c1:ff;fe00:50b%epair1b prefixlen 64 scopeid 0x2
	      nd6 options=21<PREFORMNUD,AAUTO_LINKLOCAL>
	      media: Ethernet 10Gbase-T	(10Dbase-T <full-duplex>)
	      staatus: active
	      groups: epair
     v10 /root >

     Here we see the ip address has been assigned to epair1b in
     the vnet jail

     Step 10. Enter ping -c 2 Check to see if jail has public access.
     PING ( 56	data bytes
     64	bytes from icmp_seq=0 ttl=43 time=37.337 ms
     64	bytes from icmp_seq=1 ttl=43 time=35.983 ms
     --- ping statistics ---
     2 packets transmitted, 2 packets received,	0.0% packet loss
     round-trip	min/avg/max/stddev = 35.983/36.660/37.337/0.677	ms
     v10 /root >

     Step 11. Enter exit to leave the jail console.

Trouble	shooting network problems.
     1.	Cannot access the public internet. Firewall rules or NAT problem on
     the host or gateway computer.

     2.	Cannot ping the	hosts default route ip address.	Most likely you	don't
     know what the default gateway route ip address is.	On the host issue the
     "route get	default" command and use the gateway ip	address	in the ping
     command issued from the vnet jails	console.

     3.	If nothing you try seems to solve the problem, reboot the host so the
     host firewall and vimage in the kernel can	shake hands anew. Then most
     likly what	you were doing starts working. Your experiencing the highly
     experimental part of vimage.

     4.	Another	test you can do	is try to do a ssh login on the	host to	the
     vnet jails	ip address. First you have to prepare the vnet jail. Use the
     console command to	login to the vnet jail.	Edit the vnet jails rc.conf
     file adding the sshd_enable="YES" statement.
     Then issue	"pw adduser test -m -w yes" to create a	user account named
     test with password	of test.
     Exit the jails console. Issue "qjail restart v10" and then	issue
     "ssh -p 22 -l	test" and see if you can access	the vnet jail
     this way.

     5.	Be aware that a	vnet jail running no firewall and the host is running
     ipfw does not work	in the sense that vnet jail pings to the host or pub-
     lic ip addresses just time	out. If	the host and the vnet jail both	have
     ipfw running then the vnet	jail will have public internet access if both
     the host and vnet jail ipfw rules are correct. I suggest you start	with a
     single ipfw rule of "pass all" in both the	host's and the vnet's ipfw
     rules file	and then work from there. Keep in mind that just because the
     vnet jail has it's	own network stack does not mean	it's traffic is	by-
     passing the hosts firewall. Another thing to be aware of is ipfw on the
     host means	all vnet jails have to also be running ipfw.

     6.	Be aware that you can run pf or	ipfilter firewalls on the host and no
     firewall in the vnet jail and have	conplete control of the	vnet jail's
     network traffic from the host's firewall.

     7.	Qjail does enable you to create	a vnet jail using pf or	ipfilter run-
     ning in it	but this requires that the host	is running the same firewall
     as	the vnet jail. Things in the vnet jail's firewall will look like it's
     working, but add a	deny rule for outbound port 43 to the vnet jail's
     firewall. Then issue a whois command from the vnet	console	and you	will
     see it work when it should	not. These two firewalls run in	the hosts ker-
     nel to access the network stack and have not been modified	to work	with
     vimage yet.

     8.	There are some common utilities	that require the "bfp" device to func-
     tion in a jail such as dhclient(8)	and tcpdump(1).	There is a customized
     ruleset number 50 that adds the "bfp" device. The first time you use the
     "qjail config -b 50" command it will be created automatically.

     qjail(8), qjail-intro(8) qjail-howto(8), qjail-ipv6-testing

     Joe Barbish <>

BSD			       February	16, 2017			   BSD


Want to link to this manual page? Use this URL:

home | help