Run FreeBSD on pysical/virtual servers by PXE booting mouting root over NFS4
Mount iSCSI zvol as /dev/da0 and create zpool for use by iocage
Great for testing fixes and network performance testing. This configuration prepares 5 FreeBSD enviroments that can be run on physical servers
Current Versions:
<span class="pln">zfs create ship</span><span class="pun">/</span><span class="pln">pxe zfs </span><span class="kwd">set</span><span class="pln"> sync</span><span class="pun">=</span><span class="pln">always ship</span><span class="pun">/</span><span class="pln">pxe mkdir </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="lit">11</span><span class="pln"> mkdir </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="lit">12</span><span class="pln"> mkdir </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="lit">13</span><span class="pln"> cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">configs</span><span class="pun">/</span><span class="pln">pxe</span><span class="str">/ /</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe zfs create ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD11</span><span class="pln"> zfs create ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12</span><span class="pln"> zfs create ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pln"> zfs create ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12p</span><span class="pln"> zfs create ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pln"> zfs create </span><span class="pun">-</span><span class="pln">s </span><span class="pun">-</span><span class="pln">V </span><span class="lit">128G</span><span class="pln"> </span><span class="pun">-</span><span class="pln">o refreservation</span><span class="pun">=</span><span class="pln">none ship</span><span class="pun">/</span><span class="pln">raw</span><span class="pun">/</span><span class="typ">FreeBSD11</span><span class="pln"> zfs create </span><span class="pun">-</span><span class="pln">s </span><span class="pun">-</span><span class="pln">V </span><span class="lit">128G</span><span class="pln"> </span><span class="pun">-</span><span class="pln">o refreservation</span><span class="pun">=</span><span class="pln">none ship</span><span class="pun">/</span><span class="pln">raw</span><span class="pun">/</span><span class="typ">FreeBSD12</span><span class="pln"> zfs create </span><span class="pun">-</span><span class="pln">s </span><span class="pun">-</span><span class="pln">V </span><span class="lit">128G</span><span class="pln"> </span><span class="pun">-</span><span class="pln">o refreservation</span><span class="pun">=</span><span class="pln">none ship</span><span class="pun">/</span><span class="pln">raw</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pln"> zfs create </span><span class="pun">-</span><span class="pln">s </span><span class="pun">-</span><span class="pln">V </span><span class="lit">128G</span><span class="pln"> </span><span class="pun">-</span><span class="pln">o refreservation</span><span class="pun">=</span><span class="pln">none ship</span><span class="pun">/</span><span class="pln">raw</span><span class="pun">/</span><span class="typ">FreeBSD12p</span><span class="pln"> zfs create </span><span class="pun">-</span><span class="pln">s </span><span class="pun">-</span><span class="pln">V </span><span class="lit">128G</span><span class="pln"> </span><span class="pun">-</span><span class="pln">o refreservation</span><span class="pun">=</span><span class="pln">none ship</span><span class="pun">/</span><span class="pln">raw</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pln"> edit </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">exports edit </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">ctl</span><span class="pun">.</span><span class="pln">conf pkg install tftp</span><span class="pun">-</span><span class="pln">hpa sysrc nfs_server_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc rpcbind_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc mountd_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc mountd_flags</span><span class="pun">=</span><span class="str">"-r"</span><span class="pln"> sysrc rpc_lockd_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc rpc_statd_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc ctld_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc tftpd_enable</span><span class="pun">=</span><span class="str">"YES"</span><span class="pln"> sysrc tftpd_flags</span><span class="pun">=</span><span class="str">"-s /mnt/ship/pxe"</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="lit">11</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/11.3-RELEASE/base.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/11.3-RELEASE/kernel.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/11.3-RELEASE/src.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/11.3-RELEASE/ports.txz</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> </span><span class="kwd">base</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD11</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> kernel</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD11</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> src</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD11</span><span class="pln"> </span><span class="com">#tar Jxf ports.txz -C /mnt/ship/pxe/FreeBSD11</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="lit">12</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/12.1-RELEASE/base.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/12.1-RELEASE/kernel.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/12.1-RELEASE/src.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/releases/amd64/12.1-RELEASE/ports.txz</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> </span><span class="kwd">base</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> kernel</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> src</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12</span><span class="pln"> </span><span class="com">#tar Jxf ports.txz -C /mnt/ship/pxe/FreeBSD12</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> </span><span class="kwd">base</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12p</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> kernel</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12p</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> src</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12p</span><span class="pln"> </span><span class="com">#tar Jxf ports.txz -C /mnt/ship/pxe/FreeBSD12p</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="lit">13</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/snapshots/amd64/13.0-CURRENT/base.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/snapshots/amd64/13.0-CURRENT/kernel.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/snapshots/amd64/13.0-CURRENT/src.txz</span><span class="pln"> wget https</span><span class="pun">:</span><span class="com">//download.freebsd.org/ftp/snapshots/amd64/13.0-CURRENT/ports.txz</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> </span><span class="kwd">base</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> kernel</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> src</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pln"> </span><span class="com">#tar Jxf ports.txz -C /mnt/ship/pxe/FreeBSD13</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> </span><span class="kwd">base</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> kernel</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pln"> tar </span><span class="typ">Jxf</span><span class="pln"> src</span><span class="pun">.</span><span class="pln">txz </span><span class="pun">-</span><span class="pln">C </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pln"> </span><span class="com">#tar Jxf ports.txz -C /mnt/ship/pxe/FreeBSD13p</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe cp loader</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">boot cp loader</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">boot cp loader</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">boot cp loader</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">boot cp loader</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">boot cp sysctl</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc cp sysctl</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc cp sysctl</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc cp sysctl</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc cp sysctl</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc cp fstab</span><span class="pun">.</span><span class="lit">11</span><span class="pln"> </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">fstab cp fstab</span><span class="pun">.</span><span class="lit">12</span><span class="pln"> </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">fstab cp fstab</span><span class="pun">.</span><span class="lit">13</span><span class="pln"> </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">fstab cp fstab</span><span class="pun">.</span><span class="lit">12p</span><span class="pln"> </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">fstab cp fstab</span><span class="pun">.</span><span class="lit">13p</span><span class="pln"> </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">fstab cp resolv</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc cp resolv</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc cp resolv</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc cp resolv</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc cp resolv</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc cp sshd_config </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">ssh cp sshd_config </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">ssh cp sshd_config </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">ssh cp sshd_config </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">ssh cp sshd_config </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">ssh cp rc</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc cp rc</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc cp rc</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc cp rc</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc cp rc</span><span class="pun">.</span><span class="pln">conf </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">.</span><span class="pln">ssh </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">root cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">.</span><span class="pln">ssh </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">root cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">.</span><span class="pln">ssh </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">root cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">.</span><span class="pln">ssh </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">root cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">.</span><span class="pln">ssh </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">root cp iscsi</span><span class="pun">.</span><span class="pln">conf</span><span class="pun">.</span><span class="lit">11</span><span class="pln"> </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">iscsi</span><span class="pun">.</span><span class="pln">conf cp iscsi</span><span class="pun">.</span><span class="pln">conf</span><span class="pun">.</span><span class="lit">12</span><span class="pln"> </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">iscsi</span><span class="pun">.</span><span class="pln">conf cp iscsi</span><span class="pun">.</span><span class="pln">conf</span><span class="pun">.</span><span class="lit">13</span><span class="pln"> </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">iscsi</span><span class="pun">.</span><span class="pln">conf cp iscsi</span><span class="pun">.</span><span class="pln">conf</span><span class="pun">.</span><span class="lit">12p</span><span class="pln"> </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">iscsi</span><span class="pun">.</span><span class="pln">conf cp iscsi</span><span class="pun">.</span><span class="pln">conf</span><span class="pun">.</span><span class="lit">13p</span><span class="pln"> </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">iscsi</span><span class="pun">.</span><span class="pln">conf cp iscsid </span><span class="typ">FreeBSD11</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">rc</span><span class="pun">.</span><span class="pln">d cp iscsid </span><span class="typ">FreeBSD12</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">rc</span><span class="pun">.</span><span class="pln">d cp iscsid </span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">rc</span><span class="pun">.</span><span class="pln">d cp iscsid </span><span class="typ">FreeBSD12p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">rc</span><span class="pun">.</span><span class="pln">d cp iscsid </span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">rc</span><span class="pun">.</span><span class="pln">d </span><span class="com">#currently I mount the root rw as its one directory per server and its easier to install packages.</span><span class="pln"> </span><span class="com">#If NFS mounting root ro</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD11</span><span class="pln"> mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pln"> echo </span><span class="str">"16384"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">md_size echo </span><span class="str">"32768"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">/</span><span class="pln">md_size tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip etc tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip </span><span class="kwd">var</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12</span><span class="pln"> mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pln"> echo </span><span class="str">"16384"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">md_size echo </span><span class="str">"32768"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">/</span><span class="pln">md_size tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip etc tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip </span><span class="kwd">var</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pln"> mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pln"> echo </span><span class="str">"16384"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">md_size echo </span><span class="str">"32768"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">/</span><span class="pln">md_size tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip etc tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip </span><span class="kwd">var</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12p</span><span class="pln"> mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pln"> echo </span><span class="str">"16384"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">md_size echo </span><span class="str">"32768"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">/</span><span class="pln">md_size tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip etc tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip </span><span class="kwd">var</span><span class="pln"> cd </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pln"> mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc mkdir </span><span class="pun">-</span><span class="pln">p conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pln"> echo </span><span class="str">"16384"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">md_size echo </span><span class="str">"32768"</span><span class="pun">></span><span class="pln"> conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">/</span><span class="pln">md_size tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="pln">etc</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip etc tar </span><span class="pun">-</span><span class="pln">c </span><span class="pun">-</span><span class="pln">v </span><span class="pun">-</span><span class="pln">f conf</span><span class="pun">/</span><span class="kwd">base</span><span class="pun">/</span><span class="kwd">var</span><span class="pun">.</span><span class="pln">cpio</span><span class="pun">.</span><span class="pln">gz </span><span class="pun">--</span><span class="pln">format cpio </span><span class="pun">--</span><span class="pln">gzip </span><span class="kwd">var</span><span class="pln"> </span><span class="com">#EndIf mounting NFS ro</span>
pfsense dhcp configuration: Control which server boots which FreeBSD image using the DHCP options Common: Enable network booting Next Server: 192.168.10.10 Per Host: Netboot filename: FreeBSD12/boot/pxeboot Root Path: 192.168.10.10:/mnt/ship/pxe/FreeBSD12 Current mappings: lab17:FreeBSD12 / FreeBSD13p lab18:FreeBSD12p / FreeBSD13 zfs mount -a # to mount the ZFS datasets #zfs destroy ship/pxe/FreeBSD11 #zfs destroy ship/pxe/FreeBSD12 #zfs destroy ship/pxe/FreeBSD13 #zfs destroy ship/pxe/FreeBSD12p #zfs destroy ship/pxe/FreeBSD13p #iscsid includes the additional dependency on lockd to ensure NFS locking is in place before this service starts # After changing exports or recreating zfs datasets service mountd reload service ctld reload service tftpd restart Kernel space nfs does not work with jails. There is a userspace nfs alternative https://unfs3.github.io References: #https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-diskless.html #https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-nfs.html #https://www.freebsd.org/doc/en_US.ISO8859-1/articles/remote-install/preparation.html #https://www.freebsd.org/doc/handbook/network-iscsi.html #https://mfsbsd.vx.sk
<span class="typ">On</span><span class="pln"> </span><span class="typ">FreeBSD12</span><span class="pun">(</span><span class="pln">p</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">and</span><span class="pln"> </span><span class="typ">FreeBSD13</span><span class="pun">(</span><span class="pln">p</span><span class="pun">)</span> <span class="com">#First time </span> <span class="pln">zpool create tank </span><span class="pun">/</span><span class="pln">dev</span><span class="pun">/</span><span class="pln">da0</span> <span class="pln">pkg install </span><span class="pun">-</span><span class="pln">y nano bash bash</span><span class="pun">-</span><span class="pln">completion sudo wget htop py37</span><span class="pun">-</span><span class="pln">iocage bhyve</span><span class="pun">-</span><span class="pln">firmware grub2</span><span class="pun">-</span><span class="pln">bhyve iperf3 netperf</span> <span class="pln">iocage list</span> <span class="com">#On FreeBSD13(p)</span> <span class="pln">iocage fetch </span><span class="pun">-</span><span class="pln">r </span><span class="lit">13.0</span><span class="pun">-</span><span class="pln">CURRENT </span><span class="pun">-</span><span class="pln">d ftp</span><span class="pun">/</span><span class="pln">snapshots</span><span class="pun">/</span><span class="pln">amd64</span><span class="pun">/</span> <span class="typ">After</span><span class="pln"> PXE boot</span><span class="pun">:</span> <span class="typ">FreeBSD13</span><span class="pun">(</span><span class="pln">p</span><span class="pun">)</span><span class="pln"> sometimes needs</span><span class="pun">:</span> <span class="pln"> service iscsid start</span> <span class="typ">FreeBSD12</span><span class="pun">(</span><span class="pln">p</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">and</span><span class="pln"> </span><span class="typ">FreeBSD13</span><span class="pun">(</span><span class="pln">p</span><span class="pun">)</span><span class="pln"> need</span><span class="pun">:</span> <span class="pln"> zfs mount </span><span class="pun">-</span><span class="pln">a</span> <span class="pln"> iocage start ALL</span> <span class="com">#manual steps: (not required if in rc.conf)</span> <span class="com">#on both lab17 & lab18:</span> <span class="pln">ifconfig bridge1 create</span> <span class="pln">ifconfig bridge1 ether random addm bge0 up</span> <span class="pln">ifconfig bridge2 create</span> <span class="pln">ifconfig bridge2 ether random addm bge1 up</span> <span class="pln">ifconfig bridge11 create</span> <span class="pln">ifconfig bridge11 ether random addm bnxt0 up</span> <span class="pln">ifconfig bridge12 create</span> <span class="com">#ifconfig bridge12 ether random addm bnxt1 up</span> <span class="com">#ifconfig bridge12 deletem bnxt1</span> <span class="com">#on lab17:</span> <span class="pln">ifconfig bnxt0 </span><span class="lit">10.0</span><span class="pun">.</span><span class="lit">11.17</span><span class="pun">/</span><span class="lit">24</span><span class="pln"> up</span> <span class="pln">ifconfig bnxt1 </span><span class="lit">10.0</span><span class="pun">.</span><span class="lit">12.17</span><span class="pun">/</span><span class="lit">24</span><span class="pln"> up</span> <span class="com">#on lab18:</span> <span class="pln">ifconfig bnxt0 </span><span class="lit">10.0</span><span class="pun">.</span><span class="lit">11.18</span><span class="pun">/</span><span class="lit">24</span><span class="pln"> up</span> <span class="pln">ifconfig bnxt1 </span><span class="lit">10.0</span><span class="pun">.</span><span class="lit">12.18</span><span class="pun">/</span><span class="lit">24</span><span class="pln"> up</span> <span class="com">#iocage installation:</span> <span class="pln">mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root</span> <span class="pln">cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root</span> <span class="pln">rpl </span><span class="str">"192.168.0"</span><span class="pln"> </span><span class="str">"192.168.10"</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh</span> <span class="pln">rpl </span><span class="str">"fd01::"</span><span class="pln"> </span><span class="str">"fd0a::"</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh</span> <span class="pln">cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12</span> <span class="pln">cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span> <span class="pln">cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD12p</span> <span class="pln">cp </span><span class="pun">-</span><span class="pln">r </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span> <span class="pln">rpl </span><span class="str">"12.2-RELEASE"</span><span class="pln"> </span><span class="str">"13.0-CURRENT"</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh</span> <span class="pln">rpl </span><span class="str">"12.2-RELEASE"</span><span class="pln"> </span><span class="str">"13.0-CURRENT"</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13</span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">update</span><span class="pun">.</span><span class="pln">sh</span> <span class="pln">rpl </span><span class="str">"12.2-RELEASE"</span><span class="pln"> </span><span class="str">"13.0-CURRENT"</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh</span> <span class="pln">rpl </span><span class="str">"12.2-RELEASE"</span><span class="pln"> </span><span class="str">"13.0-CURRENT"</span><span class="pln"> </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="typ">FreeBSD13p</span><span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">update</span><span class="pun">.</span><span class="pln">sh</span> <span class="pln">rmdir </span><span class="pun">-</span><span class="pln">rf </span><span class="pun">/</span><span class="pln">mnt</span><span class="pun">/</span><span class="pln">ship</span><span class="pun">/</span><span class="pln">pxe</span><span class="pun">/</span><span class="pln">root</span> <span class="com">#Create jail 117 on lab17</span> <span class="pln">bash</span> <span class="kwd">export</span><span class="pln"> JAIL</span><span class="pun">=</span><span class="pln">t</span><span class="pun">-</span><span class="lit">117</span> <span class="kwd">export</span><span class="pln"> JAILHOSTNAME</span><span class="pun">=</span><span class="pln">test</span><span class="pun">-</span><span class="lit">117</span> <span class="kwd">export</span><span class="pln"> JAILDOMAIN</span><span class="pun">=</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span> <span class="kwd">export</span><span class="pln"> JAILIP</span><span class="pun">=</span><span class="lit">117</span> <span class="kwd">export</span><span class="pln"> JAILUSER</span><span class="pun">=</span><span class="pln">X</span> <span class="kwd">export</span><span class="pln"> JAILUSERID</span><span class="pun">=</span><span class="lit">1000</span> <span class="kwd">export</span><span class="pln"> JAILUSERVNC</span><span class="pun">=</span><span class="kwd">false</span> <span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh $JAIL $JAILHOSTNAME $JAILDOMAIN $JAILIP $JAILUSER $JAILUSERID $JAILUSERVNC</span> <span class="pln">iocage </span><span class="kwd">exec</span><span class="pln"> $JAIL </span><span class="str">"pkg install -y iperf3 netperf"</span> <span class="pln">iocage </span><span class="kwd">set</span><span class="pln"> interfaces</span><span class="pun">=</span><span class="pln">vnet0</span><span class="pun">:</span><span class="pln">bridge1</span><span class="pun">,</span><span class="pln">vnet1</span><span class="pun">:</span><span class="pln">bridge11</span><span class="pun">,</span><span class="pln">vnet2</span><span class="pun">:</span><span class="pln">bridge12 ip4_addr</span><span class="pun">=</span><span class="str">"vnet0|192.168.10.$JAILIP/24,vnet1|10.0.11.$JAILIP/24,vnet2|10.0.12.$JAILIP/24"</span><span class="pln"> ip6_addr</span><span class="pun">=</span><span class="str">"vnet0|fd0a::$JAILIP/64,vnet1|2001:db8:11::$JAILIP/64,vnet2|2001:db8:12::$JAILIP/64"</span><span class="pln"> $JAIL</span> <span class="pln">iocage restart $JAIL</span> <span class="com">#Create jail 127 on lab17</span> <span class="pln">bash</span> <span class="kwd">export</span><span class="pln"> JAIL</span><span class="pun">=</span><span class="pln">t</span><span class="pun">-</span><span class="lit">127</span> <span class="kwd">export</span><span class="pln"> JAILHOSTNAME</span><span class="pun">=</span><span class="pln">test</span><span class="pun">-</span><span class="lit">127</span> <span class="kwd">export</span><span class="pln"> JAILDOMAIN</span><span class="pun">=</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span> <span class="kwd">export</span><span class="pln"> JAILIP</span><span class="pun">=</span><span class="lit">127</span> <span class="kwd">export</span><span class="pln"> JAILUSER</span><span class="pun">=</span><span class="pln">X</span> <span class="kwd">export</span><span class="pln"> JAILUSERID</span><span class="pun">=</span><span class="lit">1000</span> <span class="kwd">export</span><span class="pln"> JAILUSERVNC</span><span class="pun">=</span><span class="kwd">false</span> <span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh $JAIL $JAILHOSTNAME $JAILDOMAIN $JAILIP $JAILUSER $JAILUSERID $JAILUSERVNC</span> <span class="pln">iocage </span><span class="kwd">exec</span><span class="pln"> $JAIL </span><span class="str">"pkg install -y iperf3 netperf"</span> <span class="pln">iocage </span><span class="kwd">set</span><span class="pln"> interfaces</span><span class="pun">=</span><span class="pln">vnet0</span><span class="pun">:</span><span class="pln">bridge1</span><span class="pun">,</span><span class="pln">vnet1</span><span class="pun">:</span><span class="pln">bridge11</span><span class="pun">,</span><span class="pln">vnet2</span><span class="pun">:</span><span class="pln">bridge12 ip4_addr</span><span class="pun">=</span><span class="str">"vnet0|192.168.10.$JAILIP/24,vnet1|10.0.11.$JAILIP/24,vnet2|10.0.12.$JAILIP/24"</span><span class="pln"> ip6_addr</span><span class="pun">=</span><span class="str">"vnet0|fd0a::$JAILIP/64,vnet1|2001:db8:11::$JAILIP/64,vnet2|2001:db8:12::$JAILIP/64"</span><span class="pln"> $JAIL</span> <span class="pln">iocage restart $JAIL</span> <span class="com">#Create jail 118 on lab18</span> <span class="pln">bash</span> <span class="kwd">export</span><span class="pln"> JAIL</span><span class="pun">=</span><span class="pln">t</span><span class="pun">-</span><span class="lit">118</span> <span class="kwd">export</span><span class="pln"> JAILHOSTNAME</span><span class="pun">=</span><span class="pln">test</span><span class="pun">-</span><span class="lit">118</span> <span class="kwd">export</span><span class="pln"> JAILDOMAIN</span><span class="pun">=</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span> <span class="kwd">export</span><span class="pln"> JAILIP</span><span class="pun">=</span><span class="lit">118</span> <span class="kwd">export</span><span class="pln"> JAILUSER</span><span class="pun">=</span><span class="pln">X</span> <span class="kwd">export</span><span class="pln"> JAILUSERID</span><span class="pun">=</span><span class="lit">1000</span> <span class="kwd">export</span><span class="pln"> JAILUSERVNC</span><span class="pun">=</span><span class="kwd">false</span> <span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh $JAIL $JAILHOSTNAME $JAILDOMAIN $JAILIP $JAILUSER $JAILUSERID $JAILUSERVNC</span> <span class="pln">iocage </span><span class="kwd">exec</span><span class="pln"> $JAIL </span><span class="str">"pkg install -y iperf3 netperf"</span> <span class="pln">iocage </span><span class="kwd">set</span><span class="pln"> interfaces</span><span class="pun">=</span><span class="pln">vnet0</span><span class="pun">:</span><span class="pln">bridge1</span><span class="pun">,</span><span class="pln">vnet1</span><span class="pun">:</span><span class="pln">bridge11</span><span class="pun">,</span><span class="pln">vnet2</span><span class="pun">:</span><span class="pln">bridge12 ip4_addr</span><span class="pun">=</span><span class="str">"vnet0|192.168.10.$JAILIP/24,vnet1|10.0.11.$JAILIP/24,vnet2|10.0.12.$JAILIP/24"</span><span class="pln"> ip6_addr</span><span class="pun">=</span><span class="str">"vnet0|fd0a::$JAILIP/64,vnet1|2001:db8:11::$JAILIP/64,vnet2|2001:db8:12::$JAILIP/64"</span><span class="pln"> $JAIL</span> <span class="pln">iocage restart $JAIL</span> <span class="com">#Create jail 128 on lab18</span> <span class="pln">bash</span> <span class="kwd">export</span><span class="pln"> JAIL</span><span class="pun">=</span><span class="pln">t</span><span class="pun">-</span><span class="lit">128</span> <span class="kwd">export</span><span class="pln"> JAILHOSTNAME</span><span class="pun">=</span><span class="pln">test</span><span class="pun">-</span><span class="lit">128</span> <span class="kwd">export</span><span class="pln"> JAILDOMAIN</span><span class="pun">=</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span> <span class="kwd">export</span><span class="pln"> JAILIP</span><span class="pun">=</span><span class="lit">128</span> <span class="kwd">export</span><span class="pln"> JAILUSER</span><span class="pun">=</span><span class="pln">X</span> <span class="kwd">export</span><span class="pln"> JAILUSERID</span><span class="pun">=</span><span class="lit">1000</span> <span class="kwd">export</span><span class="pln"> JAILUSERVNC</span><span class="pun">=</span><span class="kwd">false</span> <span class="pun">/</span><span class="pln">root</span><span class="pun">/</span><span class="typ">FreeBSD</span><span class="pun">/</span><span class="pln">jails</span><span class="pun">/</span><span class="pln">create</span><span class="pun">.</span><span class="pln">sh $JAIL $JAILHOSTNAME $JAILDOMAIN $JAILIP $JAILUSER $JAILUSERID $JAILUSERVNC</span> <span class="pln">iocage </span><span class="kwd">exec</span><span class="pln"> $JAIL </span><span class="str">"pkg install -y iperf3 netperf"</span> <span class="pln">iocage </span><span class="kwd">set</span><span class="pln"> interfaces</span><span class="pun">=</span><span class="pln">vnet0</span><span class="pun">:</span><span class="pln">bridge1</span><span class="pun">,</span><span class="pln">vnet1</span><span class="pun">:</span><span class="pln">bridge11</span><span class="pun">,</span><span class="pln">vnet2</span><span class="pun">:</span><span class="pln">bridge12 ip4_addr</span><span class="pun">=</span><span class="str">"vnet0|192.168.10.$JAILIP/24,vnet1|10.0.11.$JAILIP/24,vnet2|10.0.12.$JAILIP/24"</span><span class="pln"> ip6_addr</span><span class="pun">=</span><span class="str">"vnet0|fd0a::$JAILIP/64,vnet1|2001:db8:11::$JAILIP/64,vnet2|2001:db8:12::$JAILIP/64"</span><span class="pln"> $JAIL</span> <span class="pln">iocage restart $JAIL</span> <span class="com">#Notes:</span> <span class="pln">ipfw </span><span class="kwd">module</span><span class="pln"> </span><span class="kwd">is</span><span class="pln"> </span><span class="kwd">not</span><span class="pln"> loaded </span><span class="kwd">and</span><span class="pln"> configured on the test bed</span> <span class="pln">why ether random </span><span class="kwd">when</span><span class="pln"> configuring bridge</span><span class="pun">:</span> <span class="pln"> </span><span class="typ">The</span><span class="pln"> if_bridge </span><span class="kwd">interface</span><span class="pln"> randomly chooses a link </span><span class="pun">(</span><span class="pln">MAC</span><span class="pun">)</span><span class="pln"> address </span><span class="kwd">in</span><span class="pln"> the</span> <span class="pln"> range reserved </span><span class="kwd">for</span><span class="pln"> locally administered addresses </span><span class="kwd">when</span><span class="pln"> it </span><span class="kwd">is</span><span class="pln"> created</span><span class="pun">.</span> <span class="pln"> </span><span class="typ">This</span><span class="pln"> address </span><span class="kwd">is</span><span class="pln"> guaranteed to be unique only across all if_bridge inter</span><span class="pun">-</span> <span class="pln"> faces on the </span><span class="kwd">local</span><span class="pln"> machine</span><span class="pun">.</span><span class="pln"> </span><span class="typ">Thus</span><span class="pln"> you can theoretically have two bridges</span> <span class="pln"> on the different machines </span><span class="kwd">with</span><span class="pln"> the same link addresses</span><span class="pun">.</span> <span class="pln">ssh</span><span class="pun">-</span><span class="pln">keygen </span><span class="pun">-</span><span class="pln">R lab17</span><span class="pun">.</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span><span class="pun">;</span><span class="pln"> ssh </span><span class="pun">-</span><span class="pln">o </span><span class="str">"StrictHostKeyChecking no"</span><span class="pln"> root@lab17</span><span class="pun">.</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span> <span class="pln">ssh</span><span class="pun">-</span><span class="pln">keygen </span><span class="pun">-</span><span class="pln">R lab18</span><span class="pun">.</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span><span class="pun">;</span><span class="pln"> ssh </span><span class="pun">-</span><span class="pln">o </span><span class="str">"StrictHostKeyChecking no"</span><span class="pln"> root@lab18</span><span class="pun">.</span><span class="pln">diyit</span><span class="pun">.</span><span class="pln">org</span> <span class="pln">THE TEST TRIGGER</span><span class="pun">:</span><span class="pln"> https</span><span class="pun">:</span><span class="com">//people.freebsd.org/~kp/if_bridge/stable_12/</span>
PerfTest.txt 13-current Vs 12.1-release - simple iperf test #on both lab17 & lab18: ifconfig bridge1 create ifconfig bridge1 addm bge0 up ifconfig bridge2 create ifconfig bridge2 addm bge1 up ifconfig bridge11 create ifconfig bridge11 addm bnxt0 up ifconfig bridge12 create #ifconfig bridge12 addm bnxt1 up #ifconfig bridge12 deletem bnxt1 #on lab17: ifconfig bnxt0 10.0.11.17/24 up ifconfig bnxt1 10.0.12.17/24 up #on lab18: ifconfig bnxt0 10.0.11.18/24 up ifconfig bnxt1 10.0.12.18/24 up lab17 & lab18 servers are connected over 25G SFP28 links via a switch. bnxt0 interfaces are in VLAN 11 and bnxt1 interfaces are in VLAN 12 root@lab17:~ # uname -a FreeBSD lab17 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r361567: Thu May 28 04:59:08 UTC 2020 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 root@lab17:~ # root@lab18:~ # uname -a FreeBSD lab18 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r361567: Thu May 28 04:59:08 UTC 2020 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 root@lab18:~ # Bridged Interfaces: server to server ----------------------------------------------------------- Accepted connection from 10.0.11.18, port 57363 [ 5] local 10.0.11.17 port 5201 connected to 10.0.11.18 port 20451 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 152 MBytes 1.28 Gbits/sec [ 5] 1.00-2.00 sec 258 MBytes 2.16 Gbits/sec [ 5] 2.00-3.00 sec 262 MBytes 2.20 Gbits/sec [ 5] 3.00-4.00 sec 270 MBytes 2.27 Gbits/sec [ 5] 4.00-5.00 sec 278 MBytes 2.33 Gbits/sec [ 5] 5.00-6.00 sec 281 MBytes 2.36 Gbits/sec [ 5] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec [ 5] 7.00-8.00 sec 276 MBytes 2.32 Gbits/sec [ 5] 8.00-9.00 sec 252 MBytes 2.12 Gbits/sec [ 5] 9.00-10.00 sec 265 MBytes 2.22 Gbits/sec [ 5] 10.00-10.00 sec 113 KBytes 2.56 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec receiver ----------------------------------------------------------- Connecting to host 10.0.11.17, port 5201 [ 5] local 10.0.11.18 port 20451 connected to 10.0.11.17 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 153 MBytes 1.28 Gbits/sec 7 218 KBytes [ 5] 1.00-2.00 sec 258 MBytes 2.16 Gbits/sec 15 299 KBytes [ 5] 2.00-3.00 sec 262 MBytes 2.20 Gbits/sec 12 242 KBytes [ 5] 3.00-4.00 sec 270 MBytes 2.27 Gbits/sec 10 229 KBytes [ 5] 4.00-5.00 sec 277 MBytes 2.33 Gbits/sec 13 252 KBytes [ 5] 5.00-6.00 sec 281 MBytes 2.36 Gbits/sec 15 286 KBytes [ 5] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec 12 321 KBytes [ 5] 7.00-8.00 sec 276 MBytes 2.32 Gbits/sec 11 336 KBytes [ 5] 8.00-9.00 sec 253 MBytes 2.12 Gbits/sec 135 348 KBytes [ 5] 9.00-10.00 sec 265 MBytes 2.22 Gbits/sec 13 308 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.52 GBytes 2.16 Gbits/sec 243 sender [ 5] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec receiver iperf Done. Direct Interfaces: server to server ----------------------------------------------------------- Accepted connection from 10.0.12.18, port 24137 [ 5] local 10.0.12.17 port 5201 connected to 10.0.12.18 port 19922 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 403 MBytes 3.38 Gbits/sec [ 5] 1.00-2.00 sec 757 MBytes 6.35 Gbits/sec [ 5] 2.00-3.00 sec 758 MBytes 6.35 Gbits/sec [ 5] 3.00-4.00 sec 760 MBytes 6.38 Gbits/sec [ 5] 4.00-5.00 sec 759 MBytes 6.37 Gbits/sec [ 5] 5.00-6.00 sec 760 MBytes 6.37 Gbits/sec [ 5] 6.00-7.00 sec 758 MBytes 6.36 Gbits/sec [ 5] 7.00-8.00 sec 759 MBytes 6.37 Gbits/sec [ 5] 8.00-9.00 sec 759 MBytes 6.37 Gbits/sec [ 5] 9.00-10.00 sec 759 MBytes 6.37 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 7.06 GBytes 6.07 Gbits/sec receiver ----------------------------------------------------------- root@lab18:~ # iperf3 -c 10.0.12.17 Connecting to host 10.0.12.17, port 5201 [ 5] local 10.0.12.18 port 19922 connected to 10.0.12.17 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 404 MBytes 3.39 Gbits/sec 14 268 KBytes [ 5] 1.00-2.00 sec 757 MBytes 6.35 Gbits/sec 2 354 KBytes [ 5] 2.00-3.00 sec 758 MBytes 6.35 Gbits/sec 6 259 KBytes [ 5] 3.00-4.00 sec 760 MBytes 6.38 Gbits/sec 4 349 KBytes [ 5] 4.00-5.00 sec 759 MBytes 6.37 Gbits/sec 6 254 KBytes [ 5] 5.00-6.00 sec 760 MBytes 6.37 Gbits/sec 4 344 KBytes [ 5] 6.00-7.00 sec 758 MBytes 6.36 Gbits/sec 6 247 KBytes [ 5] 7.00-8.00 sec 759 MBytes 6.37 Gbits/sec 4 338 KBytes [ 5] 8.00-9.00 sec 759 MBytes 6.37 Gbits/sec 15 269 KBytes [ 5] 9.00-10.00 sec 759 MBytes 6.37 Gbits/sec 4 356 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 7.06 GBytes 6.07 Gbits/sec 65 sender [ 5] 0.00-10.00 sec 7.06 GBytes 6.07 Gbits/sec receiver iperf Done. jail to jail on same server ----------------------------------------------------------- Accepted connection from 10.0.11.117, port 43039 [ 5] local 10.0.11.127 port 5201 connected to 10.0.11.117 port 55213 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 173 MBytes 1.45 Gbits/sec [ 5] 1.00-2.00 sec 182 MBytes 1.52 Gbits/sec [ 5] 2.00-3.00 sec 180 MBytes 1.51 Gbits/sec [ 5] 3.00-4.00 sec 185 MBytes 1.55 Gbits/sec [ 5] 4.00-5.00 sec 184 MBytes 1.54 Gbits/sec [ 5] 5.00-6.00 sec 181 MBytes 1.52 Gbits/sec [ 5] 6.00-7.00 sec 154 MBytes 1.29 Gbits/sec [ 5] 7.00-8.00 sec 133 MBytes 1.12 Gbits/sec [ 5] 8.00-9.00 sec 136 MBytes 1.14 Gbits/sec [ 5] 9.00-10.00 sec 137 MBytes 1.15 Gbits/sec [ 5] 10.00-10.00 sec 771 KBytes 1.70 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 1.61 GBytes 1.38 Gbits/sec receiver ----------------------------------------------------------- test-117 ~ # iperf3 -c 10.0.11.127 Connecting to host 10.0.11.127, port 5201 [ 5] local 10.0.11.117 port 55213 connected to 10.0.11.127 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 175 MBytes 1.46 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 182 MBytes 1.52 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 179 MBytes 1.50 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 185 MBytes 1.55 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 184 MBytes 1.55 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 181 MBytes 1.52 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 153 MBytes 1.29 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 133 MBytes 1.12 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 136 MBytes 1.14 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 137 MBytes 1.15 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.61 GBytes 1.38 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 1.61 GBytes 1.38 Gbits/sec receiver iperf Done. jail to jail on different servers ----------------------------------------------------------- Accepted connection from 10.0.12.128, port 56410 [ 5] local 10.0.12.127 port 5201 connected to 10.0.12.128 port 61265 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 126 MBytes 1.06 Gbits/sec [ 5] 1.00-2.00 sec 192 MBytes 1.61 Gbits/sec [ 5] 2.00-3.00 sec 173 MBytes 1.45 Gbits/sec [ 5] 3.00-4.00 sec 156 MBytes 1.31 Gbits/sec [ 5] 4.00-5.00 sec 170 MBytes 1.43 Gbits/sec [ 5] 5.00-6.00 sec 175 MBytes 1.46 Gbits/sec [ 5] 6.00-7.00 sec 173 MBytes 1.45 Gbits/sec [ 5] 7.00-8.00 sec 182 MBytes 1.53 Gbits/sec [ 5] 8.00-9.00 sec 195 MBytes 1.64 Gbits/sec [ 5] 9.00-10.00 sec 188 MBytes 1.58 Gbits/sec [ 5] 10.00-10.02 sec 2.47 MBytes 1.37 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.02 sec 1.69 GBytes 1.45 Gbits/sec receiver ----------------------------------------------------------- test-128 ~ # iperf3 -c 10.0.12.127 Connecting to host 10.0.12.127, port 5201 [ 5] local 10.0.12.128 port 61265 connected to 10.0.12.127 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 129 MBytes 1.08 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 192 MBytes 1.61 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 173 MBytes 1.45 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 155 MBytes 1.30 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 171 MBytes 1.43 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 174 MBytes 1.46 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 173 MBytes 1.45 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 183 MBytes 1.54 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 195 MBytes 1.63 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 188 MBytes 1.58 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.69 GBytes 1.45 Gbits/sec 0 sender [ 5] 0.00-10.02 sec 1.69 GBytes 1.45 Gbits/sec receiver iperf Done. root@lab17:~ # uname -a FreeBSD lab17 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC amd64 root@lab17:~ # root@lab18:~ # uname -a FreeBSD lab18 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC amd64 root@lab18:~ # Bridged Interfaces: server to server ----------------------------------------------------------- Accepted connection from 10.0.11.18, port 61273 [ 5] local 10.0.11.17 port 5201 connected to 10.0.11.18 port 17033 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 310 MBytes 2.60 Gbits/sec [ 5] 1.00-2.00 sec 551 MBytes 4.62 Gbits/sec [ 5] 2.00-3.00 sec 589 MBytes 4.94 Gbits/sec [ 5] 3.00-4.00 sec 591 MBytes 4.96 Gbits/sec [ 5] 4.00-5.00 sec 592 MBytes 4.97 Gbits/sec [ 5] 5.00-6.00 sec 594 MBytes 4.99 Gbits/sec [ 5] 6.00-7.00 sec 594 MBytes 4.98 Gbits/sec [ 5] 7.00-8.00 sec 593 MBytes 4.98 Gbits/sec [ 5] 8.00-9.00 sec 593 MBytes 4.97 Gbits/sec [ 5] 9.00-10.00 sec 593 MBytes 4.98 Gbits/sec [ 5] 10.00-10.00 sec 337 KBytes 4.93 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 5.47 GBytes 4.70 Gbits/sec receiver ----------------------------------------------------------- root@lab18:~ # iperf3 -c 10.0.11.17 Connecting to host 10.0.11.17, port 5201 [ 5] local 10.0.11.18 port 17033 connected to 10.0.11.17 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 312 MBytes 2.61 Gbits/sec 28 390 KBytes [ 5] 1.00-2.00 sec 551 MBytes 4.63 Gbits/sec 78 254 KBytes [ 5] 2.00-3.00 sec 589 MBytes 4.94 Gbits/sec 27 191 KBytes [ 5] 3.00-4.00 sec 591 MBytes 4.96 Gbits/sec 32 335 KBytes [ 5] 4.00-5.00 sec 592 MBytes 4.96 Gbits/sec 28 304 KBytes [ 5] 5.00-6.00 sec 595 MBytes 4.99 Gbits/sec 29 275 KBytes [ 5] 6.00-7.00 sec 593 MBytes 4.98 Gbits/sec 28 238 KBytes [ 5] 7.00-8.00 sec 593 MBytes 4.98 Gbits/sec 33 366 KBytes [ 5] 8.00-9.00 sec 593 MBytes 4.97 Gbits/sec 26 348 KBytes [ 5] 9.00-10.00 sec 593 MBytes 4.98 Gbits/sec 31 315 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 5.47 GBytes 4.70 Gbits/sec 340 sender [ 5] 0.00-10.00 sec 5.47 GBytes 4.70 Gbits/sec receiver iperf Done. Direct Interfaces: server to server ----------------------------------------------------------- Accepted connection from 10.0.12.18, port 24603 [ 5] local 10.0.12.17 port 5201 connected to 10.0.12.18 port 30690 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 997 MBytes 8.36 Gbits/sec [ 5] 1.00-2.00 sec 1.82 GBytes 15.6 Gbits/sec [ 5] 2.00-3.00 sec 1.83 GBytes 15.7 Gbits/sec [ 5] 3.00-4.00 sec 1.83 GBytes 15.7 Gbits/sec [ 5] 4.00-5.00 sec 1.83 GBytes 15.7 Gbits/sec [ 5] 5.00-6.00 sec 1.83 GBytes 15.7 Gbits/sec [ 5] 6.00-7.00 sec 1.83 GBytes 15.8 Gbits/sec [ 5] 7.00-8.00 sec 1.84 GBytes 15.8 Gbits/sec [ 5] 8.00-9.00 sec 1.83 GBytes 15.7 Gbits/sec [ 5] 9.00-10.00 sec 1.83 GBytes 15.7 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 17.4 GBytes 15.0 Gbits/sec receiver ----------------------------------------------------------- root@lab18:~ # iperf3 -c 10.0.12.17 Connecting to host 10.0.12.17, port 5201 [ 5] local 10.0.12.18 port 30690 connected to 10.0.12.17 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 998 MBytes 8.37 Gbits/sec 16 386 KBytes [ 5] 1.00-2.00 sec 1.82 GBytes 15.6 Gbits/sec 38 375 KBytes [ 5] 2.00-3.00 sec 1.83 GBytes 15.7 Gbits/sec 16 379 KBytes [ 5] 3.00-4.00 sec 1.83 GBytes 15.7 Gbits/sec 11 1.41 KBytes [ 5] 4.00-5.00 sec 1.83 GBytes 15.7 Gbits/sec 16 205 KBytes [ 5] 5.00-6.00 sec 1.83 GBytes 15.7 Gbits/sec 8 212 KBytes [ 5] 6.00-7.00 sec 1.83 GBytes 15.8 Gbits/sec 9 239 KBytes [ 5] 7.00-8.00 sec 1.84 GBytes 15.8 Gbits/sec 8 267 KBytes [ 5] 8.00-9.00 sec 1.83 GBytes 15.7 Gbits/sec 7 289 KBytes [ 5] 9.00-10.00 sec 1.83 GBytes 15.7 Gbits/sec 6 314 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 17.4 GBytes 15.0 Gbits/sec 135 sender [ 5] 0.00-10.00 sec 17.4 GBytes 15.0 Gbits/sec receiver iperf Done. jail to jail on same server ----------------------------------------------------------- Accepted connection from 10.0.11.117, port 31176 [ 5] local 10.0.11.127 port 5201 connected to 10.0.11.117 port 38861 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 203 MBytes 1.71 Gbits/sec [ 5] 1.00-2.00 sec 342 MBytes 2.87 Gbits/sec [ 5] 2.00-3.00 sec 343 MBytes 2.88 Gbits/sec [ 5] 3.00-4.00 sec 315 MBytes 2.64 Gbits/sec [ 5] 4.00-5.00 sec 224 MBytes 1.88 Gbits/sec [ 5] 5.00-6.00 sec 233 MBytes 1.96 Gbits/sec [ 5] 6.00-7.00 sec 241 MBytes 2.02 Gbits/sec [ 5] 7.00-8.00 sec 243 MBytes 2.04 Gbits/sec [ 5] 8.00-9.00 sec 243 MBytes 2.04 Gbits/sec [ 5] 9.00-10.00 sec 243 MBytes 2.04 Gbits/sec [ 5] 10.00-10.00 sec 851 KBytes 3.64 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 2.57 GBytes 2.21 Gbits/sec receiver ----------------------------------------------------------- test-117 ~ # iperf3 -c 10.0.11.127 Connecting to host 10.0.11.127, port 5201 [ 5] local 10.0.11.117 port 38861 connected to 10.0.11.127 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 204 MBytes 1.71 Gbits/sec 0 495 KBytes [ 5] 1.00-2.00 sec 342 MBytes 2.87 Gbits/sec 0 995 KBytes [ 5] 2.00-3.00 sec 343 MBytes 2.88 Gbits/sec 0 1.38 MBytes [ 5] 3.00-4.00 sec 315 MBytes 2.64 Gbits/sec 0 1.67 MBytes [ 5] 4.00-5.00 sec 224 MBytes 1.88 Gbits/sec 0 1.77 MBytes [ 5] 5.00-6.00 sec 233 MBytes 1.96 Gbits/sec 0 1.77 MBytes [ 5] 6.00-7.00 sec 241 MBytes 2.02 Gbits/sec 0 1.77 MBytes [ 5] 7.00-8.00 sec 243 MBytes 2.04 Gbits/sec 0 1.77 MBytes [ 5] 8.00-9.00 sec 243 MBytes 2.04 Gbits/sec 0 1.77 MBytes [ 5] 9.00-10.00 sec 243 MBytes 2.04 Gbits/sec 0 1.77 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.57 GBytes 2.21 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 2.57 GBytes 2.21 Gbits/sec receiver iperf Done. jail to jail on different servers ----------------------------------------------------------- Accepted connection from 10.0.12.128, port 25861 [ 5] local 10.0.12.127 port 5201 connected to 10.0.12.128 port 18803 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 221 MBytes 1.85 Gbits/sec [ 5] 1.00-2.00 sec 340 MBytes 2.85 Gbits/sec [ 5] 2.00-3.00 sec 344 MBytes 2.89 Gbits/sec [ 5] 3.00-4.00 sec 392 MBytes 3.29 Gbits/sec [ 5] 4.00-5.00 sec 341 MBytes 2.86 Gbits/sec [ 5] 5.00-6.00 sec 262 MBytes 2.20 Gbits/sec [ 5] 6.00-7.00 sec 390 MBytes 3.27 Gbits/sec [ 5] 7.00-8.00 sec 392 MBytes 3.29 Gbits/sec [ 5] 8.00-9.00 sec 397 MBytes 3.33 Gbits/sec [ 5] 9.00-10.00 sec 398 MBytes 3.34 Gbits/sec [ 5] 10.00-10.00 sec 1014 KBytes 4.11 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 3.40 GBytes 2.92 Gbits/sec receiver ----------------------------------------------------------- test-128 ~ # iperf3 -c 10.0.12.127 Connecting to host 10.0.12.127, port 5201 [ 5] local 10.0.12.128 port 18803 connected to 10.0.12.127 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 221 MBytes 1.86 Gbits/sec 0 495 KBytes [ 5] 1.00-2.00 sec 340 MBytes 2.85 Gbits/sec 0 1007 KBytes [ 5] 2.00-3.00 sec 344 MBytes 2.89 Gbits/sec 0 1.47 MBytes [ 5] 3.00-4.00 sec 392 MBytes 3.29 Gbits/sec 0 1.77 MBytes [ 5] 4.00-5.00 sec 341 MBytes 2.86 Gbits/sec 0 1.77 MBytes [ 5] 5.00-6.00 sec 262 MBytes 2.20 Gbits/sec 0 1.77 MBytes [ 5] 6.00-7.00 sec 390 MBytes 3.27 Gbits/sec 0 1.77 MBytes [ 5] 7.00-8.00 sec 392 MBytes 3.29 Gbits/sec 0 1.77 MBytes [ 5] 8.00-9.00 sec 397 MBytes 3.33 Gbits/sec 0 1.77 MBytes [ 5] 9.00-10.00 sec 399 MBytes 3.34 Gbits/sec 0 1.77 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.40 GBytes 2.92 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 3.40 GBytes 2.92 Gbits/sec receiver iperf Done.