Yocto Installer

•  TODO

The compilation and installation of the Yocto version of SEAPATH is entirely described on the GitHub repository seapath/yocto-bsp.

The build of Yocto based SEAPATH require significant ressources because the entire Linux distribution is built from the source code.

In order to build efficiently the SEAPATH project, we recommend not to use Virtual Machine. The Yocto project will ensure to multi-thread your build, so try to use a build machine with many CPU cores.

Here is a discussion on the Yocto Project mailing list: https://lists.yoctoproject.org/g/yocto/topic/72047879#48815

Tips for building

• About 250GB is needed for building SEAPATH.

• A USB attached storage may be too slow to be practical for a successful build.

• Ensure you use an ext 2/ ext3 / ext4 filesystem for the build directory. NTFS will not work.

• Watch out with only manually deleting the /tmp/work directory. Instead delete the whole tmp directory.

• When deleting the tmp, it may take a very long time, and might cause rm -rf to fail with an error. find . -delete will work better, as it will not try to index all files before deleting them.

Debian Installer

To install the cluster, you need to generate an ISO, based on Debian 11, for each host with this repository here.

...

See the below section for more details on the configuration file.

Configuration

In the configuration file, you must define these variables:

• FAI_ALLOW_UNSIGNED: Boolean to allow installation of packages from unsigned repositories (0 => true)

• UTC: Boolean to set the system clock to UTC (possible values: yes or no)

• TIMEZONE: Time to choose

• KEYMAP: Keyboard translation to choose

• ROOTPW: Crypted password for root

• STOP_ON_ERROR: TODO

• MAXPACKAGES: TODO

• username: ID of the user account to be created

• USERPW: Crypted password for the user account to be created

• usernameansible: ID of the ansible account to be created

• myrootkey: TODO

• myuserkey: TODO

• ansiblekey: TODO

• apt_cdn: TODO

• REMOTENIC: Network interface to be set

• REMOTEADDR: IP address to be set on REMOTENIC with the mask

• REMOTEGW: IP address for the gateway to be set on REMOTENIC

However, all host will be with the same IP address.

Prerequisite

When the host is installed, the ansible/playbooks/cluster_setup_prerequisdebian.yaml need to launch to finish the installation.

The inventory must define these variables to run the playbook:

...

...

admin_ip_addr: IP address for SNMP

...

cpumachinesnort: Range of allowed CPUs for no RT machines

...

cpumachines: Range of allowed CPUs for machines (RT and no RT)

...

cpumachinesrt: Range of allowed CPUs for RT machines

...

cpuovs: Range of allowed CPUs for OpenVSwitch

...

cpusystem: Range of allowed CPUs for the system

...

cpuuser: Range of allowed CPUs for the user

...

main_disk: Main disk device to observe his temperature

...

workqueuemask: The negation of the irqmask (= ~irqmask)

In this part, the playbook define the scheduling and the prioritization (see the section).

Hardening

The ansible/playbooks/cluster_setup_hardening_debian.yaml playbook enables system hardening and the ansible/playbooks/cluster_setup_unhardening_debian.yaml playbook disables it.

The hardened elements are:

• the kernel with the parameters of the command line (see below section), the sysfs and modules;
• the GRUB;
• the systemd services;
• adding of bash profiles;
• SSH server;
• adding of sudo rules;
• the shadow password suite configuration;
• the secure tty;
• the audit daemon.

Kernel

The project uses a real-time kernel, the Linux kernel with the PREEMPT_RT patch. So, he needs to have some parameters as:

• cpufreq.default_governor=performance: Use the performance governor by default (more details here).
• hugepagesz=1G: Uses 1 giga-bytes for HugeTLB pages (more details here).
• intel_pstate=disable: Disables the intel_pstate as the default scaling driver for supported processors (more details here).
• isolcpus=nohz,domain,managed_irq: nohz to disable the tick when a single task runs; domain to isolate from the general SMP balancing and scheduling algorithms; managed_irq to isolate from being targeted by managed. See the Scheduling and priorization section.
• no_debug_object: Disables object debugging.
• nosoftlockup: Disable the soft-lockup detector (more details here).
• processors.max_cstate=1 and intel_idle.max_cstate=1: Discards of all the idle states deeper than idle state 1, for the acpi_idle and intel_idle drivers, respectively (more details here).
• rcu_nocbs: See the Scheduling and priorization section.
• rcu_nocb_poll: Make the kthreads poll for callbacks.
• rcutree.kthread_prio=10: Set the SCHED_FIFO priority of the RCU per-CPU kthreads.
• skew_tick=1: Helps to smooth jitter on systems with latency-sensitive applications running.
• tsc=reliable: Disables clocksource verification at runtime, as well as the stability checks done at bootup.

In the hardening system, the kernel has these parameters:

• init_on_alloc=1: Fill newly allocated pages and heap objects with zeroes.
• init_on_free=1: Fill freed pages and heap objects with zeroes.
• slab_nomerge: Disable merging of slabs with similar size.
• pti=on: Enable the control Page Table Isolation of user and kernel address spaces.
• slub_debug=ZF: Enable red zoning (Z) and zanity checks (F) on for all slabs (more details here).
• randomize_kstack_offset=on: Enable kernel stack offset randomization.
• slab_common.usercopy_fallback=N:
• iommu=pt: Get best performance using the SR-IOV (TODO).
• security=yama: Use the yama security module to enable at boot.
• mce=0: Disables the time in us to wait for other CPUs on machine checks.
• rng_core.default_quality=500: Set the value of the entropy for the system.
• lsm=apparmor,lockdown,capability,landlock,yama,bpf: Set the order of LSM initialization.

More details on the kernel's parameters here.

Disks

The disk is composed:

1. (If the installation is in UEFI) EFI partition in /boot/efi with VFAT filesystem (512 MB).

2. Boot partition in /boot with ext4 filesystem (500 B).

3. Main partition with LVM configuration (30 GB). This partition is divided into 3 parts:

   1. Root partition in / with ext4 filesystem (7 GB).

   2. Log partition in /var/log with ext4 filesystem (1 GB).

   3. Swap partition (500 B).

This can be changed in the build_debian_iso/srv_fai_config/disk_config/ directory. There is always 2 versions (one in Legacy BIOS and an other in UEFI mode with the suffix "_EFI").

Virtual cluster

On the host, you must set these sysctl settings:

net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0

You must define 3 network interfaces on each host of your cluster.

...

)

...

On each host of your cluster, you must load the ptp_kvm module to access to PTP device.