// Module included in the following assemblies:
//
// * networking/ptp/configuring-ptp.adoc

:_mod-docs-content-type: PROCEDURE
[id="ptp-configuring-linuxptp-services-as-ha-bc-for-dual-nic_{context}"]
= Configuring linuxptp as a highly available system clock for dual-NIC Intel E810 PTP boundary clocks

[role="_abstract"]
You can configure the `linuxptp` services `ptp4l` and `phc2sys` as a highly available (HA) system clock for dual PTP boundary clocks (T-BC).

The highly available system clock uses multiple time sources from dual-NIC Intel E810 Salem channel hardware configured as two boundary clocks.
Two boundary clocks instances participate in the HA setup, each with its own configuration profile.
You connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.

Create two `PtpConfig` custom resource (CR) objects that configure the NICs as T-BC and a third `PtpConfig` CR that configures high availability between the two NICs.

[IMPORTANT]
====
You set `phc2SysOpts` options once in the `PtpConfig` CR that configures HA.
Set the `phc2sysOpts` field to an empty string in the `PtpConfig` CRs that configure the two NICs.
This prevents individual `phc2sys` processes from being set up for the two profiles.
====

The third `PtpConfig` CR configures a highly available system clock service.
The CR sets the `ptp4lOpts` field to an empty string to prevent the `ptp4l` process from running.
The CR adds profiles for the `ptp4l` configurations under the `spec.profile.ptpSettings.haProfiles` key and passes the kernel socket path of those profiles to the `phc2sys` service.
When a `ptp4l` failure occurs, the `phc2sys` service switches to the backup `ptp4l` configuration.
When the primary profile becomes active again, the `phc2sys` service reverts to the original state.

[IMPORTANT]
====
Ensure that you set `spec.recommend.priority` to the same value for all three `PtpConfig` CRs that you use to configure HA.
====

.Prerequisites

* Install the {oc-first}.

* Log in as a user with `cluster-admin` privileges.

* Install the PTP Operator.

* Configure a cluster node with Intel E810 Salem channel dual-NIC.

.Procedure

. Create two separate `PtpConfig` CRs, one for each NIC, using the CRs in "Configuring linuxptp services as boundary clocks for dual-NIC hardware" as a reference for each CR.

.. Create the `ha-ptp-config-nic1.yaml` file, specifying an empty string for the `phc2sysOpts` field.
For example:
+
[source,yaml]
----
apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: ha-ptp-config-nic1
  namespace: openshift-ptp
spec:
  profile:
  - name: "ha-ptp-config-profile1"
    ptp4lOpts: "-2 --summary_interval -4"
    ptp4lConf: |
      [ens5f1]
      masterOnly 1
      [ens5f0]
      masterOnly 0
    #...
    phc2sysOpts: ""
----
+
where:
+
--
`ptp4lConf`:: Specifies the required interfaces to start `ptp4l` as a boundary clock. For example, `ens5f0` synchronizes from a grandmaster clock and `ens5f1` synchronizes connected devices.

`phc2sysOpts: ""`:: Sets `phc2sysOpts` with an empty string.
These values are populated from the `spec.profile.ptpSettings.haProfiles` field of the `PtpConfig` CR that configures high availability.
--

.. Apply the `PtpConfig` CR for NIC 1 by running the following command:
+
[source,terminal]
----
$ oc create -f ha-ptp-config-nic1.yaml
----

.. Create the `ha-ptp-config-nic2.yaml` file, specifying an empty string for the `phc2sysOpts` field.
For example:
+
[source,yaml]
----
apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: ha-ptp-config-nic2
  namespace: openshift-ptp
spec:
  profile:
  - name: "ha-ptp-config-profile2"
    ptp4lOpts: "-2 --summary_interval -4"
    ptp4lConf: |
      [ens7f1]
      masterOnly 1
      [ens7f0]
      masterOnly 0
    #...
    phc2sysOpts: ""
----

.. Apply the `PtpConfig` CR for NIC 2 by running the following command:
+
[source,terminal]
----
$ oc create -f ha-ptp-config-nic2.yaml
----

. Create the `PtpConfig` CR that configures the HA system clock.
For example:

.. Create the `ptp-config-for-ha.yaml` file.
Set `haProfiles` to match the `metadata.name` fields that are set in the `PtpConfig` CRs that configure the two NICs.
For example: `haProfiles: ha-ptp-config-nic1,ha-ptp-config-nic2`
+
[source,yaml]
----
apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: boundary-ha
  namespace: openshift-ptp
  annotations: {}
spec:
  profile:
    - name: "boundary-ha"
      ptp4lOpts: ""
      phc2sysOpts: "-a -r -n 24"
      ptpSchedulingPolicy: SCHED_FIFO
      ptpSchedulingPriority: 10
      ptpSettings:
        logReduce: "true"
        haProfiles: "$profile1,$profile2"
  recommend:
    - profile: "boundary-ha"
      priority: 4
      match:
        - nodeLabel: "node-role.kubernetes.io/$mcp"
----
+
Set the `ptp4lOpts` field to an empty string.
If it is not empty, the `p4ptl` process starts with a critical error.
+
[IMPORTANT]
--
Do not apply the high availability `PtpConfig` CR before the `PtpConfig` CRs that configure the individual NICs.
--

.. Apply the HA `PtpConfig` CR by running the following command:
+
[source,terminal]
----
$ oc create -f ptp-config-for-ha.yaml
----

.Verification

* Verify that the PTP Operator has applied the `PtpConfig` CRs correctly.
Perform the following steps:

.. Get the list of pods in the `openshift-ptp` namespace by running the following command:
+
[source,terminal]
----
$ oc get pods -n openshift-ptp -o wide
----
+
.Example output
[source,terminal]
----
NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE
linuxptp-daemon-4xkrb           1/1     Running   0          43m   10.1.196.24      compute-0.example.com
ptp-operator-657bbq64c8-2f8sj   1/1     Running   0          43m   10.129.0.61      control-plane-1.example.com
----
+
[NOTE]
====
There should be only one `linuxptp-daemon` pod.
====

.. Check that the profile is correct by running the following command.
Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile.
+
[source,terminal]
----
$ oc logs linuxptp-daemon-4xkrb -n openshift-ptp -c linuxptp-daemon-container
----
+
.Example output
[source,terminal]
----
I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: ha-ptp-config-profile1
I1115 09:41:17.117616 4143292 daemon.go:102] Interface:
I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -2
I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r -n 24
I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------
----
