IEC 104 redundancy server modes

Multiple redundancy groups

The MZ Automation lib60870 server provides 3 different modes regarding the support of redundant connections and events queue handling:

The default mode (CS104_MODE_SINGLE_REDUNDANCY_GROUP) allows only a single active client connection.
The second mode (CS104_MODE_CONNECTION_IS_REDUNDANCY_GROUP) allows multiple active client connections.
The third mode (CS104_MODE_MULTIPLE_REDUNDANCY_GROUPS) allows multiple active client connections while preserving events when no client is connected.

In the case of this design, the south plugin will be implemented with CS104_MODE_MULTIPLE_REDUNDANCY_GROUPS server mode.

This mode allows multiple active client connections while preserving events when no client is connected. In this mode clients can be assigned to specific redundancy groups. The assignment is based on the IP address of the client. A redundancy group can have multiple simultaneous connections but only one of these connections can be active. The number of activated connections is restricted by the number of redundancy groups. Each redundancy group has a dedicated event queue.

It can be set with the CS104_Slave_setServerMode function:

CS104_Slave_setServerMode(slave, CS104_MODE_MULTIPLE_REDUNDANCY_GROUPS);

Multiple redundancy groups example

In this example, 2 control centers, center A and B, are establishing communication with the server.

Both centers have an active and a stand-by gateway for failover management.

Center A has two simultaneous connections, one active and one stand-by, assigned to redundancy group 1.

Center B has only one active connection, assigned to redundancy group 2.

IEC 104 Protocol stack configuration

The IEC 104 protocol stack configuration specifies communication parameters and is a collection of entries containing information about OSI Transport and OSI Application layers objects.

Each entry is comprised of attributes that describe the object. All the configuration data are structured using JSON.

Each entry shall be mapped with the corresponding configuration function in the chosen implementation protocol library.

Attributes definition

Attribute	Description	Expected values	Mandatory

Attribute	Description	Expected values	Mandatory
name	this identifies the protocol stack	iec104client, iec104server, tase2client, tase2server, 61850client, 61850server, etc...	Yes
version	version number of the configuration file	2 digits x.y => x = major change, y = minor change	Yes
redundancy_groups	array of redundancy groups		Yes
redundancy_groups.connections	array of connections of a given redundancy group		Yes
redundancy_groups.connections.clt_ip	IEC 104 client address	IP address	Yes
redundancy_groups.rg_name	this identifies the redundancy group	Any non empty string	Yes
srv_ip	Server IP address	IP address, machine's default IP for a given interface, default = 0.0.0.0	No
port	This defines the TCP/IP port to be used by the server.	default = 2404	No
tls	activation of TLS (see tls configuration chapter for details)	TRUE, FALSE, default = FALSE	No
k_value	Maximum number of outstanding (unacknowledged) APDU's at a given time	default = 12, range : 1 to 32767	No
w_value	Acknowledge the reception latest after this number of APDU's	default = 8, range : 1 to 32767	No
t0_timeout	time out of connection establishment	default = 30 seconds, range : 1 to 255	No
t1_timeout	time out for send or test APDU's	default = 15 seconds, range : 1 to 255	No
t2_timeout	time out for acknowledges in case of no data messages (t2 < t1)	default = 10 seconds, range : 1 to 255	No
t3_timeout	time out for sending test frames	default = 20 seconds, range : 1 to 172800	No
mode	"accept_always": accept connection or maintain connection with center independently from the south asset connection status "accept_if_south_connx_started": accept connection or maintain connection with center only if south asset connection is established and running	default = "accept_always", enum: "accept_always" or "accept_if_south_connx_started"
ca_asdu_size	size of "Common Address of ASDU"	default = 2 (byte), enum: 1 or 2	No
ioaddr_size	size of 'Information Object Address'	default = 3 (byte), enum: 1, 2 or 3	No
asdu_size	maximum ASDU size in transmission direction, if set to "0" => maximum possible value is automatically used.	default = 0 (byte), range : 0 to 255	No
asdu_queue_size	minimum number of ASDUs that can be stored in the asdu buffer	default = 100	No
time_sync	If set on "TRUE" this parameter allows to synchronize the clock of the local computer by the server. If set on "FALSE", the clock is not synchronized.	TRUE, FALSE, default = FALSE	No
cmd_exec_timeout	Defines the command execution monitoring timeout in seconds.	default = 20 seconds, range : 1 to 3 600	No
cmd_recv_timeout	This parameter defines the highest allowable deviation of received command time tag and local clock in seconds. If the difference is too big, command is ignored.	default = 0 (disabled), range : 0 to 3 600	No
accept_cmd_with_time	If set to 0, then accept no commands with timestamp, if set to 1 accept only commands with timestamp, if set to 2, then accept both	default =1, enum: 0, 1 or 2	No
filter_list	List of Authorized Originators (array) → only commands from authorized originator addresses are accepted.	default = empty	No
filter_list.orig_addr	Originator address	enum: 0, 1, ..., N
cmd_dest	Defines the destination service on which to execute the command	default = broadcast	No
south_monitoring	connection loss and gi failure handling feature		Yes
south_monitoring.assets	array of assets name used to monitor the connection and gi status information from the south	default = [CONSTAT-1, CONSTAT-2]	No

Configuration JSON structure

{
   "protocol_stack":{
      "name":"iec104server",
      "version":"1.0",
      "transport_layer":{
         "redundancy_groups":[
            {
               "connections":[
                  {
                     "clt_ip":"192.168.0.10"
                  },
                  {
                     "clt_ip":"192.168.0.11"
                  },
                  {
                     "clt_ip":"10.152.1.10"
                  },
                  {
                     "clt_ip":"10.152.1.11"
                  }
               ],
               "rg_name":"red-group-1"
            },
            {
               "connections":[
                  {
                     "clt_ip":"192.168.0.10"
                  },
                  {
                     "clt_ip":"192.168.0.11"
                  },
                  {
                     "clt_ip":"192.168.0.12"
                  },
                  {
                     "clt_ip":"192.168.0.14"
                  },
                  {
                     "clt_ip":"10.152.1.10"
                  },
                  {
                     "clt_ip":"10.152.1.11"
                  },
                  {
                     "clt_ip":"10.152.1.12"
                  },
                  {
                     "clt_ip":"10.152.1.13"
                  }
               ],
               "rg_name":"red-group-2"
            },
            {
               "rg_name":"catch-all"
            }
         ],
         "srv_ip":"0.0.0.0",
         "port":2404,
         "tls":false,
         "k_value":12,
         "w_value":8,
         "t0_timeout":30,
         "t1_timeout":15,
         "t2_timeout":10,
         "t3_timeout":20,
         "mode":"accept_always"
      },
      "application_layer":{
         "ca_asdu_size":2,
         "ioaddr_size":3,
         "asdu_size":0,
         "asdu_queue_size":100,
         "time_sync":false,
         "cmd_exec_timeout":20,
         "cmd_recv_timeout":0,
         "cmd_dest":"broadcast",
         "accept_cmd_with_time":1,
         "filter_list":[
            {
               "orig_addr":1
            },
            {
               "orig_addr":2
            }
         ]
      },
      "south_monitoring":[
         {
            "asset":"CONSTAT-1"
         },
         {
            "asset":"CONSTAT-2"
         }
      ]
   }
}

TLS configuration

The CS 104 standard can also be used with TLS to realize secure and authenticated connections.

Parameters are needed to set up the TLS secured connection:

Attribute	Description	Expected values	Mandatory

Attribute	Description	Expected values	Mandatory
private_key	server private key	valid private key	YES
own_cert	server certificate	valid certificate	YES
ca_certs	allows to specify the ca certificates if not included in the owner certificate	list of valid certificates	NO
remote_certs	allows to specify the clients certificates, so if specified, only these certificates are accepted	list of valid certificates	NO

Fledge's certificate store allows certificates to be stored and used by the south plugins.

{
   "private_key":"iec104_server.key",
   "own_cert":"iec104_server.cer",
   "ca_certs":[
      {
         "cert_file":"iec104_ca.cer"
      },
      {
         "cert_file":"iec104_ca2.cer"
      }
   ],
   "remote_certs":[
      {
         "cert_file":"iec104_client.cer"
      }
   ]
}

IEC 104 datapoint representation

This is the Datapoint representation of an IEC 104 ASDU for a command.

The current implementation does not use a json structure, so the order of parameters is mandatory.

Order	Attribute JSON (not currently used)	Description	Expected values	Mandatory

Order	Attribute JSON (not currently used)	Description	Expected values	Mandatory
0	co_type	Type of ASDU		YES
1	co_ca	Common Address of ASDU		YES
2	co_ioa	Information object address		YES
3	co_cot	Cause of Transmission		YES
4	co_negative	is negative command ?	0 or 1 = negative	YES
5	co_se	Select or Execute	0 = Direct execute or 1 = Select before Execute	YES
6	co_test	is test command ?	0 or 1 = test	YES
7	co_ts	timestamp		YES
8	co_value	Value		YES
9	co_qu	is pulse defined ?	0 or 1 = pulse defined	NO

Below is an example of an instance of an Operation object :

Connection status audits

This plugin will send Fledge audits of type SRVFL containing different messages to notify about changes in the status of the connection.

The connection status will be tracked at two different levels:

Path level: An IEC104 connection has a maximum of two path defined (A and B) and a maximum of two redundancy groups defined (0 and 1), a single connection among all of those is the active one, others are passive.
Global level: The same level as in south_events representing the global state of the connection.

Generated audit messages will have the following pattern based on their level:

Path level: <service_name>-<red_group>-<path_letter>-<status>
Global level : <service_name>-<status>

With:

<service_name>: The name of the service running this plugin (eg: iec104north_c1)
<red_group>: The ID number of the red group (0 or 1).
<path_letter>: The letter of the path (A or B).
<status>: The connection status.

The usage of an ID for the redundancy group and not the name from the protocol_stack configuration is designed to be able to send an initial audit for each possible path and red group at startup or upon reconfigure.

Sending those initial audits would not be possible if the names were being used instead of an ID as the red group names are not predictible.

These ID represent the order of appearance of the red groups in the protocol_stack.transport_layer.redundancy_groups array.

The connection status can take different values based on the level of connection, each value is sent with a given audit severity :

Path level:

Status	Severity	Description

Status	Severity	Description
unused	INFORMATION	The configuration does not include this path
disconnected	FAILURE	The path is not connected
active	SUCCESS	The path is connected and is the active path
passive	SUCCESS	The path is connected and is the passive path

Global level:

Status	Severity	Description

Status	Severity	Description
disconnected	FAILURE	None of the configured path is connected
connected	SUCCESS	At least one of the configured path is connected

These audits are sent :

Whenever the plugin is reconfigured (including at plugin startup)
Whenever the connection state they represent changes

Some audits may be repeated in the same state (in case of reconfiguration or plugin restart for example) but the implementation is designed to minimize the number of audits sent while ensuring that no state change can be missed.

Automatic command rejection

When a command operation is issued by a connected IEC104 device, that command can be rejected if it contains invalid or unwanted parameters.

Whenever this happen a command response is automatically generated from the received ASDU and sent to the IEC104 device after setting the negative flag on it and updating its COT as follow :

Rejection reason	COT

Rejection reason	COT
South plugin is not connected to its PA (visible through connection status readings)	`CS101_COT_ACTIVATION_CON`
Command contained unexpected COT (expected `CS101_COT_ACTIVATION`)	`CS101_COT_UNKNOWN_COT`
Command contained unknown type or missing information object	`CS101_COT_UNKNOWN_TYPE_ID`
Command contained unknown CA	`CS101_COT_UNKNOWN_CA`
Command contained an OA that is not allowed (not in `filter_list`if defined)	`CS101_COT_ACTIVATION_CON`
Command contained unknown IOA	`CS101_COT_UNKNOWN_IOA`
Command contained mismatching type compared to exchanged_data	`CS101_COT_UNKNOWN_TYPE_ID`
Command with timestamp contained invalid timestamp (time diff > `cmd_recv_timeout`)	`CS101_COT_ACTIVATION_CON`
Command with timestamp was received when they are disabled (`accept_cmd_with_time`= 0)	`CS101_COT_UNKNOWN_TYPE_ID`
Command without timestamp was received when they are disabled (`accept_cmd_with_time`= 1)	`CS101_COT_UNKNOWN_TYPE_ID`
Command was rejected by Fledge	`CS101_COT_ACTIVATION_CON`

FledgePower

IEC 104 north plugin