Cluster Federations

Cluster federation enables several clusters to create a wider trusted infrastructure. For instance, on a worldwide realtime game, developers may want to create a cluster per world region for game sessions to minimize latency, but may want to keep a centralized cluster for authentication, social services and matchmaking.

Federations allows to create this kind of infrastructure using mostly configuration. In a federation: - Players connected to a cluster keep their session id when connecting to another one - Scenes can send s2s requests to applications and scenes in another cluster of the federation.

This allows applications that leverage federation to (for instance) maintain player sessions and identity accross the whole federation with a single authentication service.

Security & setup

  • Each cluster in the federation MUST have an unique clusterId.

  • Each cluster in a federation MUST be associated with a private key. The subject (CN) of the certificate MUST be the same as the cluster Id in the federation.

  • All clusters in the federation MUST have in their configuration a list of all other clusters in the federation, indexed by clusterId and with the associated certificate thumbprint.

  • All clusters in the federation MUST have access to the public key of the certificate used to authenticate each other cluster in the federation

  • Each cluster in the federation must use different certificates to authentify themselves (the certificate CN must be equal to the clusterId, and certificates are use to authenticate each clusters)

Configuring nodes for federation

First, you need to make sure that each cluster in your federation has an unique clusterId. Cluster ids are set in each node config:

 "cluster": {
//Id of the cluster. Defaults to 'default'
"name": "test",

To enable SSL node to node communication and node authentication, a private & a public key pair is required for each cluster. Creating a key pair can be done using the Stormancer command line:

stormancer create-cluster-key  -c config --password xxxx --expire 01/01/2025

Note

It’s currently not possible to rotate cluster keys without cluster restart. This feature is on the roadmap.

The command creates a set of public and private key, with the private key being password protected.

Deploy the private key to all nodes in the cluster in a directory of your choice, for instance /etc/stormancer/private. and create a passwords.json file containing the private key password in another directory only accessible to the user running Stormancer. /etc/stormancer/passwords/passwords.json:

{
  "cluster-pk-password":"xxxx"
}

In order to decrypt the private key, the server needs the private key password located in passwords.json:

"configs": {
  // Paths to a protected file accessible only by the stormancer process.
  //The content of these files is added to constants at runtime.
  "secrets": "/etc/stormancer/passwords/passwords.json"
},

Then update the node configuration to provide a cluster id, require node authentication and provide the private key and password:

"cluster": {
  //Id of the cluster. Defaults to 'default'
  "name": "test",
  //The private key associated with this cluster. All nodes should use the same key, and the subject should be the clusterId (in the example, test)
  "privateKey":
  {
        "path":"/etc/stormancer/private/test.pem" ,
        password":"{secrets.cluster-pk-password}",
   },

  //Does the cluster requires node authentication? If no, node 2 node communications are not encrypted, and federation is not possible.
  //Setting to true requires setting the private key.
  "requireNodeAuthentication": true,

   [..]

Please note that the same password file is used to decrypt the SSL certificate.

The federation configuration section describes how the node can discover communicate and authenticate other clusters in the federation:

"federation": {
  //Endpoint used by nodes of other clusters in the federation to connect to this node
  //leave empty or set to null to prevent this node from accepting connections from nodes in other clusters.
  "publicEndpoint": "{publicIp}:{n2nPort}",
  "clusters": {
    //List of endpoints to try to get metadata about the remote clusters
    "endpoints": [ "http://localhost:81" ],
    //Sources providing the cluster certificates
    "certificateSources": [
      {
        "path": "{dataDirectory}/certs"
      }
    ]
  }
},

Addressing scenes in a federation

To address a scene located in another cluster, it’s necessary to use absolute scene uris.

scene:clusterId/account/app/sceneId