Protocol Details

The Hermes protocol is a flow of binary messages exchanged over a gRPC stream. Protocol Buffer specifications define the message types.

When the gRPC stream is established, the protocol starts by doing a vehicle authentication.

Once the authentication is done, each party (vehicle or orchestrator) sends status messages. These messages are asynchronous, they do not require a response, and they can be sent at any time. An orchestrator will most probably send messages if something is required from the vehicle, but it can also send the same message multiple time without anything having changed. The vehicle has a more dynamic state (position and telemetry data will change as the vehicle moves) so messages will flow more frequently. The idea is to that each state message should be independent of the other messages. The orchestrator messages will reflect what the orchestrator expects from the vehicle, and the vehicle messages will reflect what the vehicle is currently doing, and what its current state is.

+----------------+         Orchestration Message       +-----------------+
|                |  +------------------------------>   |                 |
|  Orchestrator  |            (gRPC Stream)            |     Vehicle     |
|                |  <------------------------------+   |                 |
+----------------+            Vehicle Message          +-----------------+

Take the case of a simple mission A that is requested from the orchestrator. Until the mission is done and the orchestrator is aware of it, messages from the orchestrator to the vehicle will contain mission A. Vehicle messages will contain the vehicle state (position, speed, etc) and this mission, updated with its current state. The first message may not contain the mission (if it was not received yet), then the next ones will give mission A as being executed until finally messages mark mission A as finished. At any point in time, without having to look at previous messages, we can see from the latest message what the vehicle is doing, and what the orchestrator wants the vehicle to do.

Transport layer

The protocol is designed independently of the transport layer. Technically speaking, Hermes messages can be exchanged over any transport layer: UDP, TCP, MQTT, Kafka, gRPC, etc. However, we decided to fix the transport layer to gRPC for the following benefits: - it offers streaming for bi-directionnal messaging - it gathers a nice and large ecosystem with code generators, tools and active maintenance - it deals with low levels aspects like message spliting - it is commonly accepted on the IT field

Authentication layer

Connections must be established towards Bestmile with a secured connection (TLS encryption), but without client certificate authentication. Authentication is performed using the Authorization header, using a scheme and an identifier separated by a space: Authorization: SCHEME IDENTIFIER

Valid schemes: | Scheme | Identifier | Description | | BESTMILE-JWT-V1 | JWT | JSON Web Token obtained via /v2/auth/authenticate | | BESTMILE-APIKEY-V4 | API Key | Bestmile API Key | | BESTMILE-APIKEY-V3 | API Key | Legacy Bestmile API Key |

The goal of the authentication is to exchange information that will not change throughout the whole connection life. For now this only includes the vehicle ID, but we could imagine requiring more information in the near future (vehicle information, protocol version supported, protocol feature supported, etc).

The handshake is commonly a 3 way process, but for now as we only require the vehicle ID, so we restrict it to be a single message passed from the Vehicle to the Orchestrator.

As soon as the connection is established, the vehicle needs to send an announce message. The orchestrator will reply with an error if the authentication fails or proceed to the normal message flow if the authentication is OK.

Vehicle Announce to authenticate the vehicle

// VehicleMessage 
{
  // VehicleAnnounce
  announce: {
    // VehicleID
    vehicle_id: {
      value: "theVehicleID"
    }
  }
}

Potential error from Bestmile

// OrchestratorMessage
{
  // OrchestratorError
  errors: [{
    error: OrchestratorError.Error.AUTHENTICATION_FAILED
  }]
  message: "Failed to authenticate vehicle: JWT signature does not match the locally computed signature"
}

State messages

VehicleMessage and OrchestratorMessage can be sent at any time. The main idea with these messages is that it allows the orchestrator to express what it wants from the vehicle, and for the vehicle to express its current state regarding both its physical state and the progression of the commands. The latest message reflects the state without needing to rebuild the state based on the history.

There is no need for acknowledgement and re-transmission mechanisms. Each party (vehicle or orchestrator) can send messages when some state changes or just periodically. If a message is lost, the next one will bring enough information to render the previous message obsolete.

Orchestrator messages

The orchestrator message expresses what the orchestrator expects from the vehicle. It holds a list of commands it wants the vehicle to execute. It is important to note that every command in the command list can, and should, be executed as soon as the vehicle is able to do them. The orchestrator (Bestmile) will be responsible to split the mission into small chunks of commands that can be done together.

A common use case is picking up or dropping off passengers. If at a given location, multiple passengers needs to get in or out of the vehicle, passengers will impose their order (the first one to arrive to the car will get in, or the first one to be ready to get out will do so). It would be wrong and misleading to give an order to pickups and dropoffs to do. So the orchestrator sends all commands to execute, and the vehicle reports the progress of each command as soon as they are done.

Another common pattern would be to have a list of commands holding a drive and a pickup/dropoff. The aim is mostly to pickup/dropoff a passenger, so if the car spot the passenger right before arriving to the destination, or if the passenger wants to leave a bit earlier, the vehicle can report right away that the passenger is in/out even if the drive is not finished (and most probably the orchestrator will remove the unfinished drive).

See OrchestratorMessage from src/main/protobuf/com/bestmile/vehicle/service/v2/orchestration.proto for the definition of this message.

The following example gives an orchestrator message that asks the vehicle to drive to a given location and to pickup a passenger.

// OrchestrationMessage {
  commands: [
    {
      // Command
      command: {
        // Command.Drive
        drive: {
          destination: {
            latitude: 46.51576
            longitude: 6.60821
          }
        }
        start_time: "2020-03-03T06:30:47.658Z"
      }
    },{
      command: {
        // command.Pickup
        pickup: {
          // RideID
          ride_id: {
            value: "b1bb1717-bae5-4e6f-893f-965b02249ce0"
          }
          description: "Passenger X"
        }
        start_time: "2020-03-03T06:42:47.658Z"
      }
    }
  ]
  timestamp: "2020-03-03T06:30:47.658Z"
}

Vehicle messages

As the vehicle moves (almost) constantly and because the orchestrator needs to know it soon enough to act on it or forward it to other components, it is recommended for the vehicle to send vehicle messages periodically.

Vehicle messages are divided in two parts: - VehicleTelemetry contains measurement or information about the vehicle state: position, speed, energy, emergency state, etc - VehicleCommand gives feedback about the command that the orchestrator sent

To give the feedback about orchestrator commands, the vehicle reflects the command it is currently executing (which is the list of command the orchestrator sent) with an annotation of their progress.

See VehicleMessage from src/main/protobuf/com/bestmile/vehicle/service/v2/orchestration.proto for the definition of this message.

The following example gives the response that a vehicle could send after receiving the orchestrator message from the example above:

// VehicleMessage {
  // VehicleTelemetry
  telemetry: {
    location: {latitude: ..., longitude: ...}
    speed: {speed: 16.6}
    energy: {state_of_charge: 0.8}
  }
  commands: [
    // VehicleCommand
    {
      // Command
      command: {
        // Command.Drive
        drive: {
          destination: {
            latitude: 46.51576
            longitude: 6.60821
          }
        }
      }
      start_time: "2020-03-03T06:30:47.658Z"
      // CommandState
      state: {
        ongoing: {}
      }
    },{
      command: {
        // command.Pickup
        pickup: {
          // RideID
          ride_id: {
            value: "b1bb1717-bae5-4e6f-893f-965b02249ce0"
          }
          description: "Passenger X"
          start_time: "2020-03-03T06:42:47.658Z"
        }
      }
      // CommandState
      state: {
        pending: {}
      }
    }
  ]
}