Month: July 2020

Internet of things devices can be networked wirelessly over internet and integrate easily with cloud, mobile and web applications.

WebSockets ( RFC6455 ) run on a variety of platforms including embedded systems and web browsers enabling low latency bidirectional binary socket communication over TCP/IP.

ESP8266 is a low cost Wi-Fi micro-controller compatible with Arduino open-source electronics framework designed for networked sensor, robotics and micro-electronic control applications.

WebSocket MicroServo

In a simple example a micro-servo reports position data and is controlled via a potentiometer (rotary encoder), web and mobile interfaces.

This tutorial demonstrates:

• Microservo / Potentiometer GPIO control
• ESP8266 WebSocket Client
• Binary format Messaging
• Python WebSocket Library to synchronise state between clients
• Responsive HTML5 / Javascript Browser Control Interface

Circuit Wiring

• 5v regulated DC power supply
• Microservo connected to ESP8266 GPIO 15
• 100k Potentiometer – CLK -> GPIO 12, DT -> GPIO 13

Microservo Setup

Microservo power is provided by an external regulated 5v DC power supply sharing a common ground connection to microcontroller.

Servo library initialisation and global variables –

#include <Servo.h>
int servoPin = 15;
int angle; // current angle (degrees)
int servoStartAngle = 90; // initial position
int limit = 90; // range of servo in degrees
Servo Servo1;

// in setup()
Servo1.attach(servoPin);
rotateServo(0);

Servo position is set in a function by passing angle in degrees as a parameter –

void rotateServo(int angle)
{
  Serial.print("RotateServo: ");
  Serial.println(angle);
  Servo1.write(angle);
}

Potentiometer / Rotary Encoder Setup

Potentiometer, a type of incremental rotary encoder based on variable resistance, allows relative rotary motion to be tracked, 2 out-of-phase output channels indicate direction of travel.

#define outputA 12 // Rotary Encoder #1 CLK
#define outputB 13 // Rotary Encoder #2 DT

int counter = 0; // rotary encoder incremental position
int aState; // rotary encoder state comparator
int aLastState;  

void setup() {
   pinMode(outputA,INPUT);
   pinMode(outputB,INPUT);
       
   aLastState = digitalRead(outputA);
}

Rotary encoder CLK and DT pins are compared to determine direction of rotation. Calling map() converts counter (relative position) to angle –

void readRotaryEncoder()
{  

  aState = digitalRead(outputA);

   if (aState != aLastState){     
     // If the outputB state is different to the outputA state, that means the encoder is rotating clockwise
    if (digitalRead(outputB) != aState) { 
      if (counter < limit)
      {
        counter ++;
        angle = map(counter, -90, 90, 0, 180);
        rotateServo(angle);
      }
    } else {
      if (counter + limit > 0)
      {
        counter --;
        angle = map(counter, -90, 90, 0, 180);
        rotateServo(angle);
      }
    }
  } 
  aLastState = aState;
}

Websocket Binary Message Framing

WebSocket protocol natively supports binary framed messaging, offering a compact lightweight format for fast and efficient endpoint messaging.

To report and update microservo position command messages are passed between control interfaces and microcontroller as binary data.

A Python service running on server creates WebSocket and maintains and synchronises shared application state between connected clients.

In web browser user interface, Javascript also has native support for (un)packing binary data.

Data Serialisation

A C struct data_t encapsulates two fields – “cmd” (unit8_t) and “value” (int) – two commands are defined, one to report servo position and another to set a new position, value represents an angle.

To assist serialisation data_t is wrapped in a packet union –

// data message
typedef struct data_t
{
  uint8_t cmd;
  int value;
};

// message packaging / envelope
typedef union packet_t {
 struct data_t data;
 uint8_t packet[sizeof(struct data_t)];
};

#define PACKET_SIZE sizeof(struct data_t)

#define CMD_SERVO_ANGLE 12 // command to report servo position
#define CMD_SERVO_ROTATE 13 // move servo to a specified position

Instances of data structs are created representing send and receive messages.

// send / receive msg data structures
union packet_t sendMsg;
union packet_t receiveMsg;

// messaging function prototypes
void readByteArray(uint8_t * byteArray);
void writeByteArray();
void printByteArray();

// buffer 
uint8_t byteArray[PACKET_SIZE];

Methods allow struct data to be written and read from byte array buffer –

// read bytes from buffer
void readByteArray(uint8_t * byteArray)
{
  for (int i=0; i < PACKET_SIZE; i++)
  {
    receiveMsg.packet[i] = byteArray[i];
  }
}

// write data to buffer
void writeByteArray()
{
  for(int i=0; i<PACKET_SIZE; i++)
  {
    // msg into byte array
    byteArray[i] = sendMsg.packet[i]; 
  }
}

ESP8266 Wifi / Websocket Setup

Details for setting up ESP8266 wifi can be found here.

ESP8266 Websocket library is added by including headers, defining server IP address / port / path and creating a WebSocketsClient class instance.

#include <WebSocketsClient.h>

WebSocketsClient webSocket;

const char* ws_server = "192.168.1.127";
const uint16_t ws_port = 6789;
const char* ws_path = "/";

In setup WebSocket library is initialised –

void setup() {
    ...

    // server address, port and URL
    webSocket.begin(ws_server, ws_port, ws_path);
  
    // event handler
    webSocket.onEvent(webSocketEvent);
  
    // use HTTP Basic Authorization (optional)
    //webSocket.setAuthorization("user", "Password");
  
    // try again if connection has failed
    webSocket.setReconnectInterval(5000);

WebSocket – Message Receive

Event of type “WStype_BIN” defines handling when a binary format message is received, size is reported and hexdump() displays message contents –

void webSocketEvent(WStype_t type, uint8_t * payload, size_t length) {

  switch(type) {
    ...
    case WStype_BIN:
      Serial.printf("[WSc] get binary length: %u\n", length);
      hexdump(payload, length);
      setServoPosition(payload);
      break;

In setServoPosition() received byte array is de-serialised into message data structure. Angle field is used to update servo and potentiometer position.

void setServoPosition(uint8_t * byteArray)
{
  readByteArray(byteArray);
  printByteArray(receiveMsg);

  int angle = (int) receiveMsg.data.value;
  rotateServo(angle);

  counter = map(angle, 0, 180, -90, 90); // update rotary encoder
}

WebSocket – Message Send

Servo position is reported by populating sendMsg data structure, converting to byte array and calling webSocket.sendBIN() passing a pointer payload data and size.

// send servo position to Websocket server
void sendServoPosition()
{
  sendMsg.data.cmd = CMD_SERVO_ANGLE;
  sendMsg.data.value = angle;

  // write message to buffer
  writeByteArray();
  
  webSocket.sendBIN(byteArray, PACKET_SIZE);
}

Server Side – Python WebSocket (WS) Library

A simple WebSocket server implemented in Python v3x is tasked with text / binary message exchange, tracking connected clients and maintaining shared application state.

WebSocket server event loop is started by passing function/method name, IP address and port –

start_server = websockets.serve(wsApi, "192.168.1.127", 6789)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

In event loop new clients are registered. Clients can be identified by checking protocol headers –

async def wsApi(websocket, path):
    # register(websocket) sends user_event() to websocket
    await register(websocket)

...
async def register(websocket):
    USERS.add(websocket)
    await notify_users()
    print(websocket.request_headers);
...

### debug output
Host: 192.168.1.127:6789
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: LnOM1uj7n3gE4cFGqJ1yFg==
Sec-WebSocket-Protocol: arduino
Origin: file://
User-Agent: arduino-WebSocket-Client

Received message and headers are printed and handling is defined for binary (byte array) format messages.

### <class 'bytes'>
### b'\x0c\x00\x00\x00@\x00\x00\x00'
### (12, 64)

Python struct library allows packed binary bytes representing C structs to be unpacked as native python data types.

Format specifier “Ii” represents a message containing an int and unsigned int.

    try:
        async for message in websocket:
            print('Sec-WebSocket-Key: '+websocket.request_headers['Sec-WebSocket-Key'])
            print('MessageType: '+str(type(message)))
            print(message);
            if isinstance(message, (bytes, bytearray)):

                tuple_of_data = struct.unpack("Ii", message)
                cmd = tuple_of_data[0]
                value = tuple_of_data[1]
                STATE["value"] = value
                await notify_state()
    except Exception as e:
        print(e);
    finally:
        await unregister(websocket)

Connected WebSocket clients are push notified (synchronised) when position data (state) is updated –

async def notify_state():
    if USERS: 
        binary_data = struct.pack("Ii", 12, STATE['value'])
        await asyncio.wait([user.send(binary_data) for user in USERS])

WebSocket Browser Client – Binary Messaging in JavaScript

Web Interface is responsive and runs in mobile and web browser clients.

A radial D3.js radial gauge displays current angle. HTML5 buttons (divs) and slider control allow position to be updated.

JavaScript WebSocket onmessage function handles decoding of binary framed data.

A FileReader object is used to convert Blob to ByteArray in an asynchronous function, with flow control provided by a promise (future). TypedArray Uint8Array is used to extract 4 byte int and unsigned int command and value data fields –

websocket.onmessage = function (event) {
    if (event.data instanceof Blob)  // Binary Frame
    {
      async function readBinaryData(blob) {
          let promise = new Promise((res, rej) => {

            var fileReader = new FileReader();
            fileReader.onload = function(event) {
                var arrayBuffer = event.target.result;
                res(arrayBuffer);
            };
            fileReader.readAsArrayBuffer(blob);
          });

          // wait until the promise returns us a value
          let arrayBuffer = await promise;

          var v = new Uint8Array(arrayBuffer);
          // v[0] = cmd, v[4] = value
          //console.log(v[0] + " " + v[4]);

          // update UI elements
          value.textContent = v[4];
          angleSlider.value = v[4];
          gauges.forEach(function(g) {
              g.gauge.update(v[4]);
          });
      };

      readBinaryData(event.data);

Conclusion

WebSockets offer significant advantages over HTTP request/response polling techniques for real time data exchange, principally overhead of opening connection should occur only once per client.

Message push notify model, lightweight protocol framing, client libraries for embedded devices and native support in modern web browsers make this protocol well suited to real time Internet of Things data message exchange.

While underlying TCP provides message ordering and re-transmission (of failed packets), higher level application abstractions: guaranteed message delivery (at least once, at most once), message acknowledgements and queue / persist / forward (to offline clients) are not specified by WebSocket specification.

Similarly, TLS SSL can be used at network layer to encrypt data transmission (WebSocket Secure WSS), but client authentication / authorisation is not handled by WebSocket protocol, meaning a strategy for token or key based client identification (OAuth for example) must be considered for secure use cases.

Event driven implementations supporting asynchronous non-blocking IO result in efficient, well structured and modular code with handlers dedicated to specific tasks.

Factors influencing choice between binary framing and text format messages (field delimeted, JSON) include legibility, convenience, compactness and parsing overhead. Binary format introduces complexity due to differences in compiler / platform / network architectures and data type implementations between programming languages.

Full Source Code on GitHub –

Web UI –
https://github.com/steveio/arduino/blob/master/python/wsESP8266RotaryEncoderServo.html

Python WebSocket Server –
https://github.com/steveio/arduino/blob/master/python/wsESP8266RotaryEncoderServo.html

ESP8266 WebSocket MicroServo Sketch –
https://github.com/steveio/arduino/blob/master/ESP8266RotaryEncoderServo/ESP8266RotaryEncoderServo.ino