c++ – steveio.com

Espressif Systems Shanghai launched the game changing low cost ESP-8266 microcontroller in 2014 – a key enabler for embedded Internet of Things (IOT) development.

Internet of Things (IOT)

Adding WiFi 802.11 and Bluetooth LE wireless connectivity to system on a chip (SoC) product costing roughly price of a cup of coffee meant innovators and micro electronics DIY enthusiasts could easily interface edge smart sensor or legacy hardware systems with cloud or mobile devices.

The successor chip ESP-32 (launched 2016) introduced Xtensa LX6 processor and FreeRTOS (a real time operating system (OS) for embedded) – enabling multi-threaded applications running on multi core CPU architecture.

To add context, many even relatively complex tasks especially automation, can be run on tiny embedded 8 bit processor such as Arduino. More complex applications, video, audio signal processing, image recognition or AI require much more compute power.

ESP-32 in the eco-system sits between Arduino and more powerful systems Raspberry Pi, Windows or Embedded Linux.

Sounds great, but how does it work in practice?

ESP8266

In this sketch ( https://github.com/steveio/arduino/tree/master/ESP32GPSMultiTask ) we assemble a UBLOX GPS data logger with an SD card internal storage, LoRaWAN wireless relay and an OLED display.

The goal is to demonstrate running 4 separate non-blocking tasks concurrently using FreeRTOS to schedule tasks, suspend, interrupt, queue and share data in thread-safe way with mutex semaphore locks.

Let’s take a look at the code…. concentrating on FreeRTOS multi-core multi-thread potential, rather than peripherals / sensors.

View this post on Instagram

A post shared by steveio.com (@st3v.10)

Data Structures, Multi-thread Semantics & Setup()

First, a struct to encapsulate core data model message (GPS data):

// GPS position data struct XPosit { float Lat; float Lon; float Alt; float Course; float Speed; } xPosit;

Next two semaphores to serialise reading and writing tasks to ensure data consistency:

// Semaphores to lock / serialise data structure IO SemaphoreHandle_t sema_GPS_Gate; SemaphoreHandle_t sema_Posit;

Then a pointer queue for passing messages between threads / tasks:

// GPS position data queue QueueHandle_t xQ_Posit;

Here is the related setup():

sema_GPS_Gate = xSemaphoreCreateMutex(); sema_Posit = xSemaphoreCreateMutex(); xSemaphoreGive( sema_GPS_Gate ); xSemaphoreGive( sema_Posit );

Now let’s define 4 tasks:

// task handles static TaskHandle_t xGPSTask; static TaskHandle_t xLoRATask; static TaskHandle_t xSDWriteTask; static TaskHandle_t xOLEDTask; // hexadecimal notification code define GPS_READ_BIT 0x01 define LORA_TX_BIT 0x02 define LORA_RX_BIT 0x04 define SD_WRITE_BIT 0x06 define OLED_BIT 0x08

ISR / Interrupt for Asynchronous Non-Blocking Event Loop

The system will be driven by a periodic ISR timer raising an interrupt calling a handler routine running & managing tasks.

Keeping loop() empty results in non-blocking program execution, no waiting on calls to delay() in main routine – and we only need to call ISR once every ten seconds to read GPS (rather than polling loop()) – which is better for low power consumption.

Under the hood FreeRTOS works in the same way:

> FreeRTOS implements multiple threads by having the host program call a thread tick method at regular short intervals. The thread tick method switches tasks depending on priority and a round-robin scheduling scheme.
( https://en.wikipedia.org/wiki/FreeRTOS#References )

With a task based modular event driven architecture, failure of one task – if write to SD card task blocks or fails because there is no storage card present), other tasks – reading GPS messages, LoRaWAN radio transmit continue.

// ISR timer hw_timer_t * timer = NULL; portMUX_TYPE timerMux = portMUX_INITIALIZER_UNLOCKED; unsigned long isrCounter = 0;

ISR Handler Function (Main control routine, replaces loop()):

// ISR Interupt Handler void IRAM_ATTR fLoRASendISR( void ) { portENTER_CRITICAL_ISR(&timerMux);

// Main program routine here ..

portEXIT_CRITICAL_ISR(&timerMux); }

In setup() we schedule ISR timer:

// Configure Prescaler to 80, as our timer runs @ 80Mhz // Giving an output of 80,000,000 / 80 = 1,000,000 ticks / second timer = timerBegin(0, 80, true); timerAttachInterrupt(timer, &fLoRASendISR, true); // Fire Interrupt every 10s (10 * 1 million ticks) timerAlarmWrite(timer, 10000000, true); timerAlarmEnable(timer);

FreeRTOS Task (Thread) Setup

Next in setup() we describe & initialise tasks, specify which CPU core they should run on, their priority and a few other details

// create a pointer queue to pass position data xQ_Posit = xQueueCreate( 15, sizeof( &xPosit ) );

Serial.print("Start Task fGPS_Parse() priority 0 on core 0"); xTaskCreatePinnedToCore( fGPS_Parse, "fGPS_Parse", 1000, NULL, 0, &xGPSTask, taskCore0 ); configASSERT( xGPSTask );

Serial.println("Start Task fSD_Write() priority 4 on core 1"); xTaskCreatePinnedToCore( fSD_Write, "fSD_Write", 1000, NULL, 4, &xSDWriteTask, taskCore1 ); // assigned to core 1 configASSERT( xSDWriteTask );

Serial.println("Start Task fLoRA_Send() priority 3 on core 1"); xTaskCreatePinnedToCore( fLoRA_Send, "fLoRA_Send", 1000, NULL, 3, &xLoRATask, taskCore1 ); // assigned to core 1 configASSERT( xLoRATask );

Here’s how to have a task start and wait suspended pending a notification. LoRaWAN transmit task starts and waits until read GPS notifies of new message to send; no need to constantly run radio draining battery or poll the message queue.

for( ;; )
{
/* block until task notification */
xResult = xTaskNotifyWait( LORA_TX_BIT,
                     ULONG_MAX,        /* Clear all bits on exit. */
                     &amp;ulNotifiedValue, /* Stores the notified value. */
                     portMAX_DELAY );  /* Block indefinately */

if( xResult == pdPASS ) 
{

Executing FreeRTOS threads : ISR Main Routine

In main ISR program loop, here’s how to notify tasks:

BaseType_t xHigherPriorityTaskWoken = pdFALSE;

/* Notify (trigger) Read GPS
xTaskNotifyFromISR( xGPSTask,
                   GPS_READ_BIT,
                   eSetBits,
                   &amp;xHigherPriorityTaskWoken );

/* Notify LoRA send task to transmit by setting the TX_BIT */
xTaskNotifyFromISR( xLoRATask,
                   LORA_WRITE_BIT,
                   eSetBits,
                   &amp;xHigherPriorityTaskWoken );

Inter process Communication (IPC): Semaphore Mutex Locks

Concurrency & data consistency in real time and multi threaded systems requires care to ensure serial ops – a write by one thread to a data struct for example must not corrupted by another concurrent thread.

The classic answer to this is locks – mutex, semaphores. A thread requests obtains & takes a lock and other tasks block (wait, retry). The lock is revoked only when lock holding task completes or rolls back.

Here is how it’s done in FreeRTOS:

   if ( xSemaphoreTake( sema_GPS_Gate, xTicksToWait0 ) == pdTRUE )
{
 if ( xSemaphoreTake( sema_Posit, xTicksToWait1000 ) == pdTRUE )
    {
      xPosit.Lat = gps.location.lat();
      xPosit.Lon = gps.location.lng();
      xPosit.Alt = gps.altitude.meters();
      xPosit.Course = gps.course.deg();
      xPosit.Speed = gps.speed.kmph();

      xSemaphoreGive( sema_Posit );
    }


    if ( xSemaphoreTake( sema_Posit, xTicksToWait1000 ) == pdTRUE )
    {
      Serial.println("xQueueSend()");
      pxPosit = &xPosit;
      xQueueSend( xQ_Posit, ( void * ) &amp;pxPosit, ( TickType_t ) 0 );  
      xSemaphoreGive( sema_Posit );
    }

    xSemaphoreGive( sema_GPS_Gate );
  }

FreeRTOS : Message Queue

Those interested in using Queue’s (a sized FIFO pointer linked buffer) to pass data between tasks should consult the API reference:

https://www.freertos.org/a00018.html

// Examples:

// Writing
if ( xSemaphoreTake( sema_Posit, xTicksToWait1000 ) == pdTRUE )
        {
          pxPosit = &xPosit;
          xQueueSend( xQ_Posit, ( void * ) &amp;pxPosit, ( TickType_t ) 0 );  
          xSemaphoreGive( sema_Posit );
        }

// Reading..
struct XPosit xPosit, *pxPosit;
  
 if( xQueueReceive( xQ_Posit,
                         &amp;( pxPosit ),
                         ( TickType_t ) 10 ) == pdPASS )
      {

Conclusion

Espressif changed the game in 2014 with ESP8266 – bringing WiFi & Bluetooth to Arduino’s eco-system of low cost interoperable sensors & components – the internet of things (IOT) long promised since at least 1980s as “smart home” concept finally came of age.

Modern mobile devices and laptops are now so incredibly complex as to be virtually indecipherable, especially at physical hardware / OS level – to most people, even those within IT industry.

Arduino makes it possible for students, DIY enthusiasts, makers & researchers to work with Microcontrollers in a way that is relatively simple, comprehensible and fun.

By adding WiFi, Bluetooth & LoRaWAN wireless, Espressif opened door to new innovatioon in information age of cloud connected edge sensor & control devices.

Timed Device is a library for Arduino / ESP 8266/32 embedded platforms written to emulate a simple plug in timer for an electrical device, where pins in a ring are set to define hour/minute status. Designed to be simple and lightweight optimisation is for low memory.

Arduino ATMega328 and related embedded microcontrollers provide internal high precision clock based timers suitable for events with second, millisecond and fine granularity (to clock speed 8 / 16 mhz).

When lower precision is sufficient, recurring timers for example where an event should occur on a specific minute, hour or day of week, a lightweight implementation can be achieved and storage for internal data structures can be optimised, a significant win factor on embedded systems where memory use is constrained.

The library follows an object orientated design pattern resulting in an extensible, scale-able architecture suitable for controlling from one to a large number of devices sharing a common timing core with each timed device class able to implement specific instructions for switching on/off.

Example use cases:

Switch lights or any electrical relay device on/off
Activate a pump or valve
Open/Close blinds, curtains or shutters
Control a fan, heat source or air conditioning

How to Define Time

In C time.h the tm structure has the following definition −

struct tm {
   int tm_sec;         /* seconds,  range 0 to 59          */
   int tm_min;         /* minutes, range 0 to 59           */
   int tm_hour;        /* hours, range 0 to 23             */
   int tm_mday;        /* day of the month, range 1 to 31  */
   int tm_mon;         /* month, range 0 to 11             */
   int tm_year;        /* The number of years since 1900   */
   int tm_wday;        /* day of the week, range 0 to 6    */
   int tm_yday;        /* day in the year, range 0 to 365  */
   int tm_isdst;       /* daylight saving time             */
};

While this structure is useful for point in time defined events, in case of a recurring timers this can be simplified.

If hour precision is sufficient, a single 32 bit mask can be used:

// Bitmask defines hours (from 24h hours) device is on (1) or off (0)
// 0b 00000000 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
long hourTimerBitmask =0b00000000001111111111111111100000;

Day of week (bit index 0 = Sunday – 6 Saturday) can be defined similarly:

long dayOfWeekTimerBitmask = 0b0101010;

With bitmasks we can determine if an event is scheduled using bit shifts:

// Check if bit at pos n is set in 32bit long l
bool Timer::_checkBitSet(int n, long * l)
{
  bool b = (*l >> (long) n) &amp; 1U;
  return b;
}

To check if an hour timer bitmask is on/off for 7am:

bool isSet = _checkBitSet(7, hourTimerBitmask)

For a more detailed guide to Bitmasks, shifts and bitwise operations see this article.

An on / off time occuring at specific (hour:minute) intervals can be defined as:

typedef struct tmElements_t {
  uint8_t Min;
  uint8_t Hour;
};

We can use a struct as container for an array of 1..n pairs of on/off times occurring on specified weekdays:

#define SZ_TIME_ELEMENT 2

typedef struct tmElementArray_t {
  unsigned long Wday;   // bitmap - days of week (bit index 0 = Sun - 6 Sat)
  struct tmElements_t onTime[SZ_TIME_ELEMENT];
  struct tmElements_t offTime[SZ_TIME_ELEMENT];
};

With these data structures we can setup up a pair of on/off times (08:00 -> 08:10 and 19:00 -> 19:30) to occur on Sun, Mon, Thur, Fri:

// create variables to define on/off time pairs
struct tmElements_t t1_on, t1_off, t2_on, t2_off, t3_on, t3_off;
struct tmElementArray_t timeArray;

t1_on.Hour = 8;
t1_on.Min = 0;
t1_off.Hour = 8;
t1_off.Min = 10;

t2_on.Hour = 19;
t2_on.Min = 0;
t2_off.Hour = 19;
t2_off.Min = 30;

timeArray.n = 2; // number of time pairs
timeArray.Wday = 0b00110011; // define days of week timer is active on

timeArray.onTime[0] = t1_on;
timeArray.offTime[0] = t1_off;

Check if a recurring timer is set

How we check a timer (whether bitmask or time based) depends on the type of time source we have.

Most familiar time source on Arduino is millis() function providing elapsed time in milliseconds since device reset / startup.

This is useful for timing events in a non-blocking way at recurring intervals:

if (millis() >= sampleTimer + sampleInterval)
{
  ... do something
  sampleTimer = millis();
}

But what if we want to schedule in terms of hour, minute or day of week based on current time?

To achieve this a micrcontroller is combined with a Real Time Clock module, providing a battery backed time source that once synced (for example to an NTP time source) maintains current time even when device is switched off.

Arduino Nano in pin breakout board with DS3231 Real Time Clock Module

Arduino DS1307/3231 RTC modules have supporting libraries (for example RTCLib from Adafruit) to obtain current time usually in form of:

DateTime now = rtc.now();

Where DateTime object provides an API to return specific time elements:

Serial.print(now.hour());
Serial.print(':');
Serial.println(now.minute());
Serial.print(now.dayOfTheWeek());

Is an event scheduled (at a specific point in time)?

In C function overloading can be used to provide multiple interfaces to an isScheduled() method according to timer precision / time source type :

bool isScheduled(int h);
bool isScheduled(int h, int d);
bool isScheduled(int m, int h, int d);
bool isScheduled(unsigned long ts);

(Where h = hour, m = minute, d = week day)

Most users of Unix based systems are familiar with “timestamp” a 32bit unsigned long representing elapsed seconds since a fixed point in time, the Unix Epoch which occurred 1970-01-01 00:00:00 UTC.

Unix is not unique in using a reference Epoch, they have been used in calender systems worldwide since ancient times.

We can obtain current timestamp on Linux via command line with date command:

stevee@ideapad-530S:~$ date +%s
1620155520

A timestamp as a time source can be used with any timer definition, whether point in time or recurring. It has advantage (compared to using numeric representations of individual time elements) that a single long number can be used for computation and conversion and we don’t need to worry about problems like variable number of days in month or leap years.

Timestamps and time and date conversion tricks

For a timer defining specific days of week, the first question might be how to obtain weekday from a unix timestamp (elapsed seconds since epoch)?

int weekday = (floor((ts / 86400)) + 4) % 7;

1970-01-01 was a Thursday, dividing timestamp by 86400 (number of seconds in a day: 24 * 60 * 60) gives number of days since epoch, adding 4 shifts start day to Sun and modulo 7 returns day of week.

To check if a timestamp is within range of one or more on/off time pairs (specified in hh:mm format as uint8_t Min; uint8_t Hour;) first we convert all time elements to elapsed seconds:

// convert fully qualified timestamp to elapsed secs from previous midnight
unsigned long elapsedTime = ts % SECS_PER_DAY;

unsigned long s1, s2, onTime, offTime;

// check each timeArray on/off pair
for (int i = 0; i < _timeArray->n; i++)
{
  s1 = _timeArray->onTime[i].Min * SECS_PER_MIN;
  s2 = _timeArray->onTime[i].Hour * SECS_PER_HOUR;
  onTime = s1 + s2;
  s1 = _timeArray->offTime[i].Min * SECS_PER_MIN;
  s2 = _timeArray->offTime[i].Hour * SECS_PER_HOUR;
  offTime = s1 + s2;

  if (elapsedTime >= onTime &amp;&amp; elapsedTime <= offTime)
  {
    return true;
  }
}

Conclusion

While TimedDevice library is non-blocking (it does not technique like delay()) it is not truely asynchronous or event driven as each device implementing a timer must poll for an event on each iteration of loop().

This could constrained to checking once per second/minute/hour but perhaps a better architecture would be to use either RTC DS3231 squarewave or Arduino internal timer to register and trigger an interrupt with an associated handler only when an event is due to occur.

Similarly if either a very large number of events are scheduled, or a large set of timed devices created, it would be sensible to register these with a scheduler (akin to a CPU scheduler) tasked with sorting, prioritising and actioning event queue in an efficient way, which might utilise a multi-threaded approach on multi-core CPU architectures.

Source code for Timed Device library can be found on GitHub including a full Arduino sketch example for defining a solenoid valve timer with an RTC timesource.