Rework backend add MQTT and WebSocket support

* Update back end to add MQTT and WebSocket support
* Update demo project to demonstrate MQTT and WebSockets
* Update documentation to describe newly added and modified functionallity
* Introduce separate MQTT pub/sub, HTTP get/post and WebSocket rx/tx classes
* Significant reanaming - more accurate class names
* Use PROGMEM_WWW as default
* Update README documenting PROGMEM_WWW as default
* Update README with API changes

README.md

@@ -15,10 +15,11 @@ Provides many of the features required for IoT projects:
 * Configurable WiFi - Network scanner and WiFi configuration screen
 * Configurable Access Point - Can be continuous or automatically enabled when WiFi connection fails
 * Network Time - Synchronization with NTP
+* MQTT - Connection to an MQTT broker for automation and monitoring
 * Remote Firmware Updates - Enable secured OTA updates
 * Security - Protected RESTful endpoints and a secured user interface
 
-The back end is provided by a set of RESTful endpoints and the React based front end is responsive and scales well to various screen sizes.
+The back end is provided by a set of RESTful endpoints and the responsive React based front end is built using [Material-UI](https://material-ui.com/).
 
 The front end has the prerequisite manifest file and icon, so it can be added to the home screen of a mobile device if required.
 
@@ -37,13 +38,14 @@ Pull the project and open it in PlatformIO. PlatformIO should download the ESP82
 
 The project structure is as follows:
 
-Resource | Description
+Resource                         | Description
----- | -----------
+-------------------------------- | ----------------------------------------------------------------------
-[data/](data) | The file system image directory
+[data/](data)                    | The file system image directory
-[interface/](interface) | React based front end
+[interface/](interface)          | React based front end
-[src/](src) | The main.cpp and demo project to get you started
+[lib/framework/](lib/framework)  | C++ back end for the ESP8266/ESP32 device
+[src/](src)                      | The main.cpp and demo project to get you started
+[scripts/](scripts)              | Scripts that build the React interface as part of the platformio build
 [platformio.ini](platformio.ini) | PlatformIO project configuration file
-[lib/framework/](lib/framework) | C++ back end for the ESP8266 device
 
 ### Building the firmware
 
@@ -75,13 +77,29 @@ platformio run -t upload
 
 ### Building & uploading the interface
 
-The interface has been configured with create-react-app and react-app-rewired so the build can customized for the target device. The large artefacts are gzipped and source maps and service worker are excluded from the production build. This reduces the production build to around ~200k, which easily fits on the device.
+The interface has been configured with create-react-app and react-app-rewired so the build can customized for the target device. The large artefacts are gzipped and source maps and service worker are excluded from the production build. This reduces the production build to around ~150k, which easily fits on the device.
 
-The interface will be automatically built by PlatformIO before it builds the firmware. The project can be configured to serve the interface from either SPIFFS or PROGMEM as your project requires. The default configuration is to serve the content from SPIFFS which requires an additional upload step which is documented below.
+The interface will be automatically built by PlatformIO before it builds the firmware. The project can be configured to serve the interface from either PROGMEM or SPIFFS as your project requires. The default configuration is to serve the content from PROGMEM, serving from SPIFFS requires an additional upload step which is documented below.
+
+#### Serving the interface from PROGMEM
+
+By default, the project is configured to serve the interface from PROGMEM. This can be disabled by removing the -D PROGMEM_WWW build flag in ['platformio.ini'](platformio.ini) and re-building the firmware. If this your desired approach you must manually [upload the file system image](#uploading-the-file-system-image) to the device.
+
+The interface will consume ~150k of program space which can be problematic if you already have a large binary artefact or if you have added large dependencies to the interface. The ESP32 binaries are fairly large in there simplest form so the addition of the interface resources requires us to use special partitioning for the ESP32.
+
+When building using the "node32s" profile, the project uses the custom [min_spiffs.csv](https://github.com/espressif/arduino-esp32/blob/master/tools/partitions/min_spiffs.csv) partitioning mode. You may want to disable this if you are manually uploading the file system image:
+
+
+```yaml
+[env:node32s]
+board_build.partitions = min_spiffs.csv
+platform = espressif32
+board = node32s
+```
 
 #### Uploading the file system image
 
-If service content from SPIFFS (default), build the project first. Then the compiled interface may be uploaded to the device by pressing the "Upload File System image" button:
+If service content from SPIFFS, disable the PROGMEM_WWW build flag and build the project. The compiled interface will be copied to [data/](data) by the build process and may now be uploaded to the device by pressing the "Upload File System image" button:
 
 ![uploadfs](/media/uploadfs.png?raw=true "uploadfs")
 
@@ -91,28 +109,9 @@ Alternatively run the 'uploadfs' target:
 platformio run -t uploadfs
 ```
 
-#### Serving the interface from PROGMEM
-
-You can configure the project to serve the interface from PROGMEM by uncommenting the -D PROGMEM_WWW build flag in ['platformio.ini'](platformio.ini) then re-building and uploading the firmware to the device. 
-
-Be aware that this will consume ~150k of program space which can be especially problematic if you already have a large build artefact or if you have added large javascript dependencies to the interface. The ESP32 binaries are large already, so this will be a problem if you are using one of these devices and require this type of setup.
-
-A method for working around this issue can be to reduce the amount of space allocated to SPIFFS by configuring the device to use a differnt strategy partitioning. If you don't require SPIFFS other than for storing config one approach might be to configure a minimal SPIFFS partition.
-
-For a ESP32 (4mb variant) there is a handy "min_spiffs.csv" partition table which can be enabled easily:
-
-```yaml
-[env:node32s]
-board_build.partitions = min_spiffs.csv
-platform = espressif32
-board = node32s
-```
-
-This is largley left as an exersise for the reader as everyone's requirements will vary.
-
 ### Running the interface locally
 
-You can run a local development server to allow you preview changes to the front end without the need to upload a file system image to the device after each change. 
+You can run a development server locally to allow you preview changes to the front end without the need to upload a file system image to the device after each change. 
 
 Change to the ['interface'](interface) directory with your bash shell (or Git Bash) and use the standard commands you would with any react app built with create-react-app:
 
@@ -126,27 +125,31 @@ Install the npm dependencies, if required and start the development server:
 npm install
 npm start
 ```
-
+> **Tip**: You can (optionally) speed up the build by commenting out the call to build_interface.py under "extra scripts" during local development. This will prevent the npm process from building the production release every time the firmware is compiled significantly decreasing the build time.
-> **Note**: To run the interface locally you may need to modify the endpoint root path and enable CORS.
 
 #### Changing the endpoint root
 
-The endpoint root path can be found in ['interface/.env.development'](interface/.env.development), defined as the environment variable 'REACT_APP_ENDPOINT_ROOT'. This needs to be the root URL of the device running the back end, for example:
+The interface has a development environment which is enabled when running the development server using `npm start`. The environment file can be found in ['interface/.env.development'](interface/.env.development) and contains the HTTP root URL and the WebSocket root URL:
 
-```js
+```properties
-REACT_APP_ENDPOINT_ROOT=http://192.168.0.6/rest/
+REACT_APP_HTTP_ROOT=http://192.168.0.99
+REACT_APP_WEB_SOCKET_ROOT=ws://192.168.0.99
 ```
 
+The `REACT_APP_HTTP_ROOT` and `REACT_APP_WEB_SOCKET_ROOT` properties can be modified to point a ESP device running the back end firmware.
+
+> **Tip**: You must restart the development server for changes to the environment file to come into effect.
+
 #### Enabling CORS
 
 You can enable CORS on the back end by uncommenting the -D ENABLE_CORS build flag in ['platformio.ini'](platformio.ini) then re-building and uploading the firmware to the device. The default settings assume you will be accessing the development server on the default port on [http://localhost:3000](http://localhost:3000) this can also be changed if required:
 
-```
+```properties
 -D ENABLE_CORS
 -D CORS_ORIGIN=\"http://localhost:3000\"
 ```
 
-## Device Configuration
+## Device configuration & default settings
 
 The SPIFFS image (in the ['data'](data) folder) contains a JSON settings file for each of the configurable features. 
 
@@ -154,13 +157,16 @@ The config files can be found in the ['data/config'](data/config) directory:
 
 File | Description
 ---- | -----------
-[apSettings.json](data/config/apSettings.json) | Access point settings
+[apSettings.json](data/config/apSettings.json)             | Access point settings
-[ntpSettings.json](data/config/ntpSettings.json) | NTP synchronization settings
+[mqttSettings.json](data/config/mqttSettings.json)         | MQTT connection settings
-[otaSettings.json](data/config/otaSettings.json) | OTA update configuration
+[ntpSettings.json](data/config/ntpSettings.json)           | NTP synchronization settings
+[otaSettings.json](data/config/otaSettings.json)           | OTA update configuration
 [securitySettings.json](data/config/securitySettings.json) | Security settings and user credentials
-[wifiSettings.json](data/config/wifiSettings.json) | WiFi connection settings
+[wifiSettings.json](data/config/wifiSettings.json)         | WiFi connection settings
 
-### Access point settings
+These files can be pre-loaded with default configuration and [uploaded to the device](#uploading-the-file-system-image) if required. There are sensible defaults provided by the firmware, so this is optional.
+
+### Default access point settings
 
 The default settings configure the device to bring up an access point on start up which can be used to configure the device:
 
@@ -176,7 +182,7 @@ Username | Password
 admin    | admin
 guest    | guest
 
-It is recommended that you change the JWT secret and user credentials from their defaults protect your device. You can do this in the user interface, or by modifying [securitySettings.json](data/config/securitySettings.json) before uploading the file system image. 
+It is recommended that you change the JWT secret and user credentials from their defaults protect your device. You can do this in the user interface, or by modifying [securitySettings.json](data/config/securitySettings.json) before [uploading the file system image](#uploading-the-file-system-image). 
 
 ## Building for different devices
 
@@ -252,7 +258,7 @@ You can replace the app icon is located at ['interface/public/app/icon.png'](int
 
 The app name displayed on the login page and on the menu bar can be modified by editing the REACT_APP_NAME property in ['interface/.env'](interface/.env)
 
-```js
+```properties
 REACT_APP_NAME=Funky IoT Project
 ```
 
@@ -273,7 +279,7 @@ There is also a manifest file which contains the app name to use when adding the
 }
 ```
 
-## Back end overview
+## Back end
 
 The back end is a set of REST endpoints hosted by a [ESPAsyncWebServer](https://github.com/me-no-dev/ESPAsyncWebServer) instance. The ['lib/framework'](lib/framework) directory contains the majority of the back end code. The framework contains of a number of useful utility classes which you can use when extending it. The project also comes with a demo project to give you some help getting started. 
 
@@ -283,12 +289,11 @@ The framework's source is split up by feature, for example [WiFiScanner.h](lib/f
 
 The ['src/main.cpp'](src/main.cpp) file constructs the webserver and initializes the framework. You can add endpoints to the server here to support your IoT project. The main loop is also accessable so you can run your own code easily. 
 
-The following code creates the web server, esp8266React framework and the demo project instance:
+The following code creates the web server and esp8266React framework:
 
 ```cpp
 AsyncWebServer server(80);
 ESP8266React esp8266React(&server, &SPIFFS);
-DemoProject demoProject = DemoProject(&server, &SPIFFS, esp8266React.getSecurityManager());
 ```
 
 Now in the `setup()` function the initialization is performed:
@@ -308,43 +313,217 @@ void setup() {
   // start the framework and demo project
   esp8266React.begin();
 
-  // start the demo project
-  demoProject.begin();
-
   // start the server
   server.begin();
 }
 ```
 
-Finally the loop calls the framework's loop function to service the frameworks features. You can add your own code in here, as shown with the demo project:
+Finally the loop calls the framework's loop function to service the frameworks features.
 
 ```cpp
 void loop() {
   // run the framework's loop function
   esp8266React.loop();
-
-  // run the demo project's loop function
-  demoProject.loop();
 }
 ```
 
-### Adding endpoints
+### Developing with the framework
 
-There are some simple classes that support adding configurable services/features to the device:
+The framework promotes a modular design and exposes features you may re-use to speed up the development of your project. Where possible it is recommended that you use the features the frameworks supplies. These are documented in this section and a comprehensive example is provided by the demo project.
 
-Class | Description
+The following diagram visualises how the framework's modular components fit together, each feature is described in detail below.
------ | -----------
-[SimpleService.h](lib/framework/SimpleService.h) | Exposes an endpoint to read and write settings as JSON. Extend this class and implement the functions which serialize the settings to/from JSON.
-[SettingsService.h](lib/framework/SettingsService.h) | As above, however this class also handles persisting the settings as JSON to the file system.
-[AdminSettingsService.h](lib/framework/AdminSettingsService.h) | Extends SettingsService to secure the endpoint to administrators only, the authentication predicate can be overridden if required.
 
-The demo project shows how these can be used, explore the framework classes for more examples.
+![framework diagram](/media/framework.png?raw=true "framework diagram")
+
+#### Stateful service
+
+The [StatefulService.h](lib/framework/StatefulService.h) class is a responsible for managing state and interfacing with code which wants to change or respond to changes in that state. You can define a data class to hold some state, then build a StatefulService class to manage its state:
+
+```cpp
+class LightState {
+ public:
+  bool on = false;
+  uint8_t brightness = 255;
+};
+
+class LightStateService : public StatefulService<LightState> {
+};
+```
+
+You may listen for changes to state by registering an update handler callback. It is possible to remove an update handler later if required.
+
+```cpp
+// register an update handler
+update_handler_id_t myUpdateHandler = lightStateService.addUpdateHandler(
+  [&](String originId) {
+    Serial.println("The light's state has been updated"); 
+  }
+);
+
+// remove the update handler
+lightStateService.removeUpdateHandler(myUpdateHandler);
+```
+
+An "originId" is passed to the update handler which may be used to identify the origin of the update. The default origin values the framework provides are:
+
+Origin                | Description
+--------------------- | -----------
+http                  | An update sent over REST (HttpEndpoint)
+mqtt                  | An update sent over MQTT (MqttPubSub)
+websocket:{clientId}  | An update sent over WebSocket (WebSocketRxTx)
+
+StatefulService exposes a read function which you may use to safely read the state. This function takes care of protecting against parallel access to the state in multi-core enviornments such as the ESP32.
+
+```cpp
+lightStateService.read([&](LightState& state) {
+  digitalWrite(LED_PIN, state.on ? HIGH : LOW); // apply the state update to the LED_PIN
+});
+```
+
+StatefulService also exposes an update function which allows the caller to update the state with a callback. This approach automatically calls the registered update handlers when complete. The example below turns on the lights using the arbitrary origin "timer":
+
+```cpp
+lightStateService.update([&](LightState& state) {
+  state.on = true;  // turn on the lights!
+}, "timer");
+```
+
+#### Serialization
+
+When transmitting state over HTTP, WebSockets, or MQTT it must to be marshalled into a serializable form (JSON). The framework uses ArduinoJson for serialization and the functions defined in [JsonSerializer.h](lib/framework/JsonSerializer.h) and [JsonDeserializer.h](lib/framework/JsonDeserializer.h) facilitate this.
+
+The static functions below can be used to facilitate the serialization/deserialization of the light state:
+
+```cpp
+class LightState {
+ public:
+  bool on = false;
+  uint8_t brightness = 255;
+  
+  static void serialize(LightState& state, JsonObject& root) {
+    root["on"] = state.on;
+    root["brightness"] = state.brightness;
+  }
+
+  static void deserialize(JsonObject& root, LightState& state) {
+    state.on = root["on"] | false;
+    state.brightness = root["brightness"] | 255;
+  }
+};
+```
+
+For convenience, the StatefulService class provides overloads of its `update` and `read` functions which utilize these functions.
+
+Copy the state to a JsonObject using a serializer:
+
+```cpp
+JsonObject jsonObject = jsonDocument.to<JsonObject>();
+lightStateService->read(jsonObject, serializer);
+```
+
+Update the state from a JsonObject using a deserializer:
+
+```cpp
+JsonObject jsonObject = jsonDocument.as<JsonObject>();
+lightStateService->update(jsonObject, deserializer, "timer");
+```
+
+#### Endpoints
+
+The framework provides an [HttpEndpoint.h](lib/framework/HttpEndpoint.h) class which may be used to register GET and POST handlers to read and update the state over HTTP. You may construct an HttpEndpoint as a part of the StatefulService or separately if you prefer. 
+
+The code below demonstrates how to extend the LightStateService class to provide an unsecured endpoint:
+
+```cpp
+class LightStateService : public StatefulService<LightState> {
+ public:
+  LightStateService(AsyncWebServer* server) :
+      _httpEndpoint(LightState::serialize, LightState::deserialize, this, server, "/rest/lightState") {
+  }
+
+ private:
+  HttpEndpoint<LightState> _httpEndpoint;
+};
+```
+
+Endpoint security is provided by authentication predicates which are [documented below](#security-features). The SecurityManager and authentication predicate may be provided if a secure endpoint is required. The demo project shows how endpoints can be secured.
+
+#### Persistence
+
+[FSPersistence.h](lib/framework/FSPersistence.h) allows you to save state to the filesystem. FSPersistence automatically writes changes to the file system when state is updated. This feature can be disabled by calling `disableUpdateHandler()` if manual control of persistence is required.
+
+The code below demonstrates how to extend the LightStateService class to provide persistence:
+
+```cpp
+class LightStateService : public StatefulService<LightState> {
+ public:
+  LightStateService(FS* fs) :
+      _fsPersistence(LightState::serialize, LightState::deserialize, this, fs, "/config/lightState.json") {
+  }
+
+ private:
+  FSPersistence<LightState> _fsPersistence;
+};
+```
+
+#### WebSockets
+
+[WebSocketTxRx.h](lib/framework/WebSocketTxRx.h) allows you to read and update state over a WebSocket connection. WebSocketTxRx automatically pushes changes to all connected clients when state is updated.
+
+The code below demonstrates how to extend the LightStateService class to provide an unsecured WebSocket:
+
+```cpp
+class LightStateService : public StatefulService<LightState> {
+ public:
+  LightStateService(AsyncWebServer* server) :
+      _webSocket(LightState::serialize, LightState::deserialize, this, server, "/ws/lightState"), {
+  }
+
+ private:
+  WebSocketTxRx<LightState> _webSocket;
+};
+```
+
+WebSocket security is provided by authentication predicates which are [documented below](#security-features). The SecurityManager and authentication predicate may be provided if a secure WebSocket is required. The demo project shows how WebSockets can be secured.
+
+#### MQTT
+
+The framework includes an MQTT client which can be configured via the UI. MQTT requirements will differ from project to project so the framework exposes the client for you to use as you see fit. The framework does however provide a utility to interface StatefulService to a pair of pub/sub (state/set) topics. This utility can be used to synchronize state with software such as Home Assistant.
+
+[MqttPubSub.h](lib/framework/MqttPubSub.h) allows you to publish and subscribe to synchronize state over a pair of MQTT topics. MqttPubSub automatically pushes changes to the "pub" topic and reads updates from the "sub" topic.
+
+The code below demonstrates how to extend the LightStateService class to interface with MQTT:
+
+```cpp
+class LightStateService : public StatefulService<LightState> {
+ public:
+  LightStateService(AsyncMqttClient* mqttClient) :
+    _mqttPubSub(LightState::serialize, 
+                    LightState::deserialize,
+                    this,
+                    mqttClient,
+                    "homeassistant/light/my_light/set",
+                    "homeassistant/light/my_light/state") {
+  }
+
+ private:
+  MqttPubSub<LightState> _mqttPubSub;
+};
+```
+
+You can re-configure the pub/sub topics at runtime as required:
+
+```cpp
+_mqttPubSub.configureBroker("homeassistant/light/desk_lamp/set", "homeassistant/light/desk_lamp/state");
+```
+
+The demo project allows the user to modify the MQTT topics via the UI so they can be changed without re-flashing the firmware.
 
 ### Security features
 
 The framework has security features to prevent unauthorized use of the device. This is driven by [SecurityManager.h](lib/framework/SecurityManager.h).
 
-On successful authentication, the /rest/signIn endpoint issues a JWT which is then sent using Bearer Authentication. The framework come with built in predicates for verifying a users access level. The built in AuthenticationPredicates can be found in [SecurityManager.h](lib/framework/SecurityManager.h):
+On successful authentication, the /rest/signIn endpoint issues a [JSON Web Token (JWT)](https://jwt.io/) which is then sent using Bearer Authentication. The framework come with built-in predicates for verifying a users access privileges. The built in AuthenticationPredicates can be found in [SecurityManager.h](lib/framework/SecurityManager.h) and are as follows:
 
 Predicate            | Description
 -------------------- | -----------
@@ -352,7 +531,7 @@ NONE_REQUIRED        | No authentication is required.
 IS_AUTHENTICATED     | Any authenticated principal is permitted.
 IS_ADMIN             | The authenticated principal must be an admin.
 
-You can use the security manager to wrap any web handler with an authentication predicate:
+You can use the security manager to wrap any request handler function with an authentication predicate:
 
 ```cpp
 server->on("/rest/someService", HTTP_GET, 
@@ -360,88 +539,11 @@ server->on("/rest/someService", HTTP_GET,
 );
 ```
 
-Alternatively you can extend [AdminSettingsService.h](lib/framework/AdminSettingsService.h) and optionally override `getAuthenticationPredicate()` to secure an endpoint.
-
-## Extending the framework
-
-It is recommend that you explore the framework code to gain a better understanding of how to use it's features. The framework provides APIs so you can add your own services or features or, if required, directly configure or observe changes to core framework features. Some of these capabilities are detailed below.
-
-### Adding a service with persistant settings
-
-The following code demonstrates how you might extend the framework with a feature which requires a username and password to be configured to drive an unspecified feature. 
-
-```cpp
-#include <SettingsService.h>
-
-class ExampleSettings {
- public:  
-    String username;
-    String password;
-};
-
-class ExampleSettingsService : public SettingsService<ExampleSettings> {
-
-  public:
-
-    ExampleSettingsService(AsyncWebServer* server, FS* fs)
-    : SettingsService(server, fs, "/exampleSettings", "/config/exampleSettings.json") {}
-
-    ~ExampleSettingsService(){}
-
-  protected:
-
-    void readFromJsonObject(JsonObject& root) {
-      _settings.username = root["username"] | "";
-      _settings.password = root["password"] | "";
-    }
-
-    void writeToJsonObject(JsonObject& root) {
-      root["username"] = _settings.username;
-      root["password"] = _settings.password;
-    }
-
-};
-```
-
-Now this can be constructed, added to the server, and started as such:
-
-```cpp
-ExampleSettingsService exampleSettingsService = ExampleSettingsService(&server, &SPIFFS);
-
-exampleSettingsService.begin();
-```
-
-There will now be a REST service exposed on "/exampleSettings" for reading and writing (GET/POST) the settings. Any modifications will be persisted in SPIFFS, in this case to "/config/exampleSettings.json"
-
-Sometimes you need to perform an action when the settings are updated, you can achieve this by overriding the onConfigUpdated() function which gets called every time the settings are updated. You can also perform an action when the service starts by overriding the begin() function, being sure to call SettingsService::begin(). You can also provide a "loop" function in order to allow your service class continuously perform an action, calling this from the main loop.
-
-```cpp
-
-void begin() {
-  // make sure we call super, so the settings get read!
-  SettingsService::begin();  
-  reconfigureTheService();
-}
-
-void onConfigUpdated() {
-  reconfigureTheService();
-}
-
-void reconfigureTheService() {
-  // do whatever is required to react to the new settings
-}
-
-void loop() {
-  // execute somthing as part of the main loop
-}
-
-```
-
 ### Accessing settings and services
 
-The framework supplies access to it's SettingsService instances and the SecurityManager via getter functions:
+The framework supplies access to various features via getter functions:
 
-SettingsService               | Description
+SettingsService              | Description
 ---------------------------- | ----------------------------------------------
 getSecurityManager()         | The security manager - detailed above
 getSecuritySettingsService() | Configures the users and other security settings
@@ -449,38 +551,44 @@ getWiFiSettingsService()     | Configures and manages the WiFi network connectio
 getAPSettingsService()       | Configures and manages the Access Point
 getNTPSettingsService()      | Configures and manages the network time
 getOTASettingsService()      | Configures and manages the Over-The-Air update feature
+getMqttSettingsService()     | Configures and manages the MQTT connection
+getMqttClient()              | Provides direct access to the MQTT client instance
 
-These can be used to observe changes to settings. They can also be used to fetch or update settings directly via objects, JSON strings and JsonObjects. Here are some examples of how you may use this.
+The core features use the [StatefulService.h](lib/framework/StatefulService.h) class and can therefore you can change settings or observe changes to settings through the read/update API.
 
 Inspect the current WiFi settings:
 
 ```cpp
-WiFiSettings wifiSettings = esp8266React.getWiFiSettingsService()->fetch();
+esp8266React.getWiFiSettingsService()->read([&](WiFiSettings& wifiSettings) {
-Serial.print("The ssid is:");
+  Serial.print("The ssid is:");
-Serial.println(wifiSettings.ssid);
+  Serial.println(wifiSettings.ssid);
+});
 ```
 
-Configure the SSID and password:
+Configure the WiFi SSID and password manually:
 
 ```cpp
-WiFiSettings wifiSettings = esp8266React.getWiFiSettingsService()->fetch();
+esp8266React.getWiFiSettingsService()->update([&](WiFiSettings& wifiSettings) {
-wifiSettings.ssid = "MyNetworkSSID";
+  wifiSettings.ssid = "MyNetworkSSID";
-wifiSettings.password = "MySuperSecretPassword";
+  wifiSettings.password = "MySuperSecretPassword";
-esp8266React.getWiFiSettingsService()->update(wifiSettings);
+}, "myapp");
 ```
 
 Observe changes to the WiFiSettings:
 
 ```cpp
-esp8266React.getWiFiSettingsService()->addUpdateHandler([]() {
+esp8266React.getWiFiSettingsService()->addUpdateHandler(
-   Serial.println("The WiFi Settings were updated!");
+  [&](String originId) {
-});
+    Serial.println("The WiFi Settings were updated!");
+  }
+);
 ```
 
 ## Libraries Used
 
 * [React](https://reactjs.org/)
-* [Material-UI](https://material-ui-next.com/)
+* [Material-UI](https://material-ui.com/)
 * [notistack](https://github.com/iamhosseindhv/notistack)
 * [ArduinoJson](https://github.com/bblanchon/ArduinoJson)
 * [ESPAsyncWebServer](https://github.com/me-no-dev/ESPAsyncWebServer)
+* [AsyncMqttClient](https://github.com/marvinroger/async-mqtt-client)

data/config/demoSettings.json

@@ -1,3 +0,0 @@
-{
-  "blink_speed": 100
-}

data/config/mqttSettings.json

@@ -0,0 +1,11 @@
+{
+  "enabled": false,
+  "host": "test.mosquitto.org",
+  "port": 1883,
+  "authenticated": false,
+  "username": "mqttuser",
+  "password": "mqttpassword",
+  "keepAlive": 16,
+  "cleanSession": true,
+  "maxTopicLength": 128
+}

interface/.env.development

@@ -1,3 +1,4 @@
 # Change the IP address to that of your ESP device to enable local development of the UI.
 # Remember to also enable CORS in platformio.ini before uploading the code to the device.
-REACT_APP_ENDPOINT_ROOT=http://192.168.0.21/rest/
+REACT_APP_HTTP_ROOT=http://192.168.0.99
+REACT_APP_WEB_SOCKET_ROOT=ws://192.168.0.99

interface/.env.production

@@ -1,2 +1 @@
-REACT_APP_ENDPOINT_ROOT=/rest/
 GENERATE_SOURCEMAP=false

interface/package-lock.json

@@ -1611,6 +1611,11 @@
       "resolved": "https://registry.npmjs.org/@types/jwt-decode/-/jwt-decode-2.2.1.tgz",
       "integrity": "sha512-aWw2YTtAdT7CskFyxEX2K21/zSDStuf/ikI3yBqmwpwJF0pS+/IX5DWv+1UFffZIbruP6cnT9/LAJV1gFwAT1A=="
     },
+    "@types/lodash": {
+      "version": "4.14.149",
+      "resolved": "https://registry.npmjs.org/@types/lodash/-/lodash-4.14.149.tgz",
+      "integrity": "sha512-ijGqzZt/b7BfzcK9vTrS6MFljQRPn5BFWOx8oE0GYxribu6uV+aA9zZuXI1zc/etK9E8nrgdoF2+LgUw7+9tJQ=="
+    },
     "@types/material-ui": {
       "version": "0.21.7",
       "resolved": "https://registry.npmjs.org/@types/material-ui/-/material-ui-0.21.7.tgz",
@@ -12041,6 +12046,11 @@
         "kind-of": "^3.2.0"
       }
     },
+    "sockette": {
+      "version": "2.0.6",
+      "resolved": "https://registry.npmjs.org/sockette/-/sockette-2.0.6.tgz",
+      "integrity": "sha512-W6iG8RGV6Zife3Cj+FhuyHV447E6fqFM2hKmnaQrTvg3OydINV3Msj3WPFbX76blUlUxvQSMMMdrJxce8NqI5Q=="
+    },
     "sockjs": {
       "version": "0.3.19",
       "resolved": "https://registry.npmjs.org/sockjs/-/sockjs-0.3.19.tgz",

interface/package.json

@@ -6,6 +6,7 @@
     "@material-ui/core": "^4.9.8",
     "@material-ui/icons": "^4.9.1",
     "@types/jwt-decode": "^2.2.1",
+    "@types/lodash": "^4.14.149",
     "@types/node": "^12.12.32",
     "@types/react": "^16.9.27",
     "@types/react-dom": "^16.9.5",
@@ -14,6 +15,7 @@
     "@types/react-router-dom": "^5.1.3",
     "compression-webpack-plugin": "^3.0.1",
     "jwt-decode": "^2.2.0",
+    "lodash": "^4.17.15",
     "mime-types": "^2.1.25",
     "moment": "^2.24.0",
     "notistack": "^0.9.7",
@@ -24,6 +26,7 @@
     "react-router": "^5.1.2",
     "react-router-dom": "^5.1.2",
     "react-scripts": "3.4.1",
+    "sockette": "^2.0.6",
     "typescript": "^3.7.5",
     "zlib": "^1.0.5"
   },

interface/src/AppRouting.tsx

@@ -15,6 +15,7 @@ import Security from './security/Security';
 import System from './system/System';
 
 import { PROJECT_PATH } from './api';
+import Mqtt from './mqtt/Mqtt';
 
 class AppRouting extends Component {
 
@@ -31,6 +32,7 @@ class AppRouting extends Component {
           <AuthenticatedRoute exact path="/wifi/*" component={WiFiConnection} />         
           <AuthenticatedRoute exact path="/ap/*" component={AccessPoint} />
           <AuthenticatedRoute exact path="/ntp/*" component={NetworkTime} />
+          <AuthenticatedRoute exact path="/mqtt/*" component={Mqtt} />
           <AuthenticatedRoute exact path="/security/*" component={Security} /> 
           <AuthenticatedRoute exact path="/system/*" component={System} />          
           <Redirect to="/" />

interface/src/api/Endpoints.ts

@@ -9,6 +9,8 @@ export const LIST_NETWORKS_ENDPOINT = ENDPOINT_ROOT + "listNetworks";
 export const WIFI_SETTINGS_ENDPOINT = ENDPOINT_ROOT + "wifiSettings";
 export const WIFI_STATUS_ENDPOINT = ENDPOINT_ROOT + "wifiStatus";
 export const OTA_SETTINGS_ENDPOINT = ENDPOINT_ROOT + "otaSettings";
+export const MQTT_SETTINGS_ENDPOINT = ENDPOINT_ROOT + "mqttSettings";
+export const MQTT_STATUS_ENDPOINT = ENDPOINT_ROOT + "mqttStatus";
 export const SYSTEM_STATUS_ENDPOINT = ENDPOINT_ROOT + "systemStatus";
 export const SIGN_IN_ENDPOINT = ENDPOINT_ROOT + "signIn";
 export const VERIFY_AUTHORIZATION_ENDPOINT = ENDPOINT_ROOT + "verifyAuthorization";

interface/src/api/Env.ts

@@ -1,3 +1,24 @@
 export const PROJECT_NAME = process.env.REACT_APP_PROJECT_NAME!;
 export const PROJECT_PATH = process.env.REACT_APP_PROJECT_PATH!;
-export const ENDPOINT_ROOT = process.env.REACT_APP_ENDPOINT_ROOT!;
+
+export const ENDPOINT_ROOT = calculateEndpointRoot("/rest/");
+export const WEB_SOCKET_ROOT = calculateWebSocketRoot("/ws/");
+
+function calculateEndpointRoot(endpointPath: string) {
+    const httpRoot = process.env.REACT_APP_HTTP_ROOT;
+    if (httpRoot) {
+        return httpRoot + endpointPath;
+    }
+    const location = window.location;
+    return location.protocol + "//" + location.host + endpointPath;
+}
+
+function calculateWebSocketRoot(webSocketPath: string) {
+    const webSocketRoot = process.env.REACT_APP_WEB_SOCKET_ROOT;
+    if (webSocketRoot) {
+        return webSocketRoot + webSocketPath;
+    }
+    const location = window.location;
+    const webProtocol = location.protocol === "https:" ? "wss:" : "ws:";
+    return webProtocol + "//" + location.host + webSocketPath;
+}

interface/src/authentication/Authentication.ts

@@ -61,3 +61,13 @@ export function redirectingAuthorizedFetch(url: RequestInfo, params?: RequestIni
     });
   });
 }
+
+export function addAccessTokenParameter(url: string) {
+  const accessToken = localStorage.getItem(ACCESS_TOKEN);
+  if (!accessToken) {
+    return url;
+  }
+  const parsedUrl = new URL(url);
+  parsedUrl.searchParams.set(ACCESS_TOKEN, accessToken);
+  return parsedUrl.toString();
+}

interface/src/components/MenuAppBar.tsx

@@ -13,6 +13,7 @@ import SettingsIcon from '@material-ui/icons/Settings';
 import AccessTimeIcon from '@material-ui/icons/AccessTime';
 import AccountCircleIcon from '@material-ui/icons/AccountCircle';
 import SettingsInputAntennaIcon from '@material-ui/icons/SettingsInputAntenna';
+import DeviceHubIcon from '@material-ui/icons/DeviceHub';
 import LockIcon from '@material-ui/icons/Lock';
 import MenuIcon from '@material-ui/icons/Menu';
 
@@ -136,6 +137,12 @@ class MenuAppBar extends React.Component<MenuAppBarProps, MenuAppBarState> {
             </ListItemIcon>
             <ListItemText primary="Network Time" />
           </ListItem>
+          <ListItem to='/mqtt/' selected={path.startsWith('/mqtt/')} button component={Link}>
+            <ListItemIcon>
+              <DeviceHubIcon />
+            </ListItemIcon>
+            <ListItemText primary="MQTT" />
+          </ListItem>          
           <ListItem to='/security/' selected={path.startsWith('/security/')} button component={Link} disabled={!authenticatedContext.me.admin}>
             <ListItemIcon>
               <LockIcon />

interface/src/components/RestController.tsx

@@ -5,9 +5,8 @@ import { redirectingAuthorizedFetch } from '../authentication';
 
 export interface RestControllerProps<D> extends WithSnackbarProps {
   handleValueChange: (name: keyof D) => (event: React.ChangeEvent<HTMLInputElement>) => void;
-  handleSliderChange: (name: keyof D) => (event: React.ChangeEvent<{}>, value: number | number[]) => void;
 
-  setData: (data: D) => void;
+  setData: (data: D, callback?: () => void) => void;
   saveData: () => void;
   loadData: () => void;
 
@@ -16,13 +15,7 @@ export interface RestControllerProps<D> extends WithSnackbarProps {
   errorMessage?: string;
 }
 
-interface RestControllerState<D> {
+export const extractEventValue = (event: React.ChangeEvent<HTMLInputElement>) => {
-  data?: D;
-  loading: boolean;
-  errorMessage?: string;
-}
-
-const extractValue = (event: React.ChangeEvent<HTMLInputElement>) => {
   switch (event.target.type) {
     case "number":
       return event.target.valueAsNumber;
@@ -33,6 +26,12 @@ const extractValue = (event: React.ChangeEvent<HTMLInputElement>) => {
   }
 }
 
+interface RestControllerState<D> {
+  data?: D;
+  loading: boolean;
+  errorMessage?: string;
+}
+
 export function restController<D, P extends RestControllerProps<D>>(endpointUrl: string, RestController: React.ComponentType<P & RestControllerProps<D>>) {
   return withSnackbar(
     class extends React.Component<Omit<P, keyof RestControllerProps<D>> & WithSnackbarProps, RestControllerState<D>> {
@@ -43,12 +42,12 @@ export function restController<D, P extends RestControllerProps<D>>(endpointUrl:
         errorMessage: undefined
       };
 
-      setData = (data: D) => {
+      setData = (data: D, callback?: () => void) => {
         this.setState({
           data,
           loading: false,
           errorMessage: undefined
-        });
+        }, callback);
       }
 
       loadData = () => {
@@ -95,19 +94,13 @@ export function restController<D, P extends RestControllerProps<D>>(endpointUrl:
       }
 
       handleValueChange = (name: keyof D) => (event: React.ChangeEvent<HTMLInputElement>) => {
-        const data = { ...this.state.data!, [name]: extractValue(event) };
+        const data = { ...this.state.data!, [name]: extractEventValue(event) };
         this.setState({ data });
       }
 
-      handleSliderChange = (name: keyof D) => (event: React.ChangeEvent<{}>, value: number | number[]) => {
-        const data = { ...this.state.data!, [name]: value };
-        this.setState({ data });
-      };
-
       render() {
         return <RestController
           handleValueChange={this.handleValueChange}
-          handleSliderChange={this.handleSliderChange}
           setData={this.setData}
           saveData={this.saveData}
           loadData={this.loadData}

interface/src/components/RestFormLoader.tsx

@@ -2,7 +2,8 @@ import React from 'react';
 
 import { makeStyles, Theme, createStyles } from '@material-ui/core/styles';
 import { Button, LinearProgress, Typography } from '@material-ui/core';
-import { RestControllerProps } from './RestController';
+
+import { RestControllerProps } from '.';
 
 const useStyles = makeStyles((theme: Theme) =>
   createStyles({

interface/src/components/WebSocketController.tsx

@@ -0,0 +1,133 @@
+import React from 'react';
+import Sockette from 'sockette';
+import throttle from 'lodash/throttle';
+import { withSnackbar, WithSnackbarProps } from 'notistack';
+
+import { addAccessTokenParameter } from '../authentication';
+import { extractEventValue } from '.';
+
+export interface WebSocketControllerProps<D> extends WithSnackbarProps {
+  handleValueChange: (name: keyof D) => (event: React.ChangeEvent<HTMLInputElement>) => void;
+
+  setData: (data: D, callback?: () => void) => void;
+  saveData: () => void;
+  saveDataAndClear(): () => void;
+
+  connected: boolean;
+  data?: D;
+}
+
+interface WebSocketControllerState<D> {
+  ws: Sockette;
+  connected: boolean;
+  clientId?: string;
+  data?: D;
+}
+
+enum WebSocketMessageType {
+  ID = "id",
+  PAYLOAD = "payload"
+}
+
+interface WebSocketIdMessage {
+  type: typeof WebSocketMessageType.ID;
+  id: string;
+}
+
+interface WebSocketPayloadMessage<D> {
+  type: typeof WebSocketMessageType.PAYLOAD;
+  origin_id: string;
+  payload: D;
+}
+
+export type WebSocketMessage<D> = WebSocketIdMessage | WebSocketPayloadMessage<D>;
+
+export function webSocketController<D, P extends WebSocketControllerProps<D>>(wsUrl: string, wsThrottle: number, WebSocketController: React.ComponentType<P & WebSocketControllerProps<D>>) {
+  return withSnackbar(
+    class extends React.Component<Omit<P, keyof WebSocketControllerProps<D>> & WithSnackbarProps, WebSocketControllerState<D>> {
+      constructor(props: Omit<P, keyof WebSocketControllerProps<D>> & WithSnackbarProps) {
+        super(props);
+        this.state = {
+          ws: new Sockette(addAccessTokenParameter(wsUrl), {
+            onmessage: this.onMessage,
+            onopen: this.onOpen,
+            onclose: this.onClose,
+          }),
+          connected: false
+        }
+      }
+
+      componentWillUnmount() {
+        this.state.ws.close();
+      }
+
+      onMessage = (event: MessageEvent) => {
+        const rawData = event.data;
+        if (typeof rawData === 'string' || rawData instanceof String) {
+          this.handleMessage(JSON.parse(rawData as string) as WebSocketMessage<D>);
+        }
+      }
+
+      handleMessage = (message: WebSocketMessage<D>) => {
+        switch (message.type) {
+          case WebSocketMessageType.ID:
+            this.setState({ clientId: message.id });
+            break;
+          case WebSocketMessageType.PAYLOAD:
+            const { clientId, data } = this.state;
+            if (clientId && (!data || clientId !== message.origin_id)) {
+              this.setState(
+                { data: message.payload }
+              );
+            }
+            break;
+        }
+      }
+
+      onOpen = () => {
+        this.setState({ connected: true });
+      }
+
+      onClose = () => {
+        this.setState({ connected: false, clientId: undefined, data: undefined });
+      }
+
+      setData = (data: D, callback?: () => void) => {
+        this.setState({ data }, callback);
+      }
+
+      saveData = throttle(() => {
+        const { ws, connected, data } = this.state;
+        if (connected) {
+          ws.json(data);
+        }
+      }, wsThrottle);
+
+      saveDataAndClear = throttle(() => {
+        const { ws, connected, data } = this.state;
+        if (connected) {
+          this.setState({
+            data: undefined
+          }, () => ws.json(data));
+        }
+      }, wsThrottle);
+
+      handleValueChange = (name: keyof D) => (event: React.ChangeEvent<HTMLInputElement>) => {
+        const data = { ...this.state.data!, [name]: extractEventValue(event) };
+        this.setState({ data });
+      }
+
+      render() {
+        return <WebSocketController
+          handleValueChange={this.handleValueChange}
+          setData={this.setData}
+          saveData={this.saveData}
+          saveDataAndClear={this.saveDataAndClear}
+          connected={this.state.connected}
+          data={this.state.data}
+          {...this.props as P}
+        />;
+      }
+
+    });
+}

interface/src/components/WebSocketFormLoader.tsx

@@ -0,0 +1,40 @@
+import React from 'react';
+
+import { makeStyles, Theme, createStyles } from '@material-ui/core/styles';
+import { LinearProgress, Typography } from '@material-ui/core';
+
+import { WebSocketControllerProps } from '.';
+
+const useStyles = makeStyles((theme: Theme) =>
+  createStyles({
+    loadingSettings: {
+      margin: theme.spacing(0.5),
+    },
+    loadingSettingsDetails: {
+      margin: theme.spacing(4),
+      textAlign: "center"
+    }
+  })
+);
+
+export type WebSocketFormProps<D> = Omit<WebSocketControllerProps<D>, "connected"> & { data: D };
+
+interface WebSocketFormLoaderProps<D> extends WebSocketControllerProps<D> {
+  render: (props: WebSocketFormProps<D>) => JSX.Element;
+}
+
+export default function WebSocketFormLoader<D>(props: WebSocketFormLoaderProps<D>) {
+  const { connected, render, data, ...rest } = props;
+  const classes = useStyles();
+  if (!connected || !data) {
+    return (
+      <div className={classes.loadingSettings}>
+        <LinearProgress className={classes.loadingSettingsDetails} />
+        <Typography variant="h6" className={classes.loadingSettingsDetails}>
+          Connecting to WebSocket...
+        </Typography>
+      </div>
+    );
+  }
+  return render({ ...rest, data });
+}

interface/src/components/index.ts

@@ -6,6 +6,10 @@ export { default as MenuAppBar } from './MenuAppBar';
 export { default as PasswordValidator } from './PasswordValidator';
 export { default as RestFormLoader } from './RestFormLoader';
 export { default as SectionContent } from './SectionContent';
+export { default as WebSocketFormLoader } from './WebSocketFormLoader';
 
 export * from './RestFormLoader';
 export * from './RestController';
+
+export * from './WebSocketFormLoader';
+export * from './WebSocketController';

interface/src/mqtt/Mqtt.tsx

@@ -0,0 +1,37 @@
+import React, { Component } from 'react';
+import { Redirect, Switch, RouteComponentProps } from 'react-router-dom'
+
+import { Tabs, Tab } from '@material-ui/core';
+
+import { AuthenticatedContextProps, withAuthenticatedContext, AuthenticatedRoute } from '../authentication';
+import { MenuAppBar } from '../components';
+import MqttStatusController from './MqttStatusController';
+import MqttSettingsController from './MqttSettingsController';
+
+type MqttProps = AuthenticatedContextProps & RouteComponentProps;
+
+class Mqtt extends Component<MqttProps> {
+
+  handleTabChange = (event: React.ChangeEvent<{}>, path: string) => {
+    this.props.history.push(path);
+  };
+
+  render() {
+    const { authenticatedContext } = this.props;
+    return (
+      <MenuAppBar sectionTitle="MQTT">
+        <Tabs value={this.props.match.url} onChange={this.handleTabChange} variant="fullWidth">
+          <Tab value="/mqtt/status" label="MQTT Status" />
+          <Tab value="/mqtt/settings" label="MQTT Settings" disabled={!authenticatedContext.me.admin} />
+        </Tabs>
+        <Switch>
+          <AuthenticatedRoute exact path="/mqtt/status" component={MqttStatusController} />
+          <AuthenticatedRoute exact path="/mqtt/settings" component={MqttSettingsController} />
+          <Redirect to="/mqtt/status" />
+        </Switch>
+      </MenuAppBar>
+    )
+  }
+}
+
+export default withAuthenticatedContext(Mqtt);

interface/src/mqtt/MqttSettingsController.tsx

@@ -0,0 +1,30 @@
+import React, { Component } from 'react';
+
+import {restController, RestControllerProps, RestFormLoader, SectionContent } from '../components';
+import { MQTT_SETTINGS_ENDPOINT } from '../api';
+
+import MqttSettingsForm from './MqttSettingsForm';
+import { MqttSettings } from './types';
+
+type MqttSettingsControllerProps = RestControllerProps<MqttSettings>;
+
+class MqttSettingsController extends Component<MqttSettingsControllerProps> {
+
+  componentDidMount() {
+    this.props.loadData();
+  }
+
+  render() {
+    return (
+      <SectionContent title="MQTT Settings" titleGutter>
+        <RestFormLoader
+          {...this.props}
+          render={formProps => <MqttSettingsForm {...formProps} />}
+        />
+      </SectionContent>
+    )
+  }
+
+}
+
+export default restController(MQTT_SETTINGS_ENDPOINT, MqttSettingsController);

interface/src/mqtt/MqttSettingsForm.tsx

@@ -0,0 +1,131 @@
+import React from 'react';
+import { TextValidator, ValidatorForm } from 'react-material-ui-form-validator';
+
+import { Checkbox, TextField } from '@material-ui/core';
+import SaveIcon from '@material-ui/icons/Save';
+
+import { RestFormProps, FormActions, FormButton, BlockFormControlLabel, PasswordValidator } from '../components';
+import { isIP, isHostname, or } from '../validators';
+
+import { MqttSettings } from './types';
+
+type MqttSettingsFormProps = RestFormProps<MqttSettings>;
+
+class MqttSettingsForm extends React.Component<MqttSettingsFormProps> {
+
+  componentDidMount() {
+    ValidatorForm.addValidationRule('isIPOrHostname', or(isIP, isHostname));
+  }
+
+  render() {
+    const { data, handleValueChange, saveData, loadData } = this.props;
+    return (
+      <ValidatorForm onSubmit={saveData}>
+        <BlockFormControlLabel
+          control={
+            <Checkbox
+              checked={data.enabled}
+              onChange={handleValueChange('enabled')}
+              value="enabled"
+            />
+          }
+          label="Enable MQTT?"
+        />
+        <TextValidator
+          validators={['required', 'isIPOrHostname']}
+          errorMessages={['Host is required', "Not a valid IP address or hostname"]}
+          name="host"
+          label="Host"
+          fullWidth
+          variant="outlined"
+          value={data.host}
+          onChange={handleValueChange('host')}
+          margin="normal"
+        />
+        <TextValidator
+          validators={['required', 'isNumber', 'minNumber:0', 'maxNumber:65535']}
+          errorMessages={['Port is required', "Must be a number", "Must be greater than 0 ", "Max value is 65535"]}
+          name="port"
+          label="Port"
+          fullWidth
+          variant="outlined"
+          value={data.port}
+          type="number"
+          onChange={handleValueChange('port')}
+          margin="normal"
+        />
+        <TextField
+          name="username"
+          label="Username"
+          fullWidth
+          variant="outlined"
+          value={data.username}
+          onChange={handleValueChange('username')}
+          margin="normal"
+        />
+        <PasswordValidator
+          name="password"
+          label="Password"
+          fullWidth
+          variant="outlined"
+          value={data.password}
+          onChange={handleValueChange('password')}
+          margin="normal"
+        />
+        <TextField
+          name="client_id"
+          label="Client ID (optional)"
+          fullWidth
+          variant="outlined"
+          value={data.client_id}
+          onChange={handleValueChange('client_id')}
+          margin="normal"
+        />
+        <TextValidator
+          validators={['required', 'isNumber', 'minNumber:1', 'maxNumber:65535']}
+          errorMessages={['Keep alive is required', "Must be a number", "Must be greater than 0", "Max value is 65535"]}
+          name="keep_alive"
+          label="Keep Alive (seconds)"
+          fullWidth
+          variant="outlined"
+          value={data.keep_alive}
+          type="number"
+          onChange={handleValueChange('keep_alive')}
+          margin="normal"
+        />
+        <BlockFormControlLabel
+          control={
+            <Checkbox
+              checked={data.clean_session}
+              onChange={handleValueChange('clean_session')}
+              value="clean_session"
+            />
+          }
+          label="Clean Session?"
+        />
+        <TextValidator
+          validators={['required', 'isNumber', 'minNumber:1', 'maxNumber:65535']}
+          errorMessages={['Max topic length is required', "Must be a number", "Must be greater than 0", "Max value is 65535"]}
+          name="max_topic_length"
+          label="Max Topic Length"
+          fullWidth
+          variant="outlined"
+          value={data.max_topic_length}
+          type="number"
+          onChange={handleValueChange('max_topic_length')}
+          margin="normal"
+        />
+        <FormActions>
+          <FormButton startIcon={<SaveIcon />} variant="contained" color="primary" type="submit">
+            Save
+          </FormButton>
+          <FormButton variant="contained" color="secondary" onClick={loadData}>
+            Reset
+          </FormButton>
+        </FormActions>
+      </ValidatorForm>
+    );
+  }
+}
+
+export default MqttSettingsForm;

interface/src/mqtt/MqttStatus.ts

@@ -0,0 +1,45 @@
+import { Theme } from "@material-ui/core";
+import { MqttStatus, MqttDisconnectReason } from "./types";
+
+export const mqttStatusHighlight = ({ enabled, connected }: MqttStatus, theme: Theme) => {
+  if (!enabled) {
+    return theme.palette.info.main;
+  }
+  if (connected) {
+    return theme.palette.success.main;
+  }
+  return theme.palette.error.main;
+}
+
+export const mqttStatus = ({ enabled, connected }: MqttStatus) => {
+  if (!enabled) {
+    return "Not enabled";
+  }
+  if (connected) {
+    return "Connected";
+  }
+  return "Disconnected";
+}
+
+export const disconnectReason = ({ disconnect_reason }: MqttStatus) => {
+  switch (disconnect_reason) {
+    case MqttDisconnectReason.TCP_DISCONNECTED:
+      return "TCP disconnected";
+    case MqttDisconnectReason.MQTT_UNACCEPTABLE_PROTOCOL_VERSION:
+      return "Unacceptable protocol version";
+    case MqttDisconnectReason.MQTT_IDENTIFIER_REJECTED:
+      return "Client ID rejected";
+    case MqttDisconnectReason.MQTT_SERVER_UNAVAILABLE:
+      return "Server unavailable";
+    case MqttDisconnectReason.MQTT_MALFORMED_CREDENTIALS:
+      return "Malformed credentials";
+    case MqttDisconnectReason.MQTT_NOT_AUTHORIZED:
+      return "Not authorized";
+    case MqttDisconnectReason.ESP8266_NOT_ENOUGH_SPACE:
+      return "Device out of memory";
+    case MqttDisconnectReason.TLS_BAD_FINGERPRINT:
+      return "Server fingerprint invalid";
+    default:
+      return "Unknown"
+  }
+}

interface/src/mqtt/MqttStatusController.tsx

@@ -0,0 +1,29 @@
+import React, { Component } from 'react';
+
+import {restController, RestControllerProps, RestFormLoader, SectionContent } from '../components';
+import { MQTT_STATUS_ENDPOINT } from '../api';
+
+import MqttStatusForm from './MqttStatusForm';
+import { MqttStatus } from './types';
+
+type MqttStatusControllerProps = RestControllerProps<MqttStatus>;
+
+class MqttStatusController extends Component<MqttStatusControllerProps> {
+
+  componentDidMount() {
+    this.props.loadData();
+  }
+
+  render() {
+    return (
+      <SectionContent title="MQTT Status">
+        <RestFormLoader
+          {...this.props}
+          render={formProps => <MqttStatusForm {...formProps} />}
+        />
+      </SectionContent>
+    )
+  }
+}
+
+export default restController(MQTT_STATUS_ENDPOINT, MqttStatusController);

interface/src/mqtt/MqttStatusForm.tsx

@@ -0,0 +1,83 @@
+import React, { Component, Fragment } from 'react';
+
+import { WithTheme, withTheme } from '@material-ui/core/styles';
+import { Avatar, Divider, List, ListItem, ListItemAvatar, ListItemText } from '@material-ui/core';
+
+import DeviceHubIcon from '@material-ui/icons/DeviceHub';
+import RefreshIcon from '@material-ui/icons/Refresh';
+import ReportIcon from '@material-ui/icons/Report';
+
+import { RestFormProps, FormActions, FormButton, HighlightAvatar } from '../components';
+import { mqttStatusHighlight, mqttStatus, disconnectReason } from './MqttStatus';
+import { MqttStatus } from './types';
+
+type MqttStatusFormProps = RestFormProps<MqttStatus> & WithTheme;
+
+class MqttStatusForm extends Component<MqttStatusFormProps> {
+
+  renderConnectionStatus() {
+    const { data } = this.props
+    if (data.connected) {
+      return (
+        <Fragment>
+          <ListItem>
+            <ListItemAvatar>
+              <Avatar>#</Avatar>
+            </ListItemAvatar>
+            <ListItemText primary="Client ID" secondary={data.client_id} />
+          </ListItem>
+          <Divider variant="inset" component="li" />
+        </Fragment>
+      );
+    }
+    return (
+      <Fragment>
+        <ListItem>
+          <ListItemAvatar>
+            <Avatar>
+              <ReportIcon />
+            </Avatar>
+          </ListItemAvatar>
+          <ListItemText primary="Disconnect Reason" secondary={disconnectReason(data)} />
+        </ListItem>
+        <Divider variant="inset" component="li" />
+      </Fragment>
+    );
+  }
+
+  createListItems() {
+    const { data, theme } = this.props
+    return (
+      <Fragment>
+        <ListItem>
+          <ListItemAvatar>
+            <HighlightAvatar color={mqttStatusHighlight(data, theme)}>
+              <DeviceHubIcon />
+            </HighlightAvatar>
+          </ListItemAvatar>
+          <ListItemText primary="Status" secondary={mqttStatus(data)} />
+        </ListItem>
+        <Divider variant="inset" component="li" />
+        {data.enabled && this.renderConnectionStatus()}
+      </Fragment>
+    );
+  }
+
+  render() {
+    return (
+      <Fragment>
+        <List>
+          {this.createListItems()}
+        </List>
+        <FormActions>
+          <FormButton startIcon={<RefreshIcon />} variant="contained" color="secondary" onClick={this.props.loadData}>
+            Refresh
+          </FormButton>
+        </FormActions>
+      </Fragment>
+    );
+  }
+
+}
+
+export default withTheme(MqttStatusForm);

interface/src/mqtt/types.ts

@@ -0,0 +1,29 @@
+export enum MqttDisconnectReason {
+  TCP_DISCONNECTED = 0,
+  MQTT_UNACCEPTABLE_PROTOCOL_VERSION = 1,
+  MQTT_IDENTIFIER_REJECTED = 2,
+  MQTT_SERVER_UNAVAILABLE = 3,
+  MQTT_MALFORMED_CREDENTIALS = 4,
+  MQTT_NOT_AUTHORIZED = 5,
+  ESP8266_NOT_ENOUGH_SPACE = 6,
+  TLS_BAD_FINGERPRINT = 7
+}
+
+export interface MqttStatus {
+  enabled: boolean;
+  connected: boolean;
+  client_id: string;
+  disconnect_reason: MqttDisconnectReason;
+}
+
+export interface MqttSettings {
+  enabled: boolean;
+  host: string;
+  port: number;
+  username: string;
+  password: string;
+  client_id: string;
+  keep_alive: number;
+  clean_session: boolean;
+  max_topic_length: number;
+}

interface/src/ntp/NTPSettingsForm.tsx

@@ -56,11 +56,10 @@ class NTPSettingsForm extends React.Component<NTPSettingsFormProps> {
           validators={['required']}
           errorMessages={['Time zone is required']}
           name="tz_label"
-          labelId="tz_label"
           label="Time zone"
           fullWidth
           variant="outlined"
-          native
+          native="true"
           value={selectedTimeZone(data.tz_label, data.tz_format)}
           onChange={this.changeTimeZone}
           margin="normal"

interface/src/ntp/TZ.tsx

@@ -474,6 +474,6 @@ export function selectedTimeZone(label: string, format: string) {
 
 export function timeZoneSelectItems() {
   return Object.keys(TIME_ZONES).map(label => (
-    <MenuItem value={label}>{label}</MenuItem>
+    <MenuItem key={label} value={label}>{label}</MenuItem>
   ));
 }

interface/src/project/DemoController.tsx

@@ -1,75 +0,0 @@
-import React, { Component } from 'react';
-import { ValidatorForm } from 'react-material-ui-form-validator';
-
-import { Typography, Slider, Box } from '@material-ui/core';
-import SaveIcon from '@material-ui/icons/Save';
-
-import { ENDPOINT_ROOT } from '../api';
-import { restController, RestControllerProps, RestFormLoader, RestFormProps, FormActions, FormButton, SectionContent } from '../components';
-
-export const DEMO_SETTINGS_ENDPOINT = ENDPOINT_ROOT + "demoSettings";
-
-interface DemoSettings {
-  blink_speed: number;
-}
-
-type DemoControllerProps = RestControllerProps<DemoSettings>;
-
-class DemoController extends Component<DemoControllerProps> {
-
-  componentDidMount() {
-    this.props.loadData();
-  }
-
-  render() {
-    return (
-      <SectionContent title='Demo Controller' titleGutter>
-        <RestFormLoader
-          {...this.props}
-          render={props => (
-            <DemoControllerForm {...props} />
-          )}
-        />
-      </SectionContent>
-    )
-  }
-
-}
-
-export default restController(DEMO_SETTINGS_ENDPOINT, DemoController);
-
-const valueToPercentage = (value: number) => `${Math.round(value / 255 * 100)}%`;
-
-type DemoControllerFormProps = RestFormProps<DemoSettings>;
-
-function DemoControllerForm(props: DemoControllerFormProps) {
-  const { data, saveData, loadData, handleSliderChange } = props;
-  return (
-    <ValidatorForm onSubmit={saveData}>
-      <Typography id="blink-speed-slider">
-        Blink Speed
-      </Typography>
-      <Box pt={5}>
-        <Slider
-          value={data.blink_speed}
-          valueLabelFormat={valueToPercentage}
-          aria-labelledby="blink-speed-slider"
-          valueLabelDisplay="on"
-          min={0}
-          max={255}
-          onChange={handleSliderChange('blink_speed')}
-        />
-      </Box>
-      <FormActions>
-        <FormButton startIcon={<SaveIcon />} variant="contained" color="primary" type="submit">
-          Save
-        </FormButton>
-        <FormButton variant="contained" color="secondary" onClick={loadData}>
-          Reset
-        </FormButton>
-      </FormActions>
-    </ValidatorForm>
-  );
-}
-
-

interface/src/project/DemoInformation.tsx

@@ -65,10 +65,26 @@ class DemoInformation extends Component {
             </TableRow>
             <TableRow>
               <TableCell>
-                DemoController.tsx
+                LightStateRestController.tsx
               </TableCell>
               <TableCell>
-                The demo controller tab, to control the built-in LED.
+                A form which lets the user control the LED over a REST service.
+              </TableCell>
+            </TableRow>
+            <TableRow>
+              <TableCell>
+                LightStateWebSocketController.tsx
+              </TableCell>
+              <TableCell>
+                A form which lets the user control and monitor the status of the LED over WebSockets.
+              </TableCell>
+            </TableRow>
+            <TableRow>
+              <TableCell>
+                LightMqttSettingsController.tsx
+              </TableCell>
+              <TableCell>
+                A form which lets the user change the MQTT settings for MQTT based control of the LED.
               </TableCell>
             </TableRow>
           </TableBody>

interface/src/project/DemoProject.tsx

@@ -8,7 +8,9 @@ import { MenuAppBar } from '../components';
 import { AuthenticatedRoute } from '../authentication';
 
 import DemoInformation from './DemoInformation';
-import DemoController from './DemoController';
+import LightStateRestController from './LightStateRestController';
+import LightStateWebSocketController from './LightStateWebSocketController';
+import LightMqttSettingsController from './LightMqttSettingsController';
 
 class DemoProject extends Component<RouteComponentProps> {
 
@@ -20,12 +22,16 @@ class DemoProject extends Component<RouteComponentProps> {
     return (
       <MenuAppBar sectionTitle="Demo Project">
         <Tabs value={this.props.match.url} onChange={this.handleTabChange} variant="fullWidth">
-          <Tab value={`/${PROJECT_PATH}/demo/information`} label="Demo Information" />
+          <Tab value={`/${PROJECT_PATH}/demo/information`} label="Information" />
-          <Tab value={`/${PROJECT_PATH}/demo/controller`} label="Demo Controller" />
+          <Tab value={`/${PROJECT_PATH}/demo/rest`} label="REST Controller" />
+          <Tab value={`/${PROJECT_PATH}/demo/socket`} label="WebSocket Controller" />
+          <Tab value={`/${PROJECT_PATH}/demo/mqtt`} label="MQTT Controller" />
         </Tabs>
         <Switch>
           <AuthenticatedRoute exact path={`/${PROJECT_PATH}/demo/information`} component={DemoInformation} />
-          <AuthenticatedRoute exact path={`/${PROJECT_PATH}/demo/controller`} component={DemoController} />
+          <AuthenticatedRoute exact path={`/${PROJECT_PATH}/demo/rest`} component={LightStateRestController} />
+          <AuthenticatedRoute exact path={`/${PROJECT_PATH}/demo/socket`} component={LightStateWebSocketController} />
+          <AuthenticatedRoute exact path={`/${PROJECT_PATH}/demo/mqtt`} component={LightMqttSettingsController} />
           <Redirect to={`/${PROJECT_PATH}/demo/information`} />
         </Switch>
       </MenuAppBar>

interface/src/project/LightMqttSettingsController.tsx

@@ -0,0 +1,93 @@
+import React, { Component } from 'react';
+import { ValidatorForm, TextValidator } from 'react-material-ui-form-validator';
+
+import { Typography, Box } from '@material-ui/core';
+import SaveIcon from '@material-ui/icons/Save';
+
+import { ENDPOINT_ROOT } from '../api';
+import { restController, RestControllerProps, RestFormLoader, RestFormProps, FormActions, FormButton, SectionContent } from '../components';
+
+import { LightMqttSettings } from './types';
+
+export const LIGHT_BROKER_SETTINGS_ENDPOINT = ENDPOINT_ROOT + "brokerSettings";
+
+type LightMqttSettingsControllerProps = RestControllerProps<LightMqttSettings>;
+
+class LightMqttSettingsController extends Component<LightMqttSettingsControllerProps> {
+
+  componentDidMount() {
+    this.props.loadData();
+  }
+
+  render() {
+    return (
+      <SectionContent title='MQTT Controller' titleGutter>
+        <RestFormLoader
+          {...this.props}
+          render={props => (
+            <LightMqttSettingsControllerForm {...props} />
+          )}
+        />
+      </SectionContent>
+    )
+  }
+
+}
+
+export default restController(LIGHT_BROKER_SETTINGS_ENDPOINT, LightMqttSettingsController);
+
+type LightMqttSettingsControllerFormProps = RestFormProps<LightMqttSettings>;
+
+function LightMqttSettingsControllerForm(props: LightMqttSettingsControllerFormProps) {
+  const { data, saveData, loadData, handleValueChange } = props;
+  return (
+    <ValidatorForm onSubmit={saveData}>
+      <Box bgcolor="primary.main" color="primary.contrastText" p={2} mt={2} mb={2}>
+        <Typography variant="body1">
+          The LED is controllable via MQTT with the demo project designed to work with Home Assistant's auto discovery feature.
+        </Typography>
+      </Box>
+      <TextValidator
+        validators={['required']}
+        errorMessages={['Unique ID is required']}
+        name="unique_id"
+        label="Unique ID"
+        fullWidth
+        variant="outlined"
+        value={data.unique_id}
+        onChange={handleValueChange('unique_id')}
+        margin="normal"
+      />
+      <TextValidator
+        validators={['required']}
+        errorMessages={['Name is required']}
+        name="name"
+        label="Name"
+        fullWidth
+        variant="outlined"
+        value={data.name}
+        onChange={handleValueChange('name')}
+        margin="normal"
+      />
+      <TextValidator
+        validators={['required']}
+        errorMessages={['MQTT Path is required']}
+        name="mqtt_path"
+        label="MQTT Path"
+        fullWidth
+        variant="outlined"
+        value={data.mqtt_path}
+        onChange={handleValueChange('mqtt_path')}
+        margin="normal"
+      />
+      <FormActions>
+        <FormButton startIcon={<SaveIcon />} variant="contained" color="primary" type="submit">
+          Save
+        </FormButton>
+        <FormButton variant="contained" color="secondary" onClick={loadData}>
+          Reset
+        </FormButton>
+      </FormActions>
+    </ValidatorForm>
+  );
+}

interface/src/project/LightStateRestController.tsx

@@ -0,0 +1,70 @@
+import React, { Component } from 'react';
+import { ValidatorForm } from 'react-material-ui-form-validator';
+
+import { Typography, Box, Checkbox } from '@material-ui/core';
+import SaveIcon from '@material-ui/icons/Save';
+
+import { ENDPOINT_ROOT } from '../api';
+import { restController, RestControllerProps, RestFormLoader, RestFormProps, FormActions, FormButton, SectionContent, BlockFormControlLabel } from '../components';
+
+import { LightState } from './types';
+
+export const LIGHT_SETTINGS_ENDPOINT = ENDPOINT_ROOT + "lightState";
+
+type LightStateRestControllerProps = RestControllerProps<LightState>;
+
+class LightStateRestController extends Component<LightStateRestControllerProps> {
+
+  componentDidMount() {
+    this.props.loadData();
+  }
+
+  render() {
+    return (
+      <SectionContent title='REST Controller' titleGutter>
+        <RestFormLoader
+          {...this.props}
+          render={props => (
+            <LightStateRestControllerForm {...props} />
+          )}
+        />
+      </SectionContent>
+    )
+  }
+
+}
+
+export default restController(LIGHT_SETTINGS_ENDPOINT, LightStateRestController);
+
+type LightStateRestControllerFormProps = RestFormProps<LightState>;
+
+function LightStateRestControllerForm(props: LightStateRestControllerFormProps) {
+  const { data, saveData, loadData, handleValueChange } = props;
+  return (
+    <ValidatorForm onSubmit={saveData}>
+      <Box bgcolor="primary.main" color="primary.contrastText" p={2} mt={2} mb={2}>
+        <Typography variant="body1">
+          The form below controls the LED via the RESTful service exposed by the ESP device.
+        </Typography>
+      </Box>
+      <BlockFormControlLabel
+        control={
+          <Checkbox
+            checked={data.led_on}
+            onChange={handleValueChange('led_on')}
+            color="primary"
+          />
+        }
+        label="LED State?"
+      />
+      <FormActions>
+        <FormButton startIcon={<SaveIcon />} variant="contained" color="primary" type="submit">
+          Save
+        </FormButton>
+        <FormButton variant="contained" color="secondary" onClick={loadData}>
+          Reset
+        </FormButton>
+      </FormActions>
+    </ValidatorForm>
+  );
+}

interface/src/project/LightStateWebSocketController.tsx

@@ -0,0 +1,62 @@
+import React, { Component } from 'react';
+import { ValidatorForm } from 'react-material-ui-form-validator';
+
+import { Typography, Box, Switch } from '@material-ui/core';
+import { WEB_SOCKET_ROOT } from '../api';
+import { WebSocketControllerProps, WebSocketFormLoader, WebSocketFormProps, webSocketController } from '../components';
+import { SectionContent, BlockFormControlLabel } from '../components';
+
+import { LightState } from './types';
+
+export const LIGHT_SETTINGS_WEBSOCKET_URL = WEB_SOCKET_ROOT + "lightState";
+
+type LightStateWebSocketControllerProps = WebSocketControllerProps<LightState>;
+
+class LightStateWebSocketController extends Component<LightStateWebSocketControllerProps> {
+
+  render() {
+    return (
+      <SectionContent title='WebSocket Controller' titleGutter>
+        <WebSocketFormLoader
+          {...this.props}
+          render={props => (
+            <LightStateWebSocketControllerForm {...props} />
+          )}
+        />
+      </SectionContent>
+    )
+  }
+
+}
+
+export default webSocketController(LIGHT_SETTINGS_WEBSOCKET_URL, 100, LightStateWebSocketController);
+
+type LightStateWebSocketControllerFormProps = WebSocketFormProps<LightState>;
+
+function LightStateWebSocketControllerForm(props: LightStateWebSocketControllerFormProps) {
+  const { data, saveData, setData } = props;
+
+  const changeLedOn = (event: React.ChangeEvent<HTMLInputElement>) => {
+    setData({ led_on: event.target.checked }, saveData);
+  }
+
+  return (
+    <ValidatorForm onSubmit={saveData}>
+      <Box bgcolor="primary.main" color="primary.contrastText" p={2} mt={2} mb={2}>
+        <Typography variant="body1">
+          The switch below controls the LED via the WebSocket. It will automatically update whenever the LED state changes.
+        </Typography>
+      </Box>
+      <BlockFormControlLabel
+        control={
+          <Switch
+            checked={data.led_on}
+            onChange={changeLedOn}
+            color="primary"
+          />
+        }
+        label="LED State?"
+      />
+    </ValidatorForm>
+  );
+}

interface/src/project/types.ts

@@ -0,0 +1,9 @@
+export interface LightState {
+  led_on: boolean;
+}
+
+export interface LightMqttSettings { 
+  unique_id : string;
+  name: string;
+  mqtt_path : string;
+}

interface/src/security/ManageUsersForm.tsx

@@ -154,11 +154,13 @@ class ManageUsersForm extends React.Component<ManageUsersFormProps, ManageUsersF
           </Table>
           {
             this.noAdminConfigured() &&
-            <Typography component="div" variant="body1">
+            (
               <Box bgcolor="error.main" color="error.contrastText" p={2} mt={2} mb={2}>
-                You must have at least one admin user configured.
+                <Typography variant="body1">
+                  You must have at least one admin user configured.
+                </Typography>
               </Box>
-            </Typography>
+            )
           }
           <FormActions>
             <FormButton startIcon={<SaveIcon />} variant="contained" color="primary" type="submit" disabled={this.noAdminConfigured()}>

interface/src/security/SecuritySettingsForm.tsx

@@ -33,11 +33,11 @@ class SecuritySettingsForm extends React.Component<SecuritySettingsFormProps> {
           onChange={handleValueChange('jwt_secret')}
           margin="normal"
         />
-        <Typography component="div" variant="body1">
+        <Box bgcolor="primary.main" color="primary.contrastText" p={2} mt={2} mb={2}>
-          <Box bgcolor="primary.main" color="primary.contrastText" p={2} mt={2} mb={2}>
+          <Typography variant="body1">
             If you modify the JWT Secret, all users will be logged out.
-          </Box>
+          </Typography>
-        </Typography>
+        </Box>
         <FormActions>
           <FormButton startIcon={<SaveIcon />} variant="contained" color="primary" type="submit">
             Save

lib/framework/APSettingsService.cpp

@@ -1,14 +1,18 @@
 #include <APSettingsService.h>
 
 APSettingsService::APSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
-    AdminSettingsService(server, fs, securityManager, AP_SETTINGS_SERVICE_PATH, AP_SETTINGS_FILE) {
+    _httpEndpoint(APSettings::serialize,
-}
+                  APSettings::deserialize,
-
+                  this,
-APSettingsService::~APSettingsService() {
+                  server,
+                  AP_SETTINGS_SERVICE_PATH,
+                  securityManager),
+    _fsPersistence(APSettings::serialize, APSettings::deserialize, this, fs, AP_SETTINGS_FILE) {
+  addUpdateHandler([&](String originId) { reconfigureAP(); }, false);
 }
 
 void APSettingsService::begin() {
-  SettingsService::begin();
+  _fsPersistence.readFromFS();
   reconfigureAP();
 }
 
@@ -28,8 +32,8 @@ void APSettingsService::loop() {
 
 void APSettingsService::manageAP() {
   WiFiMode_t currentWiFiMode = WiFi.getMode();
-  if (_settings.provisionMode == AP_MODE_ALWAYS ||
+  if (_state.provisionMode == AP_MODE_ALWAYS ||
-      (_settings.provisionMode == AP_MODE_DISCONNECTED && WiFi.status() != WL_CONNECTED)) {
+      (_state.provisionMode == AP_MODE_DISCONNECTED && WiFi.status() != WL_CONNECTED)) {
     if (currentWiFiMode == WIFI_OFF || currentWiFiMode == WIFI_STA) {
       startAP();
     }
@@ -42,7 +46,7 @@ void APSettingsService::manageAP() {
 
 void APSettingsService::startAP() {
   Serial.println("Starting software access point");
-  WiFi.softAP(_settings.ssid.c_str(), _settings.password.c_str());
+  WiFi.softAP(_state.ssid.c_str(), _state.password.c_str());
   if (!_dnsServer) {
     IPAddress apIp = WiFi.softAPIP();
     Serial.print("Starting captive portal on ");
@@ -68,27 +72,3 @@ void APSettingsService::handleDNS() {
     _dnsServer->processNextRequest();
   }
 }
-
-void APSettingsService::readFromJsonObject(JsonObject& root) {
-  _settings.provisionMode = root["provision_mode"] | AP_MODE_ALWAYS;
-  switch (_settings.provisionMode) {
-    case AP_MODE_ALWAYS:
-    case AP_MODE_DISCONNECTED:
-    case AP_MODE_NEVER:
-      break;
-    default:
-      _settings.provisionMode = AP_MODE_ALWAYS;
-  }
-  _settings.ssid = root["ssid"] | AP_DEFAULT_SSID;
-  _settings.password = root["password"] | AP_DEFAULT_PASSWORD;
-}
-
-void APSettingsService::writeToJsonObject(JsonObject& root) {
-  root["provision_mode"] = _settings.provisionMode;
-  root["ssid"] = _settings.ssid;
-  root["password"] = _settings.password;
-}
-
-void APSettingsService::onConfigUpdated() {
-  reconfigureAP();
-}

lib/framework/APSettingsService.h

@@ -1,7 +1,9 @@
 #ifndef APSettingsConfig_h
 #define APSettingsConfig_h
 
-#include <AdminSettingsService.h>
+#include <HttpEndpoint.h>
+#include <FSPersistence.h>
+
 #include <DNSServer.h>
 #include <IPAddress.h>
 
@@ -24,22 +26,39 @@ class APSettings {
   uint8_t provisionMode;
   String ssid;
   String password;
+
+  static void serialize(APSettings& settings, JsonObject& root) {
+    root["provision_mode"] = settings.provisionMode;
+    root["ssid"] = settings.ssid;
+    root["password"] = settings.password;
+  }
+
+  static void deserialize(JsonObject& root, APSettings& settings) {
+    settings.provisionMode = root["provision_mode"] | AP_MODE_ALWAYS;
+    switch (settings.provisionMode) {
+      case AP_MODE_ALWAYS:
+      case AP_MODE_DISCONNECTED:
+      case AP_MODE_NEVER:
+        break;
+      default:
+        settings.provisionMode = AP_MODE_ALWAYS;
+    }
+    settings.ssid = root["ssid"] | AP_DEFAULT_SSID;
+    settings.password = root["password"] | AP_DEFAULT_PASSWORD;
+  }
 };
 
-class APSettingsService : public AdminSettingsService<APSettings> {
+class APSettingsService : public StatefulService<APSettings> {
  public:
   APSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
-  ~APSettingsService();
 
   void begin();
   void loop();
 
- protected:
-  void readFromJsonObject(JsonObject& root);
-  void writeToJsonObject(JsonObject& root);
-  void onConfigUpdated();
-
  private:
+  HttpEndpoint<APSettings> _httpEndpoint;
+  FSPersistence<APSettings> _fsPersistence;
+
   // for the mangement delay loop
   unsigned long _lastManaged;
 

lib/framework/AdminSettingsService.h

@@ -1,50 +0,0 @@
-#ifndef AdminSettingsService_h
-#define AdminSettingsService_h
-
-#include <SettingsService.h>
-
-template <class T>
-class AdminSettingsService : public SettingsService<T> {
- public:
-  AdminSettingsService(AsyncWebServer* server,
-                       FS* fs,
-                       SecurityManager* securityManager,
-                       char const* servicePath,
-                       char const* filePath) :
-      SettingsService<T>(server, fs, servicePath, filePath),
-      _securityManager(securityManager) {
-  }
-
- protected:
-  // will validate the requests with the security manager
-  SecurityManager* _securityManager;
-
-  void fetchConfig(AsyncWebServerRequest* request) {
-    // verify the request against the predicate
-    Authentication authentication = _securityManager->authenticateRequest(request);
-    if (!getAuthenticationPredicate()(authentication)) {
-      request->send(401);
-      return;
-    }
-    // delegate to underlying implemetation
-    SettingsService<T>::fetchConfig(request);
-  }
-
-  void updateConfig(AsyncWebServerRequest* request, JsonDocument& jsonDocument) {
-    // verify the request against the predicate
-    Authentication authentication = _securityManager->authenticateRequest(request);
-    if (!getAuthenticationPredicate()(authentication)) {
-      request->send(401);
-      return;
-    }
-    // delegate to underlying implemetation
-    SettingsService<T>::updateConfig(request, jsonDocument);
-  }
-
-  // override this to replace the default authentication predicate, IS_ADMIN
-  AuthenticationPredicate getAuthenticationPredicate() {
-    return AuthenticationPredicates::IS_ADMIN;
-  }
-};
-
-#endif  // end AdminSettingsService

lib/framework/AsyncJsonCallbackResponse.h

@@ -1,33 +0,0 @@
-#ifndef _AsyncJsonCallbackResponse_H_
-#define _AsyncJsonCallbackResponse_H_
-
-#include <AsyncJson.h>
-#include <ESPAsyncWebServer.h>
-
-/*
- * Listens for a response being destroyed and calls a callback during said distruction.
- * used so we can take action after the response has been rendered to the client.
- *
- * Avoids having to fork ESPAsyncWebServer with a callback feature, but not nice!
- */
-
-typedef std::function<void()> AsyncJsonCallback;
-
-class AsyncJsonCallbackResponse : public AsyncJsonResponse {
- private:
-  AsyncJsonCallback _callback;
-
- public:
-  AsyncJsonCallbackResponse(AsyncJsonCallback callback,
-                            bool isArray = false,
-                            size_t maxJsonBufferSize = DYNAMIC_JSON_DOCUMENT_SIZE) :
-      AsyncJsonResponse(isArray, maxJsonBufferSize),
-      _callback{callback} {
-  }
-
-  ~AsyncJsonCallbackResponse() {
-    _callback();
-  }
-};
-
-#endif  // end _AsyncJsonCallbackResponse_H_

lib/framework/AsyncJsonWebHandler.h

@@ -1,131 +0,0 @@
-#ifndef Async_Json_Request_Web_Handler_H_
-#define Async_Json_Request_Web_Handler_H_
-
-#include <ArduinoJson.h>
-#include <ESPAsyncWebServer.h>
-
-#define ASYNC_JSON_REQUEST_DEFAULT_MAX_SIZE 1024
-#define ASYNC_JSON_REQUEST_MIMETYPE "application/json"
-
-/*
- * Handy little utility for dealing with small JSON request body payloads.
- *
- * Need to be careful using this as we are somewhat limited by RAM.
- *
- * Really only of use where there is a determinate payload size.
- */
-
-typedef std::function<void(AsyncWebServerRequest* request, JsonDocument& jsonDocument)> JsonRequestCallback;
-
-class AsyncJsonWebHandler : public AsyncWebHandler {
- private:
-  WebRequestMethodComposite _method;
-  JsonRequestCallback _onRequest;
-  size_t _maxContentLength;
-
- protected:
-  String _uri;
-
- public:
-  AsyncJsonWebHandler() :
-      _method(HTTP_POST | HTTP_PUT | HTTP_PATCH),
-      _onRequest(nullptr),
-      _maxContentLength(ASYNC_JSON_REQUEST_DEFAULT_MAX_SIZE),
-      _uri() {
-  }
-
-  ~AsyncJsonWebHandler() {
-  }
-
-  void setUri(const String& uri) {
-    _uri = uri;
-  }
-  void setMethod(WebRequestMethodComposite method) {
-    _method = method;
-  }
-  void setMaxContentLength(size_t maxContentLength) {
-    _maxContentLength = maxContentLength;
-  }
-  void onRequest(JsonRequestCallback fn) {
-    _onRequest = fn;
-  }
-
-  virtual bool canHandle(AsyncWebServerRequest* request) override final {
-    if (!_onRequest)
-      return false;
-
-    if (!(_method & request->method()))
-      return false;
-
-    if (_uri.length() && (_uri != request->url() && !request->url().startsWith(_uri + "/")))
-      return false;
-
-    if (!request->contentType().equalsIgnoreCase(ASYNC_JSON_REQUEST_MIMETYPE))
-      return false;
-
-    request->addInterestingHeader("ANY");
-    return true;
-  }
-
-  virtual void handleRequest(AsyncWebServerRequest* request) override final {
-    // no request configured
-    if (!_onRequest) {
-      Serial.print("No request callback was configured for endpoint: ");
-      Serial.println(_uri);
-      request->send(500);
-      return;
-    }
-
-    // we have been handed too much data, return a 413 (payload too large)
-    if (request->contentLength() > _maxContentLength) {
-      request->send(413);
-      return;
-    }
-
-    // parse JSON and if possible handle the request
-    if (request->_tempObject) {
-      DynamicJsonDocument jsonDocument(_maxContentLength);
-      DeserializationError error = deserializeJson(jsonDocument, (uint8_t*)request->_tempObject);
-      if (error == DeserializationError::Ok) {
-        _onRequest(request, jsonDocument);
-      } else {
-        request->send(400);
-      }
-      return;
-    }
-
-    // fallthrough, we have a null pointer, return 500.
-    // this can be due to running out of memory or never receiving body data.
-    request->send(500);
-  }
-
-  virtual void handleBody(AsyncWebServerRequest* request,
-                          uint8_t* data,
-                          size_t len,
-                          size_t index,
-                          size_t total) override final {
-    if (_onRequest) {
-      // don't allocate if data is too large
-      if (total > _maxContentLength) {
-        return;
-      }
-
-      // try to allocate memory on first call
-      // NB: the memory allocated here is freed by ~AsyncWebServerRequest
-      if (index == 0 && !request->_tempObject) {
-        request->_tempObject = malloc(total);
-      }
-
-      // copy the data into the buffer, if we have a buffer!
-      if (request->_tempObject) {
-        memcpy((uint8_t*)request->_tempObject + index, data, len);
-      }
-    }
-  }
-
-  virtual bool isRequestHandlerTrivial() override final {
-    return _onRequest ? false : true;
-  }
-};
-
-#endif  // end Async_Json_Request_Web_Handler_H_

lib/framework/AuthenticationService.cpp

@@ -1,21 +1,17 @@
 #include <AuthenticationService.h>
 
 AuthenticationService::AuthenticationService(AsyncWebServer* server, SecurityManager* securityManager) :
-    _securityManager(securityManager) {
+    _securityManager(securityManager),
+    _signInHandler(SIGN_IN_PATH,
+                   std::bind(&AuthenticationService::signIn, this, std::placeholders::_1, std::placeholders::_2)) {
   server->on(VERIFY_AUTHORIZATION_PATH,
              HTTP_GET,
              std::bind(&AuthenticationService::verifyAuthorization, this, std::placeholders::_1));
-  _signInHandler.setUri(SIGN_IN_PATH);
   _signInHandler.setMethod(HTTP_POST);
   _signInHandler.setMaxContentLength(MAX_AUTHENTICATION_SIZE);
-  _signInHandler.onRequest(
-      std::bind(&AuthenticationService::signIn, this, std::placeholders::_1, std::placeholders::_2));
   server->addHandler(&_signInHandler);
 }
 
-AuthenticationService::~AuthenticationService() {
-}
-
 /**
  * Verifys that the request supplied a valid JWT.
  */
@@ -28,10 +24,10 @@ void AuthenticationService::verifyAuthorization(AsyncWebServerRequest* request)
  * Signs in a user if the username and password match. Provides a JWT to be used in the Authorization header in
  * subsequent requests.
  */
-void AuthenticationService::signIn(AsyncWebServerRequest* request, JsonDocument& jsonDocument) {
+void AuthenticationService::signIn(AsyncWebServerRequest* request, JsonVariant& json) {
-  if (jsonDocument.is<JsonObject>()) {
+  if (json.is<JsonObject>()) {
-    String username = jsonDocument["username"];
+    String username = json["username"];
-    String password = jsonDocument["password"];
+    String password = json["password"];
     Authentication authentication = _securityManager->authenticate(username, password);
     if (authentication.authenticated) {
       User* user = authentication.user;

lib/framework/AuthenticationService.h

@@ -2,7 +2,6 @@
 #define AuthenticationService_H_
 
 #include <AsyncJson.h>
-#include <AsyncJsonWebHandler.h>
 #include <ESPAsyncWebServer.h>
 #include <SecurityManager.h>
 
@@ -14,14 +13,13 @@
 class AuthenticationService {
  public:
   AuthenticationService(AsyncWebServer* server, SecurityManager* securityManager);
-  ~AuthenticationService();
 
  private:
   SecurityManager* _securityManager;
-  AsyncJsonWebHandler _signInHandler;
+  AsyncCallbackJsonWebHandler _signInHandler;
 
   // endpoint functions
-  void signIn(AsyncWebServerRequest* request, JsonDocument& jsonDocument);
+  void signIn(AsyncWebServerRequest* request, JsonVariant& json);
   void verifyAuthorization(AsyncWebServerRequest* request);
 };
 

lib/framework/ESP8266React.cpp

@@ -6,12 +6,14 @@ ESP8266React::ESP8266React(AsyncWebServer* server, FS* fs) :
     _apSettingsService(server, fs, &_securitySettingsService),
     _ntpSettingsService(server, fs, &_securitySettingsService),
     _otaSettingsService(server, fs, &_securitySettingsService),
+    _mqttSettingsService(server, fs, &_securitySettingsService),
     _restartService(server, &_securitySettingsService),
     _authenticationService(server, &_securitySettingsService),
     _wifiScanner(server, &_securitySettingsService),
     _wifiStatus(server, &_securitySettingsService),
     _ntpStatus(server, &_securitySettingsService),
     _apStatus(server, &_securitySettingsService),
+    _mqttStatus(server, &_mqttSettingsService, &_securitySettingsService),
     _systemStatus(server, &_securitySettingsService) {
 #ifdef PROGMEM_WWW
   // Serve static resources from PROGMEM
@@ -71,11 +73,12 @@ void ESP8266React::begin() {
   _apSettingsService.begin();
   _ntpSettingsService.begin();
   _otaSettingsService.begin();
+  _mqttSettingsService.begin();
 }
 
 void ESP8266React::loop() {
   _wifiSettingsService.loop();
   _apSettingsService.loop();
-  _ntpSettingsService.loop();
   _otaSettingsService.loop();
+  _mqttSettingsService.loop();
 }

lib/framework/ESP8266React.h

@@ -16,6 +16,8 @@
 #include <APSettingsService.h>
 #include <APStatus.h>
 #include <AuthenticationService.h>
+#include <MqttSettingsService.h>
+#include <MqttStatus.h>
 #include <NTPSettingsService.h>
 #include <NTPStatus.h>
 #include <OTASettingsService.h>
@@ -41,32 +43,41 @@ class ESP8266React {
     return &_securitySettingsService;
   }
 
-   SettingsService<SecuritySettings>* getSecuritySettingsService() {
+  StatefulService<SecuritySettings>* getSecuritySettingsService() {
     return &_securitySettingsService;
   }
 
-  SettingsService<WiFiSettings>* getWiFiSettingsService() {
+  StatefulService<WiFiSettings>* getWiFiSettingsService() {
     return &_wifiSettingsService;
   }
 
-  SettingsService<APSettings>* getAPSettingsService() {
+  StatefulService<APSettings>* getAPSettingsService() {
     return &_apSettingsService;
   }
 
-  SettingsService<NTPSettings>* getNTPSettingsService() {
+  StatefulService<NTPSettings>* getNTPSettingsService() {
     return &_ntpSettingsService;
   }
 
-  SettingsService<OTASettings>* getOTASettingsService() {
+  StatefulService<OTASettings>* getOTASettingsService() {
     return &_otaSettingsService;
   }
 
+  StatefulService<MqttSettings>* getMqttSettingsService() {
+    return &_mqttSettingsService;
+  }
+
+  AsyncMqttClient* getMqttClient() {
+    return _mqttSettingsService.getMqttClient();
+  }
+
  private:
   SecuritySettingsService _securitySettingsService;
   WiFiSettingsService _wifiSettingsService;
   APSettingsService _apSettingsService;
   NTPSettingsService _ntpSettingsService;
   OTASettingsService _otaSettingsService;
+  MqttSettingsService _mqttSettingsService;
   RestartService _restartService;
 
   AuthenticationService _authenticationService;
@@ -75,6 +86,7 @@ class ESP8266React {
   WiFiStatus _wifiStatus;
   NTPStatus _ntpStatus;
   APStatus _apStatus;
+  MqttStatus _mqttStatus;
   SystemStatus _systemStatus;
 };
 

lib/framework/FSPersistence.h

@@ -0,0 +1,103 @@
+#ifndef FSPersistence_h
+#define FSPersistence_h
+
+#include <StatefulService.h>
+#include <JsonSerializer.h>
+#include <JsonDeserializer.h>
+#include <FS.h>
+
+#define MAX_FILE_SIZE 1024
+
+template <class T>
+class FSPersistence {
+ public:
+  FSPersistence(JsonSerializer<T> jsonSerializer,
+                JsonDeserializer<T> jsonDeserializer,
+                StatefulService<T>* statefulService,
+                FS* fs,
+                char const* filePath) :
+      _jsonSerializer(jsonSerializer),
+      _jsonDeserializer(jsonDeserializer),
+      _statefulService(statefulService),
+      _fs(fs),
+      _filePath(filePath) {
+    enableUpdateHandler();
+  }
+
+  void readFromFS() {
+    File settingsFile = _fs->open(_filePath, "r");
+
+    if (settingsFile) {
+      if (settingsFile.size() <= MAX_FILE_SIZE) {
+        DynamicJsonDocument jsonDocument = DynamicJsonDocument(MAX_FILE_SIZE);
+        DeserializationError error = deserializeJson(jsonDocument, settingsFile);
+        if (error == DeserializationError::Ok && jsonDocument.is<JsonObject>()) {
+          updateSettings(jsonDocument.as<JsonObject>());
+          settingsFile.close();
+          return;
+        }
+      }
+      settingsFile.close();
+    }
+
+    // If we reach here we have not been successful in loading the config,
+    // hard-coded emergency defaults are now applied.
+    applyDefaults();
+  }
+
+  bool writeToFS() {
+    // create and populate a new json object
+    DynamicJsonDocument jsonDocument = DynamicJsonDocument(MAX_FILE_SIZE);
+    JsonObject jsonObject = jsonDocument.to<JsonObject>();
+    _statefulService->read(jsonObject, _jsonSerializer);
+
+    // serialize it to filesystem
+    File settingsFile = _fs->open(_filePath, "w");
+
+    // failed to open file, return false
+    if (!settingsFile) {
+      return false;
+    }
+
+    // serialize the data to the file
+    serializeJson(jsonDocument, settingsFile);
+    settingsFile.close();
+    return true;
+  }
+
+  void disableUpdateHandler() {
+    if (_updateHandlerId) {
+      _statefulService->removeUpdateHandler(_updateHandlerId);
+      _updateHandlerId = 0;
+    }
+  }
+
+  void enableUpdateHandler() {
+    if (!_updateHandlerId) {
+      _updateHandlerId = _statefulService->addUpdateHandler([&](String originId) { writeToFS(); });
+    }
+  }
+
+ private:
+  JsonSerializer<T> _jsonSerializer;
+  JsonDeserializer<T> _jsonDeserializer;
+  StatefulService<T>* _statefulService;
+  FS* _fs;
+  char const* _filePath;
+  update_handler_id_t _updateHandlerId = 0;
+
+  // update the settings, but do not call propogate
+  void updateSettings(JsonObject root) {
+    _statefulService->updateWithoutPropagation(root, _jsonDeserializer);
+  }
+
+ protected:
+  // We assume the deserializer supplies sensible defaults if an empty object
+  // is supplied, this virtual function allows that to be changed.
+  virtual void applyDefaults() {
+    DynamicJsonDocument jsonDocument = DynamicJsonDocument(MAX_FILE_SIZE);
+    updateSettings(jsonDocument.to<JsonObject>());
+  }
+};
+
+#endif  // end FSPersistence

lib/framework/HttpEndpoint.h

@@ -0,0 +1,167 @@
+#ifndef HttpEndpoint_h
+#define HttpEndpoint_h
+
+#include <functional>
+
+#include <AsyncJson.h>
+#include <ESPAsyncWebServer.h>
+
+#include <SecurityManager.h>
+#include <StatefulService.h>
+#include <JsonSerializer.h>
+#include <JsonDeserializer.h>
+
+#define MAX_CONTENT_LENGTH 1024
+#define HTTP_ENDPOINT_ORIGIN_ID "http"
+
+template <class T>
+class HttpGetEndpoint {
+ public:
+  HttpGetEndpoint(JsonSerializer<T> jsonSerializer,
+                  StatefulService<T>* statefulService,
+                  AsyncWebServer* server,
+                  const String& servicePath,
+                  SecurityManager* securityManager,
+                  AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      _jsonSerializer(jsonSerializer), _statefulService(statefulService) {
+    server->on(servicePath.c_str(),
+               HTTP_GET,
+               securityManager->wrapRequest(std::bind(&HttpGetEndpoint::fetchSettings, this, std::placeholders::_1),
+                                            authenticationPredicate));
+  }
+
+  HttpGetEndpoint(JsonSerializer<T> jsonSerializer,
+                  StatefulService<T>* statefulService,
+                  AsyncWebServer* server,
+                  const String& servicePath) :
+      _jsonSerializer(jsonSerializer), _statefulService(statefulService) {
+    server->on(servicePath.c_str(), HTTP_GET, std::bind(&HttpGetEndpoint::fetchSettings, this, std::placeholders::_1));
+  }
+
+ protected:
+  JsonSerializer<T> _jsonSerializer;
+  StatefulService<T>* _statefulService;
+
+  void fetchSettings(AsyncWebServerRequest* request) {
+    AsyncJsonResponse* response = new AsyncJsonResponse(false, MAX_CONTENT_LENGTH);
+    JsonObject jsonObject = response->getRoot().to<JsonObject>();
+    _statefulService->read(jsonObject, _jsonSerializer);
+
+    response->setLength();
+    request->send(response);
+  }
+};
+
+template <class T>
+class HttpPostEndpoint {
+ public:
+  HttpPostEndpoint(JsonSerializer<T> jsonSerializer,
+                   JsonDeserializer<T> jsonDeserializer,
+                   StatefulService<T>* statefulService,
+                   AsyncWebServer* server,
+                   const String& servicePath,
+                   SecurityManager* securityManager,
+                   AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      _jsonSerializer(jsonSerializer),
+      _jsonDeserializer(jsonDeserializer),
+      _statefulService(statefulService),
+      _updateHandler(
+          servicePath,
+          securityManager->wrapCallback(
+              std::bind(&HttpPostEndpoint::updateSettings, this, std::placeholders::_1, std::placeholders::_2),
+              authenticationPredicate)) {
+    _updateHandler.setMethod(HTTP_POST);
+    _updateHandler.setMaxContentLength(MAX_CONTENT_LENGTH);
+    server->addHandler(&_updateHandler);
+  }
+
+  HttpPostEndpoint(JsonSerializer<T> jsonSerializer,
+                   JsonDeserializer<T> jsonDeserializer,
+                   StatefulService<T>* statefulService,
+                   AsyncWebServer* server,
+                   const String& servicePath) :
+      _jsonSerializer(jsonSerializer),
+      _jsonDeserializer(jsonDeserializer),
+      _statefulService(statefulService),
+      _updateHandler(servicePath,
+                     std::bind(&HttpPostEndpoint::updateSettings, this, std::placeholders::_1, std::placeholders::_2)) {
+    _updateHandler.setMethod(HTTP_POST);
+    _updateHandler.setMaxContentLength(MAX_CONTENT_LENGTH);
+    server->addHandler(&_updateHandler);
+  }
+
+ protected:
+  JsonSerializer<T> _jsonSerializer;
+  JsonDeserializer<T> _jsonDeserializer;
+  StatefulService<T>* _statefulService;
+  AsyncCallbackJsonWebHandler _updateHandler;
+
+  void fetchSettings(AsyncWebServerRequest* request) {
+    AsyncJsonResponse* response = new AsyncJsonResponse(false, MAX_CONTENT_LENGTH);
+    JsonObject jsonObject = response->getRoot().to<JsonObject>();
+    _statefulService->read(jsonObject, _jsonSerializer);
+
+    response->setLength();
+    request->send(response);
+  }
+
+  void updateSettings(AsyncWebServerRequest* request, JsonVariant& json) {
+    if (json.is<JsonObject>()) {
+      AsyncJsonResponse* response = new AsyncJsonResponse(false, MAX_CONTENT_LENGTH);
+
+      // use callback to update the settings once the response is complete
+      request->onDisconnect([this]() { _statefulService->callUpdateHandlers(HTTP_ENDPOINT_ORIGIN_ID); });
+
+      // update the settings, deferring the call to the update handlers to when the response is complete
+      _statefulService->updateWithoutPropagation([&](T& settings) {
+        JsonObject jsonObject = json.as<JsonObject>();
+        _jsonDeserializer(jsonObject, settings);
+        jsonObject = response->getRoot().to<JsonObject>();
+        _jsonSerializer(settings, jsonObject);
+      });
+
+      // write the response to the client
+      response->setLength();
+      request->send(response);
+    } else {
+      request->send(400);
+    }
+  }
+};
+
+template <class T>
+class HttpEndpoint : public HttpGetEndpoint<T>, public HttpPostEndpoint<T> {
+ public:
+  HttpEndpoint(JsonSerializer<T> jsonSerializer,
+               JsonDeserializer<T> jsonDeserializer,
+               StatefulService<T>* statefulService,
+               AsyncWebServer* server,
+               const String& servicePath,
+               SecurityManager* securityManager,
+               AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      HttpGetEndpoint<T>(jsonSerializer,
+                         statefulService,
+                         server,
+                         servicePath,
+                         securityManager,
+                         authenticationPredicate),
+      HttpPostEndpoint<T>(jsonSerializer,
+                          jsonDeserializer,
+                          statefulService,
+                          server,
+                          servicePath,
+                          securityManager,
+                          authenticationPredicate) {
+  }
+
+  HttpEndpoint(JsonSerializer<T> jsonSerializer,
+               JsonDeserializer<T> jsonDeserializer,
+               StatefulService<T>* statefulService,
+               AsyncWebServer* server,
+               const String& servicePath) :
+      HttpGetEndpoint<T>(jsonSerializer, statefulService, server, servicePath),
+      HttpPostEndpoint<T>(jsonSerializer, jsonDeserializer, statefulService, server, servicePath) {
+  }
+};
+
+#endif  // end HttpEndpoint

lib/framework/JsonDeserializer.h

@@ -0,0 +1,9 @@
+#ifndef JsonDeserializer_h
+#define JsonDeserializer_h
+
+#include <ArduinoJson.h>
+
+template <class T>
+using JsonDeserializer = void (*)(JsonObject& root, T& settings);
+
+#endif  // end JsonDeserializer

lib/framework/JsonSerializer.h

@@ -0,0 +1,9 @@
+#ifndef JsonSerializer_h
+#define JsonSerializer_h
+
+#include <ArduinoJson.h>
+
+template <class T>
+using JsonSerializer = void (*)(T& settings, JsonObject& root);
+
+#endif  // end JsonSerializer

lib/framework/JsonUtils.h

@@ -0,0 +1,17 @@
+#include <Arduino.h>
+#include <IPAddress.h>
+#include <ArduinoJson.h>
+
+class JsonUtils {
+ public:
+  static void readIP(JsonObject& root, String key, IPAddress& _ip) {
+    if (!root[key].is<String>() || !_ip.fromString(root[key].as<String>())) {
+      _ip = INADDR_NONE;
+    }
+  }
+  static void writeIP(JsonObject& root, String key, IPAddress& _ip) {
+    if (_ip != INADDR_NONE) {
+      root[key] = _ip.toString();
+    }
+  }
+};

lib/framework/MqttPubSub.h

@@ -0,0 +1,161 @@
+#ifndef MqttPubSub_h
+#define MqttPubSub_h
+
+#include <StatefulService.h>
+#include <JsonSerializer.h>
+#include <JsonDeserializer.h>
+#include <AsyncMqttClient.h>
+
+#define MAX_MESSAGE_SIZE 1024
+#define MQTT_ORIGIN_ID "mqtt"
+
+template <class T>
+class MqttConnector {
+ protected:
+  StatefulService<T>* _statefulService;
+  AsyncMqttClient* _mqttClient;
+
+  MqttConnector(StatefulService<T>* statefulService, AsyncMqttClient* mqttClient) :
+      _statefulService(statefulService), _mqttClient(mqttClient) {
+    _mqttClient->onConnect(std::bind(&MqttConnector::onConnect, this));
+  }
+
+  virtual void onConnect() = 0;
+};
+
+template <class T>
+class MqttPub : virtual public MqttConnector<T> {
+ public:
+  MqttPub(JsonSerializer<T> jsonSerializer,
+          StatefulService<T>* statefulService,
+          AsyncMqttClient* mqttClient,
+          String pubTopic = "") :
+      MqttConnector<T>(statefulService, mqttClient), _jsonSerializer(jsonSerializer), _pubTopic(pubTopic) {
+    MqttConnector<T>::_statefulService->addUpdateHandler([&](String originId) { publish(); }, false);
+  }
+
+  void setPubTopic(String pubTopic) {
+    _pubTopic = pubTopic;
+    publish();
+  }
+
+ protected:
+  virtual void onConnect() {
+    publish();
+  }
+
+ private:
+  JsonSerializer<T> _jsonSerializer;
+  String _pubTopic;
+
+  void publish() {
+    if (_pubTopic.length() > 0 && MqttConnector<T>::_mqttClient->connected()) {
+      // serialize to json doc
+      DynamicJsonDocument json(MAX_MESSAGE_SIZE);
+      JsonObject jsonObject = json.to<JsonObject>();
+      MqttConnector<T>::_statefulService->read(jsonObject, _jsonSerializer);
+
+      // serialize to string
+      String payload;
+      serializeJson(json, payload);
+
+      // publish the payload
+      MqttConnector<T>::_mqttClient->publish(_pubTopic.c_str(), 0, false, payload.c_str());
+    }
+  }
+};
+
+template <class T>
+class MqttSub : virtual public MqttConnector<T> {
+ public:
+  MqttSub(JsonDeserializer<T> jsonDeserializer,
+          StatefulService<T>* statefulService,
+          AsyncMqttClient* mqttClient,
+          String subTopic = "") :
+      MqttConnector<T>(statefulService, mqttClient), _jsonDeserializer(jsonDeserializer), _subTopic(subTopic) {
+    MqttConnector<T>::_mqttClient->onMessage(std::bind(&MqttSub::onMqttMessage,
+                                                       this,
+                                                       std::placeholders::_1,
+                                                       std::placeholders::_2,
+                                                       std::placeholders::_3,
+                                                       std::placeholders::_4,
+                                                       std::placeholders::_5,
+                                                       std::placeholders::_6));
+  }
+
+  void setSubTopic(String subTopic) {
+    if (!_subTopic.equals(subTopic)) {
+      // unsubscribe from the existing topic if one was set
+      if (_subTopic.length() > 0) {
+        MqttConnector<T>::_mqttClient->unsubscribe(_subTopic.c_str());
+      }
+      // set the new topic and re-configure the subscription
+      _subTopic = subTopic;
+      subscribe();
+    }
+  }
+
+ protected:
+  virtual void onConnect() {
+    subscribe();
+  }
+
+ private:
+  JsonDeserializer<T> _jsonDeserializer;
+  String _subTopic;
+
+  void subscribe() {
+    if (_subTopic.length() > 0) {
+      MqttConnector<T>::_mqttClient->subscribe(_subTopic.c_str(), 2);
+    }
+  }
+
+  void onMqttMessage(char* topic,
+                     char* payload,
+                     AsyncMqttClientMessageProperties properties,
+                     size_t len,
+                     size_t index,
+                     size_t total) {
+    // we only care about the topic we are watching in this class
+    if (strcmp(_subTopic.c_str(), topic)) {
+      return;
+    }
+
+    // deserialize from string
+    DynamicJsonDocument json(MAX_MESSAGE_SIZE);
+    DeserializationError error = deserializeJson(json, payload, len);
+    if (!error && json.is<JsonObject>()) {
+      JsonObject jsonObject = json.as<JsonObject>();
+      MqttConnector<T>::_statefulService->update(jsonObject, _jsonDeserializer, MQTT_ORIGIN_ID);
+    }
+  }
+};
+
+template <class T>
+class MqttPubSub : public MqttPub<T>, public MqttSub<T> {
+ public:
+  MqttPubSub(JsonSerializer<T> jsonSerializer,
+             JsonDeserializer<T> jsonDeserializer,
+             StatefulService<T>* statefulService,
+             AsyncMqttClient* mqttClient,
+             String pubTopic = "",
+             String subTopic = "") :
+      MqttConnector<T>(statefulService, mqttClient),
+      MqttPub<T>(jsonSerializer, statefulService, mqttClient, pubTopic = ""),
+      MqttSub<T>(jsonDeserializer, statefulService, mqttClient, subTopic = "") {
+  }
+
+ public:
+  void configureTopics(String pubTopic, String subTopic) {
+    MqttSub<T>::setSubTopic(subTopic);
+    MqttPub<T>::setPubTopic(pubTopic);
+  }
+
+ protected:
+  void onConnect() {
+    MqttSub<T>::onConnect();
+    MqttPub<T>::onConnect();
+  }
+};
+
+#endif  // end MqttPubSub

lib/framework/MqttSettingsService.cpp

@@ -0,0 +1,155 @@
+#include <MqttSettingsService.h>
+
+/**
+ * Retains a copy of the cstr provided in the pointer provided using dynamic allocation.
+ *
+ * Frees the pointer before allocation and leaves it as nullptr if cstr == nullptr.
+ */
+static char* retainCstr(const char* cstr, char** ptr) {
+  // free up previously retained value if exists
+  free(*ptr);
+  *ptr = nullptr;
+
+  // dynamically allocate and copy cstr (if non null)
+  if (cstr != nullptr) {
+    *ptr = (char*)malloc(strlen(cstr) + 1);
+    strcpy(*ptr, cstr);
+  }
+
+  // return reference to pointer for convenience
+  return *ptr;
+}
+
+MqttSettingsService::MqttSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
+    _httpEndpoint(MqttSettings::serialize,
+                  MqttSettings::deserialize,
+                  this,
+                  server,
+                  MQTT_SETTINGS_SERVICE_PATH,
+                  securityManager),
+    _fsPersistence(MqttSettings::serialize, MqttSettings::deserialize, this, fs, MQTT_SETTINGS_FILE) {
+#ifdef ESP32
+  WiFi.onEvent(
+      std::bind(&MqttSettingsService::onStationModeDisconnected, this, std::placeholders::_1, std::placeholders::_2),
+      WiFiEvent_t::SYSTEM_EVENT_STA_DISCONNECTED);
+  WiFi.onEvent(std::bind(&MqttSettingsService::onStationModeGotIP, this, std::placeholders::_1, std::placeholders::_2),
+               WiFiEvent_t::SYSTEM_EVENT_STA_GOT_IP);
+#elif defined(ESP8266)
+  _onStationModeDisconnectedHandler = WiFi.onStationModeDisconnected(
+      std::bind(&MqttSettingsService::onStationModeDisconnected, this, std::placeholders::_1));
+  _onStationModeGotIPHandler =
+      WiFi.onStationModeGotIP(std::bind(&MqttSettingsService::onStationModeGotIP, this, std::placeholders::_1));
+#endif
+  _mqttClient.onConnect(std::bind(&MqttSettingsService::onMqttConnect, this, std::placeholders::_1));
+  _mqttClient.onDisconnect(std::bind(&MqttSettingsService::onMqttDisconnect, this, std::placeholders::_1));
+  addUpdateHandler([&](String originId) { onConfigUpdated(); }, false);
+}
+
+MqttSettingsService::~MqttSettingsService() {
+}
+
+void MqttSettingsService::begin() {
+  _fsPersistence.readFromFS();
+}
+
+void MqttSettingsService::loop() {
+  if (_reconfigureMqtt || (_disconnectedAt && (unsigned long)(millis() - _disconnectedAt) >= MQTT_RECONNECTION_DELAY)) {
+    // reconfigure MQTT client
+    configureMqtt();
+
+    // clear the reconnection flags
+    _reconfigureMqtt = false;
+    _disconnectedAt = 0;
+  }
+}
+
+bool MqttSettingsService::isEnabled() {
+  return _state.enabled;
+}
+
+bool MqttSettingsService::isConnected() {
+  return _mqttClient.connected();
+}
+
+const char* MqttSettingsService::getClientId() {
+  return _mqttClient.getClientId();
+}
+
+AsyncMqttClientDisconnectReason MqttSettingsService::getDisconnectReason() {
+  return _disconnectReason;
+}
+
+AsyncMqttClient* MqttSettingsService::getMqttClient() {
+  return &_mqttClient;
+}
+
+void MqttSettingsService::onMqttConnect(bool sessionPresent) {
+  Serial.print("Connected to MQTT, ");
+  Serial.print(sessionPresent ? "with" : "without");
+  Serial.println(" persistent session");
+}
+
+void MqttSettingsService::onMqttDisconnect(AsyncMqttClientDisconnectReason reason) {
+  Serial.print("Disconnected from MQTT reason: ");
+  Serial.println((uint8_t)reason);
+  _disconnectReason = reason;
+  _disconnectedAt = millis();
+}
+
+void MqttSettingsService::onConfigUpdated() {
+  _reconfigureMqtt = true;
+  _disconnectedAt = 0;
+}
+
+#ifdef ESP32
+void MqttSettingsService::onStationModeGotIP(WiFiEvent_t event, WiFiEventInfo_t info) {
+  if (_state.enabled) {
+    Serial.println("WiFi connection dropped, starting MQTT client.");
+    onConfigUpdated();
+  }
+}
+
+void MqttSettingsService::onStationModeDisconnected(WiFiEvent_t event, WiFiEventInfo_t info) {
+  if (_state.enabled) {
+    Serial.println("WiFi connection dropped, stopping MQTT client.");
+    onConfigUpdated();
+  }
+}
+#elif defined(ESP8266)
+void MqttSettingsService::onStationModeGotIP(const WiFiEventStationModeGotIP& event) {
+  if (_state.enabled) {
+    Serial.println("WiFi connection dropped, starting MQTT client.");
+    onConfigUpdated();
+  }
+}
+
+void MqttSettingsService::onStationModeDisconnected(const WiFiEventStationModeDisconnected& event) {
+  if (_state.enabled) {
+    Serial.println("WiFi connection dropped, stopping MQTT client.");
+    onConfigUpdated();
+  }
+}
+#endif
+
+void MqttSettingsService::configureMqtt() {
+  // disconnect if currently connected
+  _mqttClient.disconnect();
+
+  // only connect if WiFi is connected and MQTT is enabled
+  if (_state.enabled && WiFi.isConnected()) {
+    Serial.println("Connecting to MQTT...");
+    _mqttClient.setServer(retainCstr(_state.host.c_str(), &_retainedHost), _state.port);
+    if (_state.username.length() > 0) {
+      _mqttClient.setCredentials(
+          retainCstr(_state.username.c_str(), &_retainedUsername),
+          retainCstr(_state.password.length() > 0 ? _state.password.c_str() : nullptr, &_retainedPassword));
+    } else {
+      _mqttClient.setCredentials(retainCstr(nullptr, &_retainedUsername), retainCstr(nullptr, &_retainedPassword));
+    }
+    _mqttClient.setClientId(retainCstr(_state.clientId.c_str(), &_retainedClientId));
+    _mqttClient.setKeepAlive(_state.keepAlive);
+    _mqttClient.setCleanSession(_state.cleanSession);
+    _mqttClient.setMaxTopicLength(_state.maxTopicLength);
+    _mqttClient.connect();
+  }
+}

lib/framework/MqttSettingsService.h

@@ -0,0 +1,125 @@
+#ifndef MqttSettingsService_h
+#define MqttSettingsService_h
+
+#include <StatefulService.h>
+#include <HttpEndpoint.h>
+#include <FSPersistence.h>
+#include <AsyncMqttClient.h>
+
+#define MQTT_RECONNECTION_DELAY 5000
+
+#define MQTT_SETTINGS_FILE "/config/mqttSettings.json"
+#define MQTT_SETTINGS_SERVICE_PATH "/rest/mqttSettings"
+
+#define MQTT_SETTINGS_SERVICE_DEFAULT_ENABLED false
+#define MQTT_SETTINGS_SERVICE_DEFAULT_HOST "test.mosquitto.org"
+#define MQTT_SETTINGS_SERVICE_DEFAULT_PORT 1883
+#define MQTT_SETTINGS_SERVICE_DEFAULT_USERNAME ""
+#define MQTT_SETTINGS_SERVICE_DEFAULT_PASSWORD ""
+#define MQTT_SETTINGS_SERVICE_DEFAULT_CLIENT_ID generateClientId()
+#define MQTT_SETTINGS_SERVICE_DEFAULT_KEEP_ALIVE 16
+#define MQTT_SETTINGS_SERVICE_DEFAULT_CLEAN_SESSION true
+#define MQTT_SETTINGS_SERVICE_DEFAULT_MAX_TOPIC_LENGTH 128
+
+static String generateClientId() {
+#ifdef ESP32
+  return "esp32-" + String((unsigned long)ESP.getEfuseMac(), HEX);
+#elif defined(ESP8266)
+  return "esp8266-" + String(ESP.getChipId(), HEX);
+#endif
+}
+
+class MqttSettings {
+ public:
+  // host and port - if enabled
+  bool enabled;
+  String host;
+  uint16_t port;
+
+  // username and password
+  String username;
+  String password;
+
+  // client id settings
+  String clientId;
+
+  // connection settings
+  uint16_t keepAlive;
+  bool cleanSession;
+  uint16_t maxTopicLength;
+
+  static void serialize(MqttSettings& settings, JsonObject& root) {
+    root["enabled"] = settings.enabled;
+    root["host"] = settings.host;
+    root["port"] = settings.port;
+    root["username"] = settings.username;
+    root["password"] = settings.password;
+    root["client_id"] = settings.clientId;
+    root["keep_alive"] = settings.keepAlive;
+    root["clean_session"] = settings.cleanSession;
+    root["max_topic_length"] = settings.maxTopicLength;
+  }
+
+  static void deserialize(JsonObject& root, MqttSettings& settings) {
+    settings.enabled = root["enabled"] | MQTT_SETTINGS_SERVICE_DEFAULT_ENABLED;
+    settings.host = root["host"] | MQTT_SETTINGS_SERVICE_DEFAULT_HOST;
+    settings.port = root["port"] | MQTT_SETTINGS_SERVICE_DEFAULT_PORT;
+    settings.username = root["username"] | MQTT_SETTINGS_SERVICE_DEFAULT_USERNAME;
+    settings.password = root["password"] | MQTT_SETTINGS_SERVICE_DEFAULT_PASSWORD;
+    settings.clientId = root["client_id"] | MQTT_SETTINGS_SERVICE_DEFAULT_CLIENT_ID;
+    settings.keepAlive = root["keep_alive"] | MQTT_SETTINGS_SERVICE_DEFAULT_KEEP_ALIVE;
+    settings.cleanSession = root["clean_session"] | MQTT_SETTINGS_SERVICE_DEFAULT_CLEAN_SESSION;
+    settings.maxTopicLength = root["max_topic_length"] | MQTT_SETTINGS_SERVICE_DEFAULT_MAX_TOPIC_LENGTH;
+  }
+};
+
+class MqttSettingsService : public StatefulService<MqttSettings> {
+ public:
+  MqttSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
+  ~MqttSettingsService();
+
+  void begin();
+  void loop();
+  bool isEnabled();
+  bool isConnected();
+  const char* getClientId();
+  AsyncMqttClientDisconnectReason getDisconnectReason();
+  AsyncMqttClient* getMqttClient();
+
+ protected:
+  void onConfigUpdated();
+
+ private:
+  HttpEndpoint<MqttSettings> _httpEndpoint;
+  FSPersistence<MqttSettings> _fsPersistence;
+
+  // Pointers to hold retained copies of the mqtt client connection strings.
+  // Required as AsyncMqttClient holds refrences to the supplied connection strings.
+  char* _retainedHost = nullptr;
+  char* _retainedClientId = nullptr;
+  char* _retainedUsername = nullptr;
+  char* _retainedPassword = nullptr;
+
+  AsyncMqttClient _mqttClient;
+  bool _reconfigureMqtt;
+  unsigned long _disconnectedAt;
+
+  // connection status
+  AsyncMqttClientDisconnectReason _disconnectReason;
+
+#ifdef ESP32
+  void onStationModeGotIP(WiFiEvent_t event, WiFiEventInfo_t info);
+  void onStationModeDisconnected(WiFiEvent_t event, WiFiEventInfo_t info);
+#elif defined(ESP8266)
+  WiFiEventHandler _onStationModeDisconnectedHandler;
+  WiFiEventHandler _onStationModeGotIPHandler;
+  void onStationModeGotIP(const WiFiEventStationModeGotIP& event);
+  void onStationModeDisconnected(const WiFiEventStationModeDisconnected& event);
+#endif
+
+  void onMqttConnect(bool sessionPresent);
+  void onMqttDisconnect(AsyncMqttClientDisconnectReason reason);
+  void configureMqtt();
+};
+
+#endif  // end MqttSettingsService_h

lib/framework/MqttStatus.cpp

@@ -0,0 +1,24 @@
+#include <MqttStatus.h>
+
+MqttStatus::MqttStatus(AsyncWebServer* server,
+                       MqttSettingsService* mqttSettingsService,
+                       SecurityManager* securityManager) :
+    _mqttSettingsService(mqttSettingsService) {
+  server->on(MQTT_STATUS_SERVICE_PATH,
+             HTTP_GET,
+             securityManager->wrapRequest(std::bind(&MqttStatus::mqttStatus, this, std::placeholders::_1),
+                                          AuthenticationPredicates::IS_AUTHENTICATED));
+}
+
+void MqttStatus::mqttStatus(AsyncWebServerRequest* request) {
+  AsyncJsonResponse* response = new AsyncJsonResponse(false, MAX_MQTT_STATUS_SIZE);
+  JsonObject root = response->getRoot();
+
+  root["enabled"] = _mqttSettingsService->isEnabled();
+  root["connected"] = _mqttSettingsService->isConnected();
+  root["client_id"] = _mqttSettingsService->getClientId();
+  root["disconnect_reason"] = (uint8_t)_mqttSettingsService->getDisconnectReason();
+
+  response->setLength();
+  request->send(response);
+}

lib/framework/MqttStatus.h

@@ -0,0 +1,31 @@
+#ifndef MqttStatus_h
+#define MqttStatus_h
+
+#ifdef ESP32
+#include <WiFi.h>
+#include <AsyncTCP.h>
+#elif defined(ESP8266)
+#include <ESP8266WiFi.h>
+#include <ESPAsyncTCP.h>
+#endif
+
+#include <MqttSettingsService.h>
+#include <ArduinoJson.h>
+#include <AsyncJson.h>
+#include <ESPAsyncWebServer.h>
+#include <SecurityManager.h>
+
+#define MAX_MQTT_STATUS_SIZE 1024
+#define MQTT_STATUS_SERVICE_PATH "/rest/mqttStatus"
+
+class MqttStatus {
+ public:
+  MqttStatus(AsyncWebServer* server, MqttSettingsService* mqttSettingsService, SecurityManager* securityManager);
+
+ private:
+  MqttSettingsService* _mqttSettingsService;
+
+  void mqttStatus(AsyncWebServerRequest* request);
+};
+
+#endif  // end MqttStatus_h

lib/framework/NTPSettingsService.cpp

@@ -1,7 +1,13 @@
 #include <NTPSettingsService.h>
 
 NTPSettingsService::NTPSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
-    AdminSettingsService(server, fs, securityManager, NTP_SETTINGS_SERVICE_PATH, NTP_SETTINGS_FILE) {
+    _httpEndpoint(NTPSettings::serialize,
+                  NTPSettings::deserialize,
+                  this,
+                  server,
+                  NTP_SETTINGS_SERVICE_PATH,
+                  securityManager),
+    _fsPersistence(NTPSettings::serialize, NTPSettings::deserialize, this, fs, NTP_SETTINGS_FILE) {
 #ifdef ESP32
   WiFi.onEvent(
       std::bind(&NTPSettingsService::onStationModeDisconnected, this, std::placeholders::_1, std::placeholders::_2),
@@ -14,68 +20,43 @@ NTPSettingsService::NTPSettingsService(AsyncWebServer* server, FS* fs, SecurityM
   _onStationModeGotIPHandler =
       WiFi.onStationModeGotIP(std::bind(&NTPSettingsService::onStationModeGotIP, this, std::placeholders::_1));
 #endif
+  addUpdateHandler([&](String originId) { configureNTP(); }, false);
 }
 
-NTPSettingsService::~NTPSettingsService() {
+void NTPSettingsService::begin() {
-}
+  _fsPersistence.readFromFS();
-
+  configureNTP();
-void NTPSettingsService::loop() {
-  // detect when we need to re-configure NTP and do it in the main loop
-  if (_reconfigureNTP) {
-    _reconfigureNTP = false;
-    configureNTP();
-  }
-}
-
-void NTPSettingsService::readFromJsonObject(JsonObject& root) {
-  _settings.enabled = root["enabled"] | NTP_SETTINGS_SERVICE_DEFAULT_ENABLED;
-  _settings.server = root["server"] | NTP_SETTINGS_SERVICE_DEFAULT_SERVER;
-  _settings.tzLabel = root["tz_label"] | NTP_SETTINGS_SERVICE_DEFAULT_TIME_ZONE_LABEL;
-  _settings.tzFormat = root["tz_format"] | NTP_SETTINGS_SERVICE_DEFAULT_TIME_ZONE_FORMAT;
-}
-
-void NTPSettingsService::writeToJsonObject(JsonObject& root) {
-  root["enabled"] = _settings.enabled;
-  root["server"] = _settings.server;
-  root["tz_label"] = _settings.tzLabel;
-  root["tz_format"] = _settings.tzFormat;
-}
-
-void NTPSettingsService::onConfigUpdated() {
-  _reconfigureNTP = true;
 }
 
 #ifdef ESP32
 void NTPSettingsService::onStationModeGotIP(WiFiEvent_t event, WiFiEventInfo_t info) {
   Serial.println("Got IP address, starting NTP Synchronization");
-  _reconfigureNTP = true;
+  configureNTP();
 }
 
 void NTPSettingsService::onStationModeDisconnected(WiFiEvent_t event, WiFiEventInfo_t info) {
   Serial.println("WiFi connection dropped, stopping NTP.");
-  _reconfigureNTP = false;
+  configureNTP();
-  sntp_stop();
 }
 #elif defined(ESP8266)
 void NTPSettingsService::onStationModeGotIP(const WiFiEventStationModeGotIP& event) {
   Serial.println("Got IP address, starting NTP Synchronization");
-  _reconfigureNTP = true;
+  configureNTP();
 }
 
 void NTPSettingsService::onStationModeDisconnected(const WiFiEventStationModeDisconnected& event) {
   Serial.println("WiFi connection dropped, stopping NTP.");
-  _reconfigureNTP = false;
+  configureNTP();
-  sntp_stop();
 }
 #endif
 
 void NTPSettingsService::configureNTP() {
-  Serial.println("Configuring NTP...");
+  if (WiFi.isConnected() && _state.enabled) {
-  if (_settings.enabled) {
+    Serial.println("Starting NTP...");
 #ifdef ESP32
-    configTzTime(_settings.tzFormat.c_str(), _settings.server.c_str());
+    configTzTime(_state.tzFormat.c_str(), _state.server.c_str());
 #elif defined(ESP8266)
-    configTime(_settings.tzFormat.c_str(), _settings.server.c_str());
+    configTime(_state.tzFormat.c_str(), _state.server.c_str());
 #endif
   } else {
     sntp_stop();

lib/framework/NTPSettingsService.h

@@ -1,7 +1,8 @@
 #ifndef NTPSettingsService_h
 #define NTPSettingsService_h
 
-#include <AdminSettingsService.h>
+#include <HttpEndpoint.h>
+#include <FSPersistence.h>
 
 #include <time.h>
 #ifdef ESP32
@@ -16,10 +17,6 @@
 #define NTP_SETTINGS_SERVICE_DEFAULT_TIME_ZONE_FORMAT "GMT0BST,M3.5.0/1,M10.5.0"
 #define NTP_SETTINGS_SERVICE_DEFAULT_SERVER "time.google.com"
 
-// min poll delay of 60 secs, max 1 day
-#define NTP_SETTINGS_MIN_INTERVAL 60
-#define NTP_SETTINGS_MAX_INTERVAL 86400
-
 #define NTP_SETTINGS_FILE "/config/ntpSettings.json"
 #define NTP_SETTINGS_SERVICE_PATH "/rest/ntpSettings"
 
@@ -29,22 +26,31 @@ class NTPSettings {
   String tzLabel;
   String tzFormat;
   String server;
+
+  static void serialize(NTPSettings& settings, JsonObject& root) {
+    root["enabled"] = settings.enabled;
+    root["server"] = settings.server;
+    root["tz_label"] = settings.tzLabel;
+    root["tz_format"] = settings.tzFormat;
+  }
+
+  static void deserialize(JsonObject& root, NTPSettings& settings) {
+    settings.enabled = root["enabled"] | NTP_SETTINGS_SERVICE_DEFAULT_ENABLED;
+    settings.server = root["server"] | NTP_SETTINGS_SERVICE_DEFAULT_SERVER;
+    settings.tzLabel = root["tz_label"] | NTP_SETTINGS_SERVICE_DEFAULT_TIME_ZONE_LABEL;
+    settings.tzFormat = root["tz_format"] | NTP_SETTINGS_SERVICE_DEFAULT_TIME_ZONE_FORMAT;
+  }
 };
 
-class NTPSettingsService : public AdminSettingsService<NTPSettings> {
+class NTPSettingsService : public StatefulService<NTPSettings> {
  public:
   NTPSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
-  ~NTPSettingsService();
 
-  void loop();
+  void begin();
-
- protected:
-  void readFromJsonObject(JsonObject& root);
-  void writeToJsonObject(JsonObject& root);
-  void onConfigUpdated();
 
  private:
-  bool _reconfigureNTP = false;
+  HttpEndpoint<NTPSettings> _httpEndpoint;
+  FSPersistence<NTPSettings> _fsPersistence;
 
 #ifdef ESP32
   void onStationModeGotIP(WiFiEvent_t event, WiFiEventInfo_t info);
@@ -56,7 +62,6 @@ class NTPSettingsService : public AdminSettingsService<NTPSettings> {
   void onStationModeGotIP(const WiFiEventStationModeGotIP& event);
   void onStationModeDisconnected(const WiFiEventStationModeDisconnected& event);
 #endif
-
   void configureNTP();
 };
 

lib/framework/OTASettingsService.cpp

@@ -1,7 +1,13 @@
 #include <OTASettingsService.h>
 
 OTASettingsService::OTASettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
-    AdminSettingsService(server, fs, securityManager, OTA_SETTINGS_SERVICE_PATH, OTA_SETTINGS_FILE) {
+    _httpEndpoint(OTASettings::serialize,
+                  OTASettings::deserialize,
+                  this,
+                  server,
+                  OTA_SETTINGS_SERVICE_PATH,
+                  securityManager),
+    _fsPersistence(OTASettings::serialize, OTASettings::deserialize, this, fs, OTA_SETTINGS_FILE) {
 #ifdef ESP32
   WiFi.onEvent(std::bind(&OTASettingsService::onStationModeGotIP, this, std::placeholders::_1, std::placeholders::_2),
                WiFiEvent_t::SYSTEM_EVENT_STA_GOT_IP);
@@ -9,31 +15,18 @@ OTASettingsService::OTASettingsService(AsyncWebServer* server, FS* fs, SecurityM
   _onStationModeGotIPHandler =
       WiFi.onStationModeGotIP(std::bind(&OTASettingsService::onStationModeGotIP, this, std::placeholders::_1));
 #endif
+  addUpdateHandler([&](String originId) { configureArduinoOTA(); }, false);
 }
 
-OTASettingsService::~OTASettingsService() {
+void OTASettingsService::begin() {
-}
+  _fsPersistence.readFromFS();
-
-void OTASettingsService::loop() {
-  if ( _settings.enabled && _arduinoOTA) {
-    _arduinoOTA->handle();
-  }
-}
-
-void OTASettingsService::onConfigUpdated() {
   configureArduinoOTA();
 }
 
-void OTASettingsService::readFromJsonObject(JsonObject& root) {
+void OTASettingsService::loop() {
-  _settings.enabled = root["enabled"] | DEFAULT_OTA_ENABLED;
+  if (_state.enabled && _arduinoOTA) {
-  _settings.port = root["port"] | DEFAULT_OTA_PORT;
+    _arduinoOTA->handle();
-  _settings.password = root["password"] | DEFAULT_OTA_PASSWORD;
+  }
-}
-
-void OTASettingsService::writeToJsonObject(JsonObject& root) {
-  root["enabled"] = _settings.enabled;
-  root["port"] = _settings.port;
-  root["password"] = _settings.password;
 }
 
 void OTASettingsService::configureArduinoOTA() {
@@ -44,11 +37,11 @@ void OTASettingsService::configureArduinoOTA() {
     delete _arduinoOTA;
     _arduinoOTA = nullptr;
   }
-  if (_settings.enabled) {
+  if (_state.enabled) {
-    Serial.println("Starting OTA Update Service");
+    Serial.println("Starting OTA Update Service...");
     _arduinoOTA = new ArduinoOTAClass;
-    _arduinoOTA->setPort(_settings.port);
+    _arduinoOTA->setPort(_state.port);
-    _arduinoOTA->setPassword(_settings.password.c_str());
+    _arduinoOTA->setPassword(_state.password.c_str());
     _arduinoOTA->onStart([]() { Serial.println("Starting"); });
     _arduinoOTA->onEnd([]() { Serial.println("\nEnd"); });
     _arduinoOTA->onProgress([](unsigned int progress, unsigned int total) {
@@ -70,6 +63,7 @@ void OTASettingsService::configureArduinoOTA() {
     _arduinoOTA->begin();
   }
 }
+
 #ifdef ESP32
 void OTASettingsService::onStationModeGotIP(WiFiEvent_t event, WiFiEventInfo_t info) {
   configureArduinoOTA();

lib/framework/OTASettingsService.h

@@ -1,7 +1,8 @@
 #ifndef OTASettingsService_h
 #define OTASettingsService_h
 
-#include <AdminSettingsService.h>
+#include <HttpEndpoint.h>
+#include <FSPersistence.h>
 
 #ifdef ESP32
 #include <ESPmDNS.h>
@@ -25,21 +26,30 @@ class OTASettings {
   bool enabled;
   int port;
   String password;
+
+  static void serialize(OTASettings& settings, JsonObject& root) {
+    root["enabled"] = settings.enabled;
+    root["port"] = settings.port;
+    root["password"] = settings.password;
+  }
+
+  static void deserialize(JsonObject& root, OTASettings& settings) {
+    settings.enabled = root["enabled"] | DEFAULT_OTA_ENABLED;
+    settings.port = root["port"] | DEFAULT_OTA_PORT;
+    settings.password = root["password"] | DEFAULT_OTA_PASSWORD;
+  }
 };
 
-class OTASettingsService : public AdminSettingsService<OTASettings> {
+class OTASettingsService : public StatefulService<OTASettings> {
  public:
   OTASettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
-  ~OTASettingsService();
 
+  void begin();
   void loop();
 
- protected:
-  void onConfigUpdated();
-  void readFromJsonObject(JsonObject& root);
-  void writeToJsonObject(JsonObject& root);
-
  private:
+  HttpEndpoint<OTASettings> _httpEndpoint;
+  FSPersistence<OTASettings> _fsPersistence;
   ArduinoOTAClass* _arduinoOTA;
 
   void configureArduinoOTA();

lib/framework/SecurityManager.h

@@ -3,10 +3,13 @@
 
 #include <ArduinoJsonJWT.h>
 #include <ESPAsyncWebServer.h>
+#include <AsyncJson.h>
 #include <list>
 
 #define DEFAULT_JWT_SECRET "esp8266-react"
 
+#define ACCESS_TOKEN_PARAMATER "access_token"
+
 #define AUTHORIZATION_HEADER "Authorization"
 #define AUTHORIZATION_HEADER_PREFIX "Bearer "
 #define AUTHORIZATION_HEADER_PREFIX_LEN 7
@@ -59,7 +62,7 @@ class SecurityManager {
   /*
    * Authenticate, returning the user if found
    */
-  virtual Authentication authenticate(String username, String password) = 0;
+  virtual Authentication authenticate(String& username, String& password) = 0;
 
   /*
    * Check the request header for the Authorization token
@@ -71,11 +74,22 @@ class SecurityManager {
    */
   virtual String generateJWT(User* user) = 0;
 
+  /**
+   * Filter a request with the provided predicate, only returning true if the predicate matches.
+   */
+  virtual ArRequestFilterFunction filterRequest(AuthenticationPredicate predicate) = 0;
+
   /**
    * Wrap the provided request to provide validation against an AuthenticationPredicate.
    */
   virtual ArRequestHandlerFunction wrapRequest(ArRequestHandlerFunction onRequest,
                                                AuthenticationPredicate predicate) = 0;
+
+  /**
+   * Wrap the provided json request callback to provide validation against an AuthenticationPredicate.
+   */
+  virtual ArJsonRequestHandlerFunction wrapCallback(ArJsonRequestHandlerFunction callback,
+                                                    AuthenticationPredicate predicate) = 0;
 };
 
 #endif  // end SecurityManager_h

lib/framework/SecuritySettingsService.cpp

@@ -1,62 +1,48 @@
 #include <SecuritySettingsService.h>
 
 SecuritySettingsService::SecuritySettingsService(AsyncWebServer* server, FS* fs) :
-    AdminSettingsService(server, fs, this, SECURITY_SETTINGS_PATH, SECURITY_SETTINGS_FILE),
+    _httpEndpoint(SecuritySettings::serialize,
-    SecurityManager() {
+                  SecuritySettings::deserialize,
-}
+                  this,
-SecuritySettingsService::~SecuritySettingsService() {
+                  server,
+                  SECURITY_SETTINGS_PATH,
+                  this),
+    _fsPersistence(SecuritySettings::serialize, SecuritySettings::deserialize, this, fs, SECURITY_SETTINGS_FILE) {
+  addUpdateHandler([&](String originId) { configureJWTHandler(); }, false);
 }
 
-void SecuritySettingsService::readFromJsonObject(JsonObject& root) {
+void SecuritySettingsService::begin() {
-  // secret
+  _fsPersistence.readFromFS();
-  _jwtHandler.setSecret(root["jwt_secret"] | DEFAULT_JWT_SECRET);
+  configureJWTHandler();
-
-  // users
-  _settings.users.clear();
-  if (root["users"].is<JsonArray>()) {
-    for (JsonVariant user : root["users"].as<JsonArray>()) {
-     _settings.users.push_back(User(user["username"], user["password"], user["admin"]));
-    }
-  } else {
-    _settings.users.push_back(User(DEFAULT_ADMIN_USERNAME, DEFAULT_ADMIN_USERNAME, true));
-    _settings.users.push_back(User(DEFAULT_GUEST_USERNAME, DEFAULT_GUEST_USERNAME, false));
-  }
 }
 
-void SecuritySettingsService::writeToJsonObject(JsonObject& root) {
+Authentication SecuritySettingsService::authenticateRequest(AsyncWebServerRequest* request) {
-  // secret
+  AsyncWebHeader* authorizationHeader = request->getHeader(AUTHORIZATION_HEADER);
-  root["jwt_secret"] = _jwtHandler.getSecret();
-
-  // users
-  JsonArray users = root.createNestedArray("users");
-  for (User _user : _settings.users) {
-    JsonObject user = users.createNestedObject();
-    user["username"] = _user.username;
-    user["password"] = _user.password;
-    user["admin"] = _user.admin;
-  }
-}
-
-
-Authentication SecuritySettingsService::authenticateRequest(AsyncWebServerRequest *request) {
-  AsyncWebHeader *authorizationHeader = request->getHeader(AUTHORIZATION_HEADER);
   if (authorizationHeader) {
     String value = authorizationHeader->value();
     if (value.startsWith(AUTHORIZATION_HEADER_PREFIX)) {
       value = value.substring(AUTHORIZATION_HEADER_PREFIX_LEN);
       return authenticateJWT(value);
     }
+  } else if (request->hasParam(ACCESS_TOKEN_PARAMATER)) {
+    AsyncWebParameter* tokenParamater = request->getParam(ACCESS_TOKEN_PARAMATER);
+    String value = tokenParamater->value();
+    return authenticateJWT(value);
   }
   return Authentication();
 }
 
-Authentication SecuritySettingsService::authenticateJWT(String jwt) {
+void SecuritySettingsService::configureJWTHandler() {
+  _jwtHandler.setSecret(_state.jwtSecret);
+}
+
+Authentication SecuritySettingsService::authenticateJWT(String& jwt) {
   DynamicJsonDocument payloadDocument(MAX_JWT_SIZE);
   _jwtHandler.parseJWT(jwt, payloadDocument);
   if (payloadDocument.is<JsonObject>()) {
     JsonObject parsedPayload = payloadDocument.as<JsonObject>();
     String username = parsedPayload["username"];
-    for (User _user : _settings.users) {
+    for (User _user : _state.users) {
       if (_user.username == username && validatePayload(parsedPayload, &_user)) {
         return Authentication(_user);
       }
@@ -65,8 +51,8 @@ Authentication SecuritySettingsService::authenticateJWT(String jwt) {
   return Authentication();
 }
 
-Authentication SecuritySettingsService::authenticate(String username, String password) {
+Authentication SecuritySettingsService::authenticate(String& username, String& password) {
-  for (User _user : _settings.users) {
+  for (User _user : _state.users) {
     if (_user.username == username && _user.password == password) {
       return Authentication(_user);
     }
@@ -74,28 +60,35 @@ Authentication SecuritySettingsService::authenticate(String username, String pas
   return Authentication();
 }
 
-inline void populateJWTPayload(JsonObject &payload, User *user) {
+inline void populateJWTPayload(JsonObject& payload, User* user) {
   payload["username"] = user->username;
   payload["admin"] = user->admin;
 }
 
-boolean SecuritySettingsService::validatePayload(JsonObject &parsedPayload, User *user) {
+boolean SecuritySettingsService::validatePayload(JsonObject& parsedPayload, User* user) {
-  DynamicJsonDocument _jsonDocument(MAX_JWT_SIZE);
+  DynamicJsonDocument jsonDocument(MAX_JWT_SIZE);
-  JsonObject payload = _jsonDocument.to<JsonObject>();
+  JsonObject payload = jsonDocument.to<JsonObject>();
   populateJWTPayload(payload, user);
   return payload == parsedPayload;
 }
 
-String SecuritySettingsService::generateJWT(User *user) {
+String SecuritySettingsService::generateJWT(User* user) {
-  DynamicJsonDocument _jsonDocument(MAX_JWT_SIZE);
+  DynamicJsonDocument jsonDocument(MAX_JWT_SIZE);
-  JsonObject payload = _jsonDocument.to<JsonObject>();
+  JsonObject payload = jsonDocument.to<JsonObject>();
   populateJWTPayload(payload, user);
   return _jwtHandler.buildJWT(payload);
 }
 
+ArRequestFilterFunction SecuritySettingsService::filterRequest(AuthenticationPredicate predicate) {
+  return [this, predicate](AsyncWebServerRequest* request) {
+    Authentication authentication = authenticateRequest(request);
+    return predicate(authentication);
+  };
+}
+
 ArRequestHandlerFunction SecuritySettingsService::wrapRequest(ArRequestHandlerFunction onRequest,
-                                                      AuthenticationPredicate predicate) {
+                                                              AuthenticationPredicate predicate) {
-  return [this, onRequest, predicate](AsyncWebServerRequest *request) {
+  return [this, onRequest, predicate](AsyncWebServerRequest* request) {
     Authentication authentication = authenticateRequest(request);
     if (!predicate(authentication)) {
       request->send(401);
@@ -104,3 +97,15 @@ ArRequestHandlerFunction SecuritySettingsService::wrapRequest(ArRequestHandlerFu
     onRequest(request);
   };
 }
+
+ArJsonRequestHandlerFunction SecuritySettingsService::wrapCallback(ArJsonRequestHandlerFunction callback,
+                                                                   AuthenticationPredicate predicate) {
+  return [this, callback, predicate](AsyncWebServerRequest* request, JsonVariant& json) {
+    Authentication authentication = authenticateRequest(request);
+    if (!predicate(authentication)) {
+      request->send(401);
+      return;
+    }
+    callback(request, json);
+  };
+}

lib/framework/SecuritySettingsService.h

@@ -1,8 +1,9 @@
 #ifndef SecuritySettingsService_h
 #define SecuritySettingsService_h
 
-#include <AdminSettingsService.h>
 #include <SecurityManager.h>
+#include <HttpEndpoint.h>
+#include <FSPersistence.h>
 
 #define DEFAULT_ADMIN_USERNAME "admin"
 #define DEFAULT_GUEST_USERNAME "guest"
@@ -14,30 +15,63 @@ class SecuritySettings {
  public:
   String jwtSecret;
   std::list<User> users;
+
+  static void serialize(SecuritySettings& settings, JsonObject& root) {
+    // secret
+    root["jwt_secret"] = settings.jwtSecret;
+
+    // users
+    JsonArray users = root.createNestedArray("users");
+    for (User user : settings.users) {
+      JsonObject userRoot = users.createNestedObject();
+      userRoot["username"] = user.username;
+      userRoot["password"] = user.password;
+      userRoot["admin"] = user.admin;
+    }
+  }
+
+  static void deserialize(JsonObject& root, SecuritySettings& settings) {
+    // secret
+    settings.jwtSecret = root["jwt_secret"] | DEFAULT_JWT_SECRET;
+
+    // users
+    settings.users.clear();
+    if (root["users"].is<JsonArray>()) {
+      for (JsonVariant user : root["users"].as<JsonArray>()) {
+        settings.users.push_back(User(user["username"], user["password"], user["admin"]));
+      }
+    } else {
+      settings.users.push_back(User(DEFAULT_ADMIN_USERNAME, DEFAULT_ADMIN_USERNAME, true));
+      settings.users.push_back(User(DEFAULT_GUEST_USERNAME, DEFAULT_GUEST_USERNAME, false));
+    }
+  }
 };
 
-class SecuritySettingsService : public AdminSettingsService<SecuritySettings>, public SecurityManager {
+class SecuritySettingsService : public StatefulService<SecuritySettings>, public SecurityManager {
  public:
   SecuritySettingsService(AsyncWebServer* server, FS* fs);
-  ~SecuritySettingsService();
+
+  void begin();
 
   // Functions to implement SecurityManager
-  Authentication authenticate(String username, String password);
+  Authentication authenticate(String& username, String& password);
   Authentication authenticateRequest(AsyncWebServerRequest* request);
   String generateJWT(User* user);
+  ArRequestFilterFunction filterRequest(AuthenticationPredicate predicate);
   ArRequestHandlerFunction wrapRequest(ArRequestHandlerFunction onRequest, AuthenticationPredicate predicate);
-
+  ArJsonRequestHandlerFunction wrapCallback(ArJsonRequestHandlerFunction callback, AuthenticationPredicate predicate);
- protected:
-  void readFromJsonObject(JsonObject& root);
-  void writeToJsonObject(JsonObject& root);
 
  private:
+  HttpEndpoint<SecuritySettings> _httpEndpoint;
+  FSPersistence<SecuritySettings> _fsPersistence;
   ArduinoJsonJWT _jwtHandler = ArduinoJsonJWT(DEFAULT_JWT_SECRET);
 
+  void configureJWTHandler();
+
   /*
    * Lookup the user by JWT
    */
-  Authentication authenticateJWT(String jwt);
+  Authentication authenticateJWT(String& jwt);
 
   /*
    * Verify the payload is correct

lib/framework/SettingsPersistence.h

@@ -1,96 +0,0 @@
-#ifndef SettingsPersistence_h
-#define SettingsPersistence_h
-
-#include <ArduinoJson.h>
-#include <AsyncJson.h>
-#include <AsyncJsonWebHandler.h>
-#include <ESPAsyncWebServer.h>
-#include <FS.h>
-
-/**
- * At the moment, not expecting settings service to have to deal with large JSON
- * files this could be made configurable fairly simply, it's exposed on
- * AsyncJsonWebHandler with a setter.
- */
-#define MAX_SETTINGS_SIZE 1024
-
-/*
- * Mixin for classes which need to save settings to/from a file on the the file system as JSON.
- */
-class SettingsPersistence {
- protected:
-  // will store and retrieve config from the file system
-  FS* _fs;
-
-  // the file path our settings will be saved to
-  char const* _filePath;
-
-  bool writeToFS() {
-    // create and populate a new json object
-    DynamicJsonDocument jsonDocument = DynamicJsonDocument(MAX_SETTINGS_SIZE);
-    JsonObject root = jsonDocument.to<JsonObject>();
-    writeToJsonObject(root);
-
-    // serialize it to filesystem
-    File configFile = _fs->open(_filePath, "w");
-
-    // failed to open file, return false
-    if (!configFile) {
-      return false;
-    }
-
-    serializeJson(jsonDocument, configFile);
-    configFile.close();
-
-    return true;
-  }
-
-  void readFromFS() {
-    File configFile = _fs->open(_filePath, "r");
-
-    // use defaults if no config found
-    if (configFile) {
-      // Protect against bad data uploaded to file system
-      // We never expect the config file to get very large, so cap it.
-      size_t size = configFile.size();
-      if (size <= MAX_SETTINGS_SIZE) {
-        DynamicJsonDocument jsonDocument = DynamicJsonDocument(MAX_SETTINGS_SIZE);
-        DeserializationError error = deserializeJson(jsonDocument, configFile);
-        if (error == DeserializationError::Ok && jsonDocument.is<JsonObject>()) {
-          JsonObject root = jsonDocument.as<JsonObject>();
-          readFromJsonObject(root);
-          configFile.close();
-          return;
-        }
-      }
-      configFile.close();
-    }
-
-    // If we reach here we have not been successful in loading the config,
-    // hard-coded emergency defaults are now applied.
-    applyDefaultConfig();
-  }
-
-  // serialization routene, from local config to JsonObject
-  virtual void readFromJsonObject(JsonObject& root) {
-  }
-  virtual void writeToJsonObject(JsonObject& root) {
-  }
-
-  // We assume the readFromJsonObject supplies sensible defaults if an empty object
-  // is supplied, this virtual function allows that to be changed.
-  virtual void applyDefaultConfig() {
-    DynamicJsonDocument jsonDocument = DynamicJsonDocument(MAX_SETTINGS_SIZE);
-    JsonObject root = jsonDocument.to<JsonObject>();
-    readFromJsonObject(root);
-  }
-
- public:
-  SettingsPersistence(FS* fs, char const* filePath) : _fs(fs), _filePath(filePath) {
-  }
-
-  virtual ~SettingsPersistence() {
-  }
-};
-
-#endif  // end SettingsPersistence

lib/framework/SettingsService.h

@@ -1,166 +0,0 @@
-#ifndef SettingsService_h
-#define SettingsService_h
-
-#include <functional>
-
-#ifdef ESP32
-#include <WiFi.h>
-#include <AsyncTCP.h>
-#elif defined(ESP8266)
-#include <ESP8266WiFi.h>
-#include <ESPAsyncTCP.h>
-#endif
-
-#include <ArduinoJson.h>
-#include <AsyncJson.h>
-#include <AsyncJsonCallbackResponse.h>
-#include <AsyncJsonWebHandler.h>
-#include <ESPAsyncWebServer.h>
-#include <SecurityManager.h>
-#include <SettingsPersistence.h>
-
-typedef size_t update_handler_id_t;
-typedef std::function<void(void)> SettingsUpdateCallback;
-static update_handler_id_t currentUpdateHandlerId;
-
-typedef struct SettingsUpdateHandlerInfo {
-  update_handler_id_t _id;
-  SettingsUpdateCallback _cb;
-  bool _allowRemove;
-  SettingsUpdateHandlerInfo(SettingsUpdateCallback cb, bool allowRemove) :
-      _id(++currentUpdateHandlerId),
-      _cb(cb),
-      _allowRemove(allowRemove){};
-} SettingsUpdateHandlerInfo_t;
-
-/*
- * Abstraction of a service which stores it's settings as JSON in a file system.
- */
-template <class T>
-class SettingsService : public SettingsPersistence {
- public:
-  SettingsService(AsyncWebServer* server, FS* fs, char const* servicePath, char const* filePath) :
-      SettingsPersistence(fs, filePath),
-      _servicePath(servicePath) {
-    server->on(_servicePath, HTTP_GET, std::bind(&SettingsService::fetchConfig, this, std::placeholders::_1));
-    _updateHandler.setUri(servicePath);
-    _updateHandler.setMethod(HTTP_POST);
-    _updateHandler.setMaxContentLength(MAX_SETTINGS_SIZE);
-    _updateHandler.onRequest(
-        std::bind(&SettingsService::updateConfig, this, std::placeholders::_1, std::placeholders::_2));
-    server->addHandler(&_updateHandler);
-  }
-
-  virtual ~SettingsService() {
-  }
-
-  update_handler_id_t addUpdateHandler(SettingsUpdateCallback cb, bool allowRemove = true) {
-    if (!cb) {
-      return 0;
-    }
-    SettingsUpdateHandlerInfo_t updateHandler(cb, allowRemove);
-    _settingsUpdateHandlers.push_back(updateHandler);
-    return updateHandler._id;
-  }
-
-  void removeUpdateHandler(update_handler_id_t id) {
-    for (auto i = _settingsUpdateHandlers.begin(); i != _settingsUpdateHandlers.end();) {
-      if ((*i)._id == id) {
-        i = _settingsUpdateHandlers.erase(i);
-      } else {
-        ++i;
-      }
-    }
-  }
-
-  T fetch() {
-    return _settings;
-  }
-
-  void update(T& settings) {
-    _settings = settings;
-    writeToFS();
-    callUpdateHandlers();
-  }
-
-  void fetchAsString(String& config) {
-    DynamicJsonDocument jsonDocument(MAX_SETTINGS_SIZE);
-    fetchAsDocument(jsonDocument);
-    serializeJson(jsonDocument, config);
-  }
-
-  void updateFromString(String& config) {
-    DynamicJsonDocument jsonDocument(MAX_SETTINGS_SIZE);
-    deserializeJson(jsonDocument, config);
-    updateFromDocument(jsonDocument);
-  }
-
-  void fetchAsDocument(JsonDocument& jsonDocument) {
-    JsonObject jsonObject = jsonDocument.to<JsonObject>();
-    writeToJsonObject(jsonObject);
-  }
-
-  void updateFromDocument(JsonDocument& jsonDocument) {
-    if (jsonDocument.is<JsonObject>()) {
-      JsonObject newConfig = jsonDocument.as<JsonObject>();
-      readFromJsonObject(newConfig);
-      writeToFS();
-      callUpdateHandlers();
-    }
-  }
-
-  void begin() {
-    // read the initial data from the file system
-    readFromFS();
-  }
-
- protected:
-  T _settings;
-  char const* _servicePath;
-  AsyncJsonWebHandler _updateHandler;
-  std::list<SettingsUpdateHandlerInfo_t> _settingsUpdateHandlers;
-
-  virtual void fetchConfig(AsyncWebServerRequest* request) {
-    // handle the request
-    AsyncJsonResponse* response = new AsyncJsonResponse(false, MAX_SETTINGS_SIZE);
-    JsonObject jsonObject = response->getRoot();
-    writeToJsonObject(jsonObject);
-    response->setLength();
-    request->send(response);
-  }
-
-  virtual void updateConfig(AsyncWebServerRequest* request, JsonDocument& jsonDocument) {
-    // handle the request
-    if (jsonDocument.is<JsonObject>()) {
-      JsonObject newConfig = jsonDocument.as<JsonObject>();
-      readFromJsonObject(newConfig);
-      writeToFS();
-
-      // write settings back with a callback to reconfigure the wifi
-      AsyncJsonCallbackResponse* response =
-          new AsyncJsonCallbackResponse([this]() { callUpdateHandlers(); }, false, MAX_SETTINGS_SIZE);
-      JsonObject jsonObject = response->getRoot();
-      writeToJsonObject(jsonObject);
-      response->setLength();
-      request->send(response);
-    } else {
-      request->send(400);
-    }
-  }
-
-  void callUpdateHandlers() {
-    // call the classes own config update function
-    onConfigUpdated();
-
-    // call all setting update handlers
-    for (const SettingsUpdateHandlerInfo_t& handler : _settingsUpdateHandlers) {
-      handler._cb();
-    }
-  }
-
-  // implement to perform action when config has been updated
-  virtual void onConfigUpdated() {
-  }
-};
-
-#endif  // end SettingsService

lib/framework/SimpleService.h

@@ -1,87 +0,0 @@
-#ifndef Service_h
-#define Service_h
-
-#ifdef ESP32
-#include <WiFi.h>
-#include <AsyncTCP.h>
-#elif defined(ESP8266)
-#include <ESP8266WiFi.h>
-#include <ESPAsyncTCP.h>
-#endif
-
-#include <ArduinoJson.h>
-#include <AsyncJson.h>
-#include <AsyncJsonCallbackResponse.h>
-#include <AsyncJsonWebHandler.h>
-#include <ESPAsyncWebServer.h>
-
-/**
- * At the moment, not expecting services to have to deal with large JSON
- * files this could be made configurable fairly simply, it's exposed on
- * AsyncJsonWebHandler with a setter.
- */
-#define MAX_SETTINGS_SIZE 1024
-
-/*
- * Abstraction of a service which reads and writes data from an endpoint.
- *
- * Not currently used, but indended for use by features which do not
- * require setting persistance.
- */
-class SimpleService {
- private:
-  AsyncJsonWebHandler _updateHandler;
-
-  void fetchConfig(AsyncWebServerRequest* request) {
-    AsyncJsonResponse* response = new AsyncJsonResponse(false, MAX_SETTINGS_SIZE);
-    JsonObject jsonObject = response->getRoot();
-    writeToJsonObject(jsonObject);
-    response->setLength();
-    request->send(response);
-  }
-
-  void updateConfig(AsyncWebServerRequest* request, JsonDocument& jsonDocument) {
-    if (jsonDocument.is<JsonObject>()) {
-      JsonObject newConfig = jsonDocument.as<JsonObject>();
-      readFromJsonObject(newConfig);
-
-      // write settings back with a callback to reconfigure the wifi
-      AsyncJsonCallbackResponse* response =
-          new AsyncJsonCallbackResponse([this]() { onConfigUpdated(); }, false, MAX_SETTINGS_SIZE);
-      JsonObject jsonObject = response->getRoot();
-      writeToJsonObject(jsonObject);
-      response->setLength();
-      request->send(response);
-    } else {
-      request->send(400);
-    }
-  }
-
- protected:
-  // reads the local config from the
-  virtual void readFromJsonObject(JsonObject& root) {
-  }
-  virtual void writeToJsonObject(JsonObject& root) {
-  }
-
-  // implement to perform action when config has been updated
-  virtual void onConfigUpdated() {
-  }
-
- public:
-  SimpleService(AsyncWebServer* server, char const* servicePath) {
-    server->on(servicePath, HTTP_GET, std::bind(&SimpleService::fetchConfig, this, std::placeholders::_1));
-
-    _updateHandler.setUri(servicePath);
-    _updateHandler.setMethod(HTTP_POST);
-    _updateHandler.setMaxContentLength(MAX_SETTINGS_SIZE);
-    _updateHandler.onRequest(
-        std::bind(&SimpleService::updateConfig, this, std::placeholders::_1, std::placeholders::_2));
-    server->addHandler(&_updateHandler);
-  }
-
-  virtual ~SimpleService() {
-  }
-};
-
-#endif  // end SimpleService

lib/framework/StatefulService.h

@@ -0,0 +1,137 @@
+#ifndef StatefulService_h
+#define StatefulService_h
+
+#include <Arduino.h>
+#include <JsonDeserializer.h>
+#include <JsonSerializer.h>
+
+#include <list>
+#include <functional>
+#ifdef ESP32
+#include <freertos/FreeRTOS.h>
+#include <freertos/semphr.h>
+#endif
+
+typedef size_t update_handler_id_t;
+typedef std::function<void(String originId)> StateUpdateCallback;
+static update_handler_id_t currentUpdatedHandlerId;
+
+typedef struct StateUpdateHandlerInfo {
+  update_handler_id_t _id;
+  StateUpdateCallback _cb;
+  bool _allowRemove;
+  StateUpdateHandlerInfo(StateUpdateCallback cb, bool allowRemove) :
+      _id(++currentUpdatedHandlerId), _cb(cb), _allowRemove(allowRemove){};
+} StateUpdateHandlerInfo_t;
+
+template <class T>
+class StatefulService {
+ public:
+  template <typename... Args>
+#ifdef ESP32
+  StatefulService(Args&&... args) :
+      _state(std::forward<Args>(args)...), _accessMutex(xSemaphoreCreateRecursiveMutex()) {
+  }
+#else
+  StatefulService(Args&&... args) : _state(std::forward<Args>(args)...) {
+  }
+#endif
+
+  update_handler_id_t addUpdateHandler(StateUpdateCallback cb, bool allowRemove = true) {
+    if (!cb) {
+      return 0;
+    }
+    StateUpdateHandlerInfo_t updateHandler(cb, allowRemove);
+    _updateHandlers.push_back(updateHandler);
+    return updateHandler._id;
+  }
+
+  void removeUpdateHandler(update_handler_id_t id) {
+    for (auto i = _updateHandlers.begin(); i != _updateHandlers.end();) {
+      if ((*i)._allowRemove && (*i)._id == id) {
+        i = _updateHandlers.erase(i);
+      } else {
+        ++i;
+      }
+    }
+  }
+
+  void updateWithoutPropagation(std::function<void(T&)> callback) {
+#ifdef ESP32
+    xSemaphoreTakeRecursive(_accessMutex, portMAX_DELAY);
+#endif
+    callback(_state);
+#ifdef ESP32
+    xSemaphoreGiveRecursive(_accessMutex);
+#endif
+  }
+
+  void updateWithoutPropagation(JsonObject& jsonObject, JsonDeserializer<T> deserializer) {
+#ifdef ESP32
+    xSemaphoreTakeRecursive(_accessMutex, portMAX_DELAY);
+#endif
+    deserializer(jsonObject, _state);
+#ifdef ESP32
+    xSemaphoreGiveRecursive(_accessMutex);
+#endif
+  }
+
+  void update(std::function<void(T&)> callback, String originId) {
+#ifdef ESP32
+    xSemaphoreTakeRecursive(_accessMutex, portMAX_DELAY);
+#endif
+    callback(_state);
+    callUpdateHandlers(originId);
+#ifdef ESP32
+    xSemaphoreGiveRecursive(_accessMutex);
+#endif
+  }
+
+  void update(JsonObject& jsonObject, JsonDeserializer<T> deserializer, String originId) {
+#ifdef ESP32
+    xSemaphoreTakeRecursive(_accessMutex, portMAX_DELAY);
+#endif
+    deserializer(jsonObject, _state);
+    callUpdateHandlers(originId);
+#ifdef ESP32
+    xSemaphoreGiveRecursive(_accessMutex);
+#endif
+  }
+
+  void read(std::function<void(T&)> callback) {
+#ifdef ESP32
+    xSemaphoreTakeRecursive(_accessMutex, portMAX_DELAY);
+#endif
+    callback(_state);
+#ifdef ESP32
+    xSemaphoreGiveRecursive(_accessMutex);
+#endif
+  }
+
+  void read(JsonObject& jsonObject, JsonSerializer<T> serializer) {
+#ifdef ESP32
+    xSemaphoreTakeRecursive(_accessMutex, portMAX_DELAY);
+#endif
+    serializer(_state, jsonObject);
+#ifdef ESP32
+    xSemaphoreGiveRecursive(_accessMutex);
+#endif
+  }
+
+  void callUpdateHandlers(String originId) {
+    for (const StateUpdateHandlerInfo_t& updateHandler : _updateHandlers) {
+      updateHandler._cb(originId);
+    }
+  }
+
+ protected:
+  T _state;
+
+ private:
+#ifdef ESP32
+  SemaphoreHandle_t _accessMutex;
+#endif
+  std::list<StateUpdateHandlerInfo_t> _updateHandlers;
+};
+
+#endif  // end StatefulService_h

lib/framework/WebSocketTxRx.h

@@ -0,0 +1,242 @@
+#ifndef WebSocketTxRx_h
+#define WebSocketTxRx_h
+
+#include <StatefulService.h>
+#include <JsonSerializer.h>
+#include <JsonDeserializer.h>
+#include <ESPAsyncWebServer.h>
+
+#define WEB_SOCKET_MSG_SIZE 1024
+#define WEB_SOCKET_CLIENT_ID_MSG_SIZE 128
+
+#define WEB_SOCKET_ORIGIN "websocket"
+#define WEB_SOCKET_ORIGIN_CLIENT_ID_PREFIX "websocket:"
+
+template <class T>
+class WebSocketConnector {
+ protected:
+  StatefulService<T>* _statefulService;
+  AsyncWebServer* _server;
+  AsyncWebSocket _webSocket;
+
+  WebSocketConnector(StatefulService<T>* statefulService,
+                     AsyncWebServer* server,
+                     char const* webSocketPath,
+                     SecurityManager* securityManager,
+                     AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      _statefulService(statefulService), _server(server), _webSocket(webSocketPath) {
+    _webSocket.setFilter(securityManager->filterRequest(authenticationPredicate));
+    _webSocket.onEvent(std::bind(&WebSocketConnector::onWSEvent,
+                                 this,
+                                 std::placeholders::_1,
+                                 std::placeholders::_2,
+                                 std::placeholders::_3,
+                                 std::placeholders::_4,
+                                 std::placeholders::_5,
+                                 std::placeholders::_6));
+    _server->addHandler(&_webSocket);
+    _server->on(webSocketPath, HTTP_GET, std::bind(&WebSocketConnector::forbidden, this, std::placeholders::_1));
+  }
+
+  WebSocketConnector(StatefulService<T>* statefulService, AsyncWebServer* server, char const* webSocketPath) :
+      _statefulService(statefulService), _server(server), _webSocket(webSocketPath) {
+    _webSocket.onEvent(std::bind(&WebSocketConnector::onWSEvent,
+                                 this,
+                                 std::placeholders::_1,
+                                 std::placeholders::_2,
+                                 std::placeholders::_3,
+                                 std::placeholders::_4,
+                                 std::placeholders::_5,
+                                 std::placeholders::_6));
+    _server->addHandler(&_webSocket);
+  }
+
+  virtual void onWSEvent(AsyncWebSocket* server,
+                         AsyncWebSocketClient* client,
+                         AwsEventType type,
+                         void* arg,
+                         uint8_t* data,
+                         size_t len) = 0;
+
+  String clientId(AsyncWebSocketClient* client) {
+    return WEB_SOCKET_ORIGIN_CLIENT_ID_PREFIX + String(client->id());
+  }
+
+ private:
+  void forbidden(AsyncWebServerRequest* request) {
+    request->send(403);
+  }
+};
+
+template <class T>
+class WebSocketTx : virtual public WebSocketConnector<T> {
+ public:
+  WebSocketTx(JsonSerializer<T> jsonSerializer,
+              StatefulService<T>* statefulService,
+              AsyncWebServer* server,
+              char const* webSocketPath,
+              SecurityManager* securityManager,
+              AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      WebSocketConnector<T>(statefulService, server, webSocketPath, securityManager, authenticationPredicate),
+      _jsonSerializer(jsonSerializer) {
+    WebSocketConnector<T>::_statefulService->addUpdateHandler([&](String originId) { transmitData(nullptr, originId); },
+                                                              false);
+  }
+
+  WebSocketTx(JsonSerializer<T> jsonSerializer,
+              StatefulService<T>* statefulService,
+              AsyncWebServer* server,
+              char const* webSocketPath) :
+      WebSocketConnector<T>(statefulService, server, webSocketPath), _jsonSerializer(jsonSerializer) {
+    WebSocketConnector<T>::_statefulService->addUpdateHandler([&](String originId) { transmitData(nullptr, originId); },
+                                                              false);
+  }
+
+ protected:
+  virtual void onWSEvent(AsyncWebSocket* server,
+                         AsyncWebSocketClient* client,
+                         AwsEventType type,
+                         void* arg,
+                         uint8_t* data,
+                         size_t len) {
+    if (type == WS_EVT_CONNECT) {
+      // when a client connects, we transmit it's id and the current payload
+      transmitId(client);
+      transmitData(client, WEB_SOCKET_ORIGIN);
+    }
+  }
+
+ private:
+  JsonSerializer<T> _jsonSerializer;
+
+  void transmitId(AsyncWebSocketClient* client) {
+    DynamicJsonDocument jsonDocument = DynamicJsonDocument(WEB_SOCKET_CLIENT_ID_MSG_SIZE);
+    JsonObject root = jsonDocument.to<JsonObject>();
+    root["type"] = "id";
+    root["id"] = WebSocketConnector<T>::clientId(client);
+    size_t len = measureJson(jsonDocument);
+    AsyncWebSocketMessageBuffer* buffer = WebSocketConnector<T>::_webSocket.makeBuffer(len);
+    if (buffer) {
+      serializeJson(jsonDocument, (char*)buffer->get(), len + 1);
+      client->text(buffer);
+    }
+  }
+
+  /**
+   * Broadcasts the payload to the destination, if provided. Otherwise broadcasts to all clients except the origin, if
+   * specified.
+   *
+   * Original implementation sent clients their own IDs so they could ignore updates they initiated. This approach
+   * simplifies the client and the server implementation but may not be sufficent for all use-cases.
+   */
+  void transmitData(AsyncWebSocketClient* client, String originId) {
+    DynamicJsonDocument jsonDocument = DynamicJsonDocument(WEB_SOCKET_MSG_SIZE);
+    JsonObject root = jsonDocument.to<JsonObject>();
+    root["type"] = "payload";
+    root["origin_id"] = originId;
+    JsonObject payload = root.createNestedObject("payload");
+    WebSocketConnector<T>::_statefulService->read(payload, _jsonSerializer);
+
+    size_t len = measureJson(jsonDocument);
+    AsyncWebSocketMessageBuffer* buffer = WebSocketConnector<T>::_webSocket.makeBuffer(len);
+    if (buffer) {
+      serializeJson(jsonDocument, (char*)buffer->get(), len + 1);
+      if (client) {
+        client->text(buffer);
+      } else {
+        WebSocketConnector<T>::_webSocket.textAll(buffer);
+      }
+    }
+  }
+};
+
+template <class T>
+class WebSocketRx : virtual public WebSocketConnector<T> {
+ public:
+  WebSocketRx(JsonDeserializer<T> jsonDeserializer,
+              StatefulService<T>* statefulService,
+              AsyncWebServer* server,
+              char const* webSocketPath,
+              SecurityManager* securityManager,
+              AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      WebSocketConnector<T>(statefulService, server, webSocketPath, securityManager, authenticationPredicate),
+      _jsonDeserializer(jsonDeserializer) {
+  }
+
+  WebSocketRx(JsonDeserializer<T> jsonDeserializer,
+              StatefulService<T>* statefulService,
+              AsyncWebServer* server,
+              char const* webSocketPath) :
+      WebSocketConnector<T>(statefulService, server, webSocketPath), _jsonDeserializer(jsonDeserializer) {
+  }
+
+ protected:
+  virtual void onWSEvent(AsyncWebSocket* server,
+                         AsyncWebSocketClient* client,
+                         AwsEventType type,
+                         void* arg,
+                         uint8_t* data,
+                         size_t len) {
+    if (type == WS_EVT_DATA) {
+      AwsFrameInfo* info = (AwsFrameInfo*)arg;
+      if (info->final && info->index == 0 && info->len == len) {
+        if (info->opcode == WS_TEXT) {
+          DynamicJsonDocument jsonDocument = DynamicJsonDocument(WEB_SOCKET_MSG_SIZE);
+          DeserializationError error = deserializeJson(jsonDocument, (char*)data);
+          if (!error && jsonDocument.is<JsonObject>()) {
+            JsonObject jsonObject = jsonDocument.as<JsonObject>();
+            WebSocketConnector<T>::_statefulService->update(
+                jsonObject, _jsonDeserializer, WebSocketConnector<T>::clientId(client));
+          }
+        }
+      }
+    }
+  }
+
+ private:
+  JsonDeserializer<T> _jsonDeserializer;
+};
+
+template <class T>
+class WebSocketTxRx : public WebSocketTx<T>, public WebSocketRx<T> {
+ public:
+  WebSocketTxRx(JsonSerializer<T> jsonSerializer,
+                JsonDeserializer<T> jsonDeserializer,
+                StatefulService<T>* statefulService,
+                AsyncWebServer* server,
+                char const* webSocketPath,
+                SecurityManager* securityManager,
+                AuthenticationPredicate authenticationPredicate = AuthenticationPredicates::IS_ADMIN) :
+      WebSocketConnector<T>(statefulService, server, webSocketPath, securityManager, authenticationPredicate),
+      WebSocketTx<T>(jsonSerializer, statefulService, server, webSocketPath, securityManager, authenticationPredicate),
+      WebSocketRx<T>(jsonDeserializer,
+                     statefulService,
+                     server,
+                     webSocketPath,
+                     securityManager,
+                     authenticationPredicate) {
+  }
+
+  WebSocketTxRx(JsonSerializer<T> jsonSerializer,
+                JsonDeserializer<T> jsonDeserializer,
+                StatefulService<T>* statefulService,
+                AsyncWebServer* server,
+                char const* webSocketPath) :
+      WebSocketConnector<T>(statefulService, server, webSocketPath),
+      WebSocketTx<T>(jsonSerializer, statefulService, server, webSocketPath),
+      WebSocketRx<T>(jsonDeserializer, statefulService, server, webSocketPath) {
+  }
+
+ protected:
+  void onWSEvent(AsyncWebSocket* server,
+                 AsyncWebSocketClient* client,
+                 AwsEventType type,
+                 void* arg,
+                 uint8_t* data,
+                 size_t len) {
+    WebSocketRx<T>::onWSEvent(server, client, type, arg, data, len);
+    WebSocketTx<T>::onWSEvent(server, client, type, arg, data, len);
+  }
+};
+
+#endif

lib/framework/WiFiSettingsService.cpp

@@ -1,7 +1,13 @@
 #include <WiFiSettingsService.h>
 
 WiFiSettingsService::WiFiSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
-    AdminSettingsService(server, fs, securityManager, WIFI_SETTINGS_SERVICE_PATH, WIFI_SETTINGS_FILE) {
+    _httpEndpoint(WiFiSettings::serialize,
+                  WiFiSettings::deserialize,
+                  this,
+                  server,
+                  WIFI_SETTINGS_SERVICE_PATH,
+                  securityManager),
+    _fsPersistence(WiFiSettings::serialize, WiFiSettings::deserialize, this, fs, WIFI_SETTINGS_FILE) {
   // We want the device to come up in opmode=0 (WIFI_OFF), when erasing the flash this is not the default.
   // If needed, we save opmode=0 before disabling persistence so the device boots with WiFi disabled in the future.
   if (WiFi.getMode() != WIFI_OFF) {
@@ -24,60 +30,12 @@ WiFiSettingsService::WiFiSettingsService(AsyncWebServer* server, FS* fs, Securit
   _onStationModeDisconnectedHandler = WiFi.onStationModeDisconnected(
       std::bind(&WiFiSettingsService::onStationModeDisconnected, this, std::placeholders::_1));
 #endif
-}
 
-WiFiSettingsService::~WiFiSettingsService() {
+  addUpdateHandler([&](String originId) { reconfigureWiFiConnection(); }, false);
 }
 
 void WiFiSettingsService::begin() {
-  SettingsService::begin();
+  _fsPersistence.readFromFS();
-  reconfigureWiFiConnection();
-}
-
-void WiFiSettingsService::readFromJsonObject(JsonObject& root) {
-  _settings.ssid = root["ssid"] | "";
-  _settings.password = root["password"] | "";
-  _settings.hostname = root["hostname"] | "";
-  _settings.staticIPConfig = root["static_ip_config"] | false;
-
-  // extended settings
-  readIP(root, "local_ip", _settings.localIP);
-  readIP(root, "gateway_ip", _settings.gatewayIP);
-  readIP(root, "subnet_mask", _settings.subnetMask);
-  readIP(root, "dns_ip_1", _settings.dnsIP1);
-  readIP(root, "dns_ip_2", _settings.dnsIP2);
-
-  // Swap around the dns servers if 2 is populated but 1 is not
-  if (_settings.dnsIP1 == INADDR_NONE && _settings.dnsIP2 != INADDR_NONE) {
-    _settings.dnsIP1 = _settings.dnsIP2;
-    _settings.dnsIP2 = INADDR_NONE;
-  }
-
-  // Turning off static ip config if we don't meet the minimum requirements
-  // of ipAddress, gateway and subnet. This may change to static ip only
-  // as sensible defaults can be assumed for gateway and subnet
-  if (_settings.staticIPConfig &&
-      (_settings.localIP == INADDR_NONE || _settings.gatewayIP == INADDR_NONE || _settings.subnetMask == INADDR_NONE)) {
-    _settings.staticIPConfig = false;
-  }
-}
-
-void WiFiSettingsService::writeToJsonObject(JsonObject& root) {
-  // connection settings
-  root["ssid"] = _settings.ssid;
-  root["password"] = _settings.password;
-  root["hostname"] = _settings.hostname;
-  root["static_ip_config"] = _settings.staticIPConfig;
-
-  // extended settings
-  writeIP(root, "local_ip", _settings.localIP);
-  writeIP(root, "gateway_ip", _settings.gatewayIP);
-  writeIP(root, "subnet_mask", _settings.subnetMask);
-  writeIP(root, "dns_ip_1", _settings.dnsIP1);
-  writeIP(root, "dns_ip_2", _settings.dnsIP2);
-}
-
-void WiFiSettingsService::onConfigUpdated() {
   reconfigureWiFiConnection();
 }
 
@@ -95,18 +53,6 @@ void WiFiSettingsService::reconfigureWiFiConnection() {
 #endif
 }
 
-void WiFiSettingsService::readIP(JsonObject& root, String key, IPAddress& _ip) {
-  if (!root[key].is<String>() || !_ip.fromString(root[key].as<String>())) {
-    _ip = INADDR_NONE;
-  }
-}
-
-void WiFiSettingsService::writeIP(JsonObject& root, String key, IPAddress& _ip) {
-  if (_ip != INADDR_NONE) {
-    root[key] = _ip.toString();
-  }
-}
-
 void WiFiSettingsService::loop() {
   unsigned long currentMillis = millis();
   if (!_lastConnectionAttempt || (unsigned long)(currentMillis - _lastConnectionAttempt) >= WIFI_RECONNECTION_DELAY) {
@@ -117,27 +63,27 @@ void WiFiSettingsService::loop() {
 
 void WiFiSettingsService::manageSTA() {
   // Abort if already connected, or if we have no SSID
-  if (WiFi.isConnected() || _settings.ssid.length() == 0) {
+  if (WiFi.isConnected() || _state.ssid.length() == 0) {
     return;
   }
   // Connect or reconnect as required
   if ((WiFi.getMode() & WIFI_STA) == 0) {
     Serial.println("Connecting to WiFi.");
-    if (_settings.staticIPConfig) {
+    if (_state.staticIPConfig) {
       // configure for static IP
-      WiFi.config(_settings.localIP, _settings.gatewayIP, _settings.subnetMask, _settings.dnsIP1, _settings.dnsIP2);
+      WiFi.config(_state.localIP, _state.gatewayIP, _state.subnetMask, _state.dnsIP1, _state.dnsIP2);
     } else {
       // configure for DHCP
 #ifdef ESP32
       WiFi.config(INADDR_NONE, INADDR_NONE, INADDR_NONE);
-      WiFi.setHostname(_settings.hostname.c_str());
+      WiFi.setHostname(_state.hostname.c_str());
 #elif defined(ESP8266)
       WiFi.config(INADDR_ANY, INADDR_ANY, INADDR_ANY);
-      WiFi.hostname(_settings.hostname);
+      WiFi.hostname(_state.hostname);
 #endif
     }
     // attempt to connect to the network
-    WiFi.begin(_settings.ssid.c_str(), _settings.password.c_str());
+    WiFi.begin(_state.ssid.c_str(), _state.password.c_str());
   }
 }
 

lib/framework/WiFiSettingsService.h

@@ -1,8 +1,10 @@
 #ifndef WiFiSettingsService_h
 #define WiFiSettingsService_h
 
-#include <AdminSettingsService.h>
+#include <StatefulService.h>
-#include <IPAddress.h>
+#include <FSPersistence.h>
+#include <HttpEndpoint.h>
+#include <JsonUtils.h>
 
 #define WIFI_SETTINGS_FILE "/config/wifiSettings.json"
 #define WIFI_SETTINGS_SERVICE_PATH "/rest/wifiSettings"
@@ -22,22 +24,61 @@ class WiFiSettings {
   IPAddress subnetMask;
   IPAddress dnsIP1;
   IPAddress dnsIP2;
+
+  static void serialize(WiFiSettings& settings, JsonObject& root) {
+    // connection settings
+    root["ssid"] = settings.ssid;
+    root["password"] = settings.password;
+    root["hostname"] = settings.hostname;
+    root["static_ip_config"] = settings.staticIPConfig;
+
+    // extended settings
+    JsonUtils::writeIP(root, "local_ip", settings.localIP);
+    JsonUtils::writeIP(root, "gateway_ip", settings.gatewayIP);
+    JsonUtils::writeIP(root, "subnet_mask", settings.subnetMask);
+    JsonUtils::writeIP(root, "dns_ip_1", settings.dnsIP1);
+    JsonUtils::writeIP(root, "dns_ip_2", settings.dnsIP2);
+  }
+
+  static void deserialize(JsonObject& root, WiFiSettings& settings) {
+    settings.ssid = root["ssid"] | "";
+    settings.password = root["password"] | "";
+    settings.hostname = root["hostname"] | "";
+    settings.staticIPConfig = root["static_ip_config"] | false;
+
+    // extended settings
+    JsonUtils::readIP(root, "local_ip", settings.localIP);
+    JsonUtils::readIP(root, "gateway_ip", settings.gatewayIP);
+    JsonUtils::readIP(root, "subnet_mask", settings.subnetMask);
+    JsonUtils::readIP(root, "dns_ip_1", settings.dnsIP1);
+    JsonUtils::readIP(root, "dns_ip_2", settings.dnsIP2);
+
+    // Swap around the dns servers if 2 is populated but 1 is not
+    if (settings.dnsIP1 == INADDR_NONE && settings.dnsIP2 != INADDR_NONE) {
+      settings.dnsIP1 = settings.dnsIP2;
+      settings.dnsIP2 = INADDR_NONE;
+    }
+
+    // Turning off static ip config if we don't meet the minimum requirements
+    // of ipAddress, gateway and subnet. This may change to static ip only
+    // as sensible defaults can be assumed for gateway and subnet
+    if (settings.staticIPConfig &&
+        (settings.localIP == INADDR_NONE || settings.gatewayIP == INADDR_NONE || settings.subnetMask == INADDR_NONE)) {
+      settings.staticIPConfig = false;
+    }
+  }
 };
 
-class WiFiSettingsService : public AdminSettingsService<WiFiSettings> {
+class WiFiSettingsService : public StatefulService<WiFiSettings> {
  public:
   WiFiSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
-  ~WiFiSettingsService();
 
   void begin();
   void loop();
 
- protected:
-  void readFromJsonObject(JsonObject& root);
-  void writeToJsonObject(JsonObject& root);
-  void onConfigUpdated();
-
  private:
+  HttpEndpoint<WiFiSettings> _httpEndpoint;
+  FSPersistence<WiFiSettings> _fsPersistence;
   unsigned long _lastConnectionAttempt;
 
 #ifdef ESP32
@@ -49,8 +90,6 @@ class WiFiSettingsService : public AdminSettingsService<WiFiSettings> {
   void onStationModeDisconnected(const WiFiEventStationModeDisconnected& event);
 #endif
 
-  void readIP(JsonObject& root, String key, IPAddress& _ip);
-  void writeIP(JsonObject& root, String key, IPAddress& _ip);
   void reconfigureWiFiConnection();
   void manageSTA();
 };

platformio.ini

@@ -9,7 +9,7 @@ build_flags=
   ;-D ENABLE_CORS
   -D CORS_ORIGIN=\"http://localhost:3000\"
   ; Uncomment PROGMEM_WWW to enable the storage of the WWW data in PROGMEM
-  ;-D PROGMEM_WWW
+  -D PROGMEM_WWW
 
 ; ensure transitive dependencies are included for correct platforms only
 lib_compat_mode = strict
@@ -24,11 +24,12 @@ framework = arduino
 monitor_speed = 115200
 
 extra_scripts = 
-  pre:scripts/build_interface.py
+ pre:scripts/build_interface.py
 
 lib_deps =
   ArduinoJson@>=6.0.0,<7.0.0
   ESP Async WebServer@>=1.2.0,<2.0.0
+  AsyncMqttClient@>=0.8.2,<1.0.0
   
 [env:esp12e]
 platform = espressif8266
@@ -36,6 +37,7 @@ board = esp12e
 board_build.f_cpu = 160000000L
 
 [env:node32s]
-;board_build.partitions = min_spiffs.csv
+; Comment out min_spiffs.csv setting if disabling PROGMEM_WWW with ESP32
+board_build.partitions = min_spiffs.csv
 platform = espressif32
 board = node32s

src/DemoProject.cpp

@@ -1,27 +0,0 @@
-#include <DemoProject.h>
-
-DemoProject::DemoProject(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
-    AdminSettingsService(server, fs, securityManager, DEMO_SETTINGS_PATH, DEMO_SETTINGS_FILE) {
-  pinMode(BLINK_LED, OUTPUT);
-}
-
-DemoProject::~DemoProject() {
-}
-
-void DemoProject::loop() {
-  unsigned delay = MAX_DELAY / 255 * (255 - _settings.blinkSpeed);
-  unsigned long currentMillis = millis();
-  if (!_lastBlink || (unsigned long)(currentMillis - _lastBlink) >= delay) {
-    _lastBlink = currentMillis;
-    digitalWrite(BLINK_LED, !digitalRead(BLINK_LED));
-  }
-}
-
-void DemoProject::readFromJsonObject(JsonObject& root) {
-  _settings.blinkSpeed = root["blink_speed"] | DEFAULT_BLINK_SPEED;
-}
-
-void DemoProject::writeToJsonObject(JsonObject& root) {
-  // connection settings
-  root["blink_speed"] = _settings.blinkSpeed;
-}

src/DemoProject.h

@@ -1,34 +0,0 @@
-#ifndef DemoProject_h
-#define DemoProject_h
-
-#include <AdminSettingsService.h>
-#include <ESP8266React.h>
-
-#define BLINK_LED 2
-#define MAX_DELAY 1000
-
-#define DEFAULT_BLINK_SPEED 100
-#define DEMO_SETTINGS_FILE "/config/demoSettings.json"
-#define DEMO_SETTINGS_PATH "/rest/demoSettings"
-
-class DemoSettings {
- public:
-  uint8_t blinkSpeed;
-};
-
-class DemoProject : public AdminSettingsService<DemoSettings> {
- public:
-  DemoProject(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
-  ~DemoProject();
-
-  void loop();
-
- private:
-  unsigned long _lastBlink = 0;
-
- protected:
-  void readFromJsonObject(JsonObject& root);
-  void writeToJsonObject(JsonObject& root);
-};
-
-#endif

src/LightMqttSettingsService.cpp

@@ -0,0 +1,16 @@
+#include <LightMqttSettingsService.h>
+
+LightMqttSettingsService::LightMqttSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager) :
+    _httpEndpoint(LightMqttSettings::serialize,
+                  LightMqttSettings::deserialize,
+                  this,
+                  server,
+                  LIGHT_BROKER_SETTINGS_PATH,
+                  securityManager,
+                  AuthenticationPredicates::IS_AUTHENTICATED),
+    _fsPersistence(LightMqttSettings::serialize, LightMqttSettings::deserialize, this, fs, LIGHT_BROKER_SETTINGS_FILE) {
+}
+
+void LightMqttSettingsService::begin() {
+  _fsPersistence.readFromFS();
+}

src/LightMqttSettingsService.h

@@ -0,0 +1,47 @@
+#ifndef LightMqttSettingsService_h
+#define LightMqttSettingsService_h
+
+#include <HttpEndpoint.h>
+#include <FSPersistence.h>
+
+#define LIGHT_BROKER_SETTINGS_FILE "/config/brokerSettings.json"
+#define LIGHT_BROKER_SETTINGS_PATH "/rest/brokerSettings"
+
+static String defaultDeviceValue(String prefix = "") {
+#ifdef ESP32
+  return prefix + String((unsigned long)ESP.getEfuseMac(), HEX);
+#elif defined(ESP8266)
+  return prefix + String(ESP.getChipId(), HEX);
+#endif
+}
+
+class LightMqttSettings {
+ public:
+  String mqttPath;
+  String name;
+  String uniqueId;
+
+  static void serialize(LightMqttSettings& settings, JsonObject& root) {
+    root["mqtt_path"] = settings.mqttPath;
+    root["name"] = settings.name;
+    root["unique_id"] = settings.uniqueId;
+  }
+
+  static void deserialize(JsonObject& root, LightMqttSettings& settings) {
+    settings.mqttPath = root["mqtt_path"] | defaultDeviceValue("homeassistant/light/");
+    settings.name = root["name"] | defaultDeviceValue("light-");
+    settings.uniqueId = root["unique_id"] | defaultDeviceValue("light-");
+  }
+};
+
+class LightMqttSettingsService : public StatefulService<LightMqttSettings> {
+ public:
+  LightMqttSettingsService(AsyncWebServer* server, FS* fs, SecurityManager* securityManager);
+  void begin();
+
+ private:
+  HttpEndpoint<LightMqttSettings> _httpEndpoint;
+  FSPersistence<LightMqttSettings> _fsPersistence;
+};
+
+#endif  // end LightMqttSettingsService_h

src/LightStateService.cpp

@@ -0,0 +1,73 @@
+#include <LightStateService.h>
+
+LightStateService::LightStateService(AsyncWebServer* server,
+                                     SecurityManager* securityManager,
+                                     AsyncMqttClient* mqttClient,
+                                     LightMqttSettingsService* lightMqttSettingsService) :
+    _httpEndpoint(LightState::serialize,
+                  LightState::deserialize,
+                  this,
+                  server,
+                  LIGHT_SETTINGS_ENDPOINT_PATH,
+                  securityManager,
+                  AuthenticationPredicates::IS_AUTHENTICATED),
+    _mqttPubSub(LightState::haSerialize, LightState::haDeserialize, this, mqttClient),
+    _webSocket(LightState::serialize,
+               LightState::deserialize,
+               this,
+               server,
+               LIGHT_SETTINGS_SOCKET_PATH,
+               securityManager,
+               AuthenticationPredicates::IS_AUTHENTICATED),
+    _mqttClient(mqttClient),
+    _lightMqttSettingsService(lightMqttSettingsService) {
+  // configure blink led to be output
+  pinMode(BLINK_LED, OUTPUT);
+
+  // configure MQTT callback
+  _mqttClient->onConnect(std::bind(&LightStateService::registerConfig, this));
+
+  // configure update handler for when the light settings change
+  _lightMqttSettingsService->addUpdateHandler([&](String originId) { registerConfig(); }, false);
+
+  // configure settings service update handler to update LED state
+  addUpdateHandler([&](String originId) { onConfigUpdated(); }, false);
+}
+
+void LightStateService::begin() {
+  _state.ledOn = DEFAULT_LED_STATE;
+  onConfigUpdated();
+}
+
+void LightStateService::onConfigUpdated() {
+  digitalWrite(BLINK_LED, _state.ledOn ? LED_ON : LED_OFF);
+}
+
+void LightStateService::registerConfig() {
+  if (!_mqttClient->connected()) {
+    return;
+  }
+  String configTopic;
+  String setTopic;
+  String stateTopic;
+
+  DynamicJsonDocument doc(256);
+  _lightMqttSettingsService->read([&](LightMqttSettings& settings) {
+    configTopic = settings.mqttPath + "/config";
+    setTopic = settings.mqttPath + "/set";
+    stateTopic = settings.mqttPath + "/state";
+    doc["~"] = settings.mqttPath;
+    doc["name"] = settings.name;
+    doc["unique_id"] = settings.uniqueId;
+  });
+  doc["cmd_t"] = "~/set";
+  doc["stat_t"] = "~/state";
+  doc["schema"] = "json";
+  doc["brightness"] = false;
+
+  String payload;
+  serializeJson(doc, payload);
+  _mqttClient->publish(configTopic.c_str(), 0, false, payload.c_str());
+
+  _mqttPubSub.configureTopics(stateTopic, setTopic);
+}

src/LightStateService.h

@@ -0,0 +1,71 @@
+#ifndef LightStateService_h
+#define LightStateService_h
+
+#include <LightMqttSettingsService.h>
+
+#include <HttpEndpoint.h>
+#include <MqttPubSub.h>
+#include <WebSocketTxRx.h>
+
+#define BLINK_LED 2
+#define PRINT_DELAY 5000
+
+#define DEFAULT_LED_STATE false
+#define OFF_STATE "OFF"
+#define ON_STATE "ON"
+
+// Note that the built-in LED is on when the pin is low on most NodeMCU boards.
+// This is because the anode is tied to VCC and the cathode to the GPIO 4 (Arduino pin 2).
+#ifdef ESP32
+#define LED_ON 0x1
+#define LED_OFF 0x0
+#elif defined(ESP8266)
+#define LED_ON 0x0
+#define LED_OFF 0x1
+#endif
+
+#define LIGHT_SETTINGS_ENDPOINT_PATH "/rest/lightState"
+#define LIGHT_SETTINGS_SOCKET_PATH "/ws/lightState"
+
+class LightState {
+ public:
+  bool ledOn;
+
+  static void serialize(LightState& settings, JsonObject& root) {
+    root["led_on"] = settings.ledOn;
+  }
+
+  static void deserialize(JsonObject& root, LightState& settings) {
+    settings.ledOn = root["led_on"] | DEFAULT_LED_STATE;
+  }
+
+  static void haSerialize(LightState& settings, JsonObject& root) {
+    root["state"] = settings.ledOn ? ON_STATE : OFF_STATE;
+  }
+
+  static void haDeserialize(JsonObject& root, LightState& settings) {
+    String state = root["state"];
+    settings.ledOn = strcmp(ON_STATE, state.c_str()) ? false : true;
+  }
+};
+
+class LightStateService : public StatefulService<LightState> {
+ public:
+  LightStateService(AsyncWebServer* server,
+                    SecurityManager* securityManager,
+                    AsyncMqttClient* mqttClient,
+                    LightMqttSettingsService* lightMqttSettingsService);
+  void begin();
+
+ private:
+  HttpEndpoint<LightState> _httpEndpoint;
+  MqttPubSub<LightState> _mqttPubSub;
+  WebSocketTxRx<LightState> _webSocket;
+  AsyncMqttClient* _mqttClient;
+  LightMqttSettingsService* _lightMqttSettingsService;
+
+  void registerConfig();
+  void onConfigUpdated();
+};
+
+#endif

src/main.cpp

@@ -1,12 +1,18 @@
-#include <DemoProject.h>
 #include <ESP8266React.h>
+#include <LightMqttSettingsService.h>
+#include <LightStateService.h>
 #include <FS.h>
 
 #define SERIAL_BAUD_RATE 115200
 
 AsyncWebServer server(80);
 ESP8266React esp8266React(&server, &SPIFFS);
-DemoProject demoProject = DemoProject(&server, &SPIFFS, esp8266React.getSecurityManager());
+LightMqttSettingsService lightMqttSettingsService =
+    LightMqttSettingsService(&server, &SPIFFS, esp8266React.getSecurityManager());
+LightStateService lightStateService = LightStateService(&server,
+                                                        esp8266React.getSecurityManager(),
+                                                        esp8266React.getMqttClient(),
+                                                        &lightMqttSettingsService);
 
 void setup() {
   // start serial and filesystem
@@ -22,8 +28,11 @@ void setup() {
   // start the framework and demo project
   esp8266React.begin();
 
-  // start the demo project
+  // load the initial light settings
-  demoProject.begin();
+  lightStateService.begin();
+
+  // start the light service
+  lightMqttSettingsService.begin();
 
   // start the server
   server.begin();
@@ -32,7 +41,4 @@ void setup() {
 void loop() {
   // run the framework's loop function
   esp8266React.loop();
-
-  // run the demo project's loop function
-  demoProject.loop();
 }