Settings placeholder substitution (#164)

* Use text formatting for default factory values to produce dynamic names.
Header files contains duplicates of factory values defined in factory_settings.ini Removed them to simplify the code.

* Use text formatting for default factory values to produce dynamic names.
Header files contains duplicates of factory values defined in factory_settings.ini Removed them to simplify the code.

* Configured the WiFi host name to contain the device id by default

* Removed possibility to use placeholders for FACTORY_WIFI_SSID factory setting.

* Update README.md

Updated documentation

* Use text formatting for default factory values to produce dynamic names.
Header files contains duplicates of factory values defined in factory_settings.ini Removed them to simplify the code.

* Configured the WiFi host name to contain the device id by default

* Removed possibility to use placeholders for FACTORY_WIFI_SSID factory setting.

* Added a space to the end of the file to comply project code style

* fix typos
clang formatting
use 2 spaces in ini files
use ${platform}-${chip_id} for hostname
put chip id in brackets in AP SSID

* restore (and update) factory setting ifndefs

- this is so src can be built without an exaustive set build-time defines
- standardize ordering of defines: factory settings, paths, config

* format and modify comment

* escape spaces in pio defines
experiment with removing $'s from our format strings (they are being substituted with empty values by pio)

* fix formatting in readme
rename FactoryValue to SettingValue, put in own header
give example of direct usage of FactorySetting::format in README.md

* auto format

* use hash to delimit placeholders

* fix factory_settings.ini

* remove flash string helpers

* format ini file

* use MAC address instead of chip id for properly unique identifier

* use lower case hex encoding for unique id
use chip id and unique id for more secure secret

* fix comment

* Use random values for JWT secret
Arduino uses the ESP random number generator for "true random" numbers on both esp32 and esp8266
This makes a better JWT secret and may be useful for other factory defaults too
In addition a modification has been made to force the FSPersistance to save the file if applying defaults

* Don't use spaces in default AP SSID

* restore helpful comment in factory_settings.ini
fix default defines

Co-authored-by: kasedy <kasedy@gmail.com>
This commit is contained in:
rjwats 2021-01-03 17:00:36 +00:00 committed by GitHub
parent 6e22893051
commit e771ab134a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 191 additions and 123 deletions

View File

@ -169,12 +169,12 @@ Many of the framework's built in features may be enabled or disabled as required
Customize the settings as you see fit. A value of 0 will disable the specified feature:
```ini
-D FT_PROJECT=1
-D FT_SECURITY=1
-D FT_MQTT=1
-D FT_NTP=1
-D FT_OTA=1
-D FT_UPLOAD_FIRMWARE=1
-D FT_PROJECT=1
-D FT_SECURITY=1
-D FT_MQTT=1
-D FT_NTP=1
-D FT_OTA=1
-D FT_UPLOAD_FIRMWARE=1
```
Flag | Description
@ -193,9 +193,9 @@ The framework has built-in factory settings which act as default values for the
Customize the settings as you see fit, for example you might configure your home WiFi network as the factory default:
```ini
-D FACTORY_WIFI_SSID=\"My Awesome WiFi Network\"
-D FACTORY_WIFI_PASSWORD=\"secret\"
-D FACTORY_WIFI_HOSTNAME=\"awesome_light_controller\"
-D FACTORY_WIFI_SSID=\"My Awesome WiFi Network\"
-D FACTORY_WIFI_PASSWORD=\"secret\"
-D FACTORY_WIFI_HOSTNAME=\"awesome_light_controller\"
```
### Default access point settings
@ -221,15 +221,31 @@ It is recommended that you change the user credentials from their defaults bette
Changing factory time zone setting is a common requirement. This requires a little effort because the time zone name and POSIX format are stored as separate values for the moment. The time zone names and POSIX formats are contained in the UI code in [TZ.tsx](interface/src/ntp/TZ.tsx). Take the appropriate pair of values from there, for example, for Los Angeles you would use:
```ini
-D FACTORY_NTP_TIME_ZONE_LABEL=\"America/Los_Angeles\"
-D FACTORY_NTP_TIME_ZONE_FORMAT=\"PST8PDT,M3.2.0,M11.1.0\"
-D FACTORY_NTP_TIME_ZONE_LABEL=\"America/Los_Angeles\"
-D FACTORY_NTP_TIME_ZONE_FORMAT=\"PST8PDT,M3.2.0,M11.1.0\"
```
### Device ID factory defaults
### Placeholder substitution
If not overridden with a build flag, the framework will use the device ID to generate factory defaults for settings such as the JWT secret and MQTT client ID.
Various settings support placeholder substitution, indicated by comments in [factory_settings.ini](factory_settings.ini). This can be particularly useful where settings need to be unique, such as the Access Point SSID or MQTT client id. The following placeholders are supported:
> **Tip**: Random values are generally better defaults for these settings, so it is recommended you leave these flags undefined.
Placeholder | Substituted value
----------- | -----------------
#{platform} | The microcontroller platform, e.g. "esp32" or "esp8266"
#{unique_id} | A unique identifier derived from the MAC address, e.g. "0b0a859d6816"
#{random} | A random number encoded as a hex string, e.g. "55722f94"
You may use SettingValue::format in your own code if you require the use of these placeholders. This is demonstrated in the demo project:
```cpp
static StateUpdateResult update(JsonObject& root, LightMqttSettings& settings) {
settings.mqttPath = root["mqtt_path"] | SettingValue::format("homeassistant/light/#{unique_id}");
settings.name = root["name"] | SettingValue::format("light-#{unique_id}");
settings.uniqueId = root["unique_id"] | SettingValue::format("light-#{unique_id}");
return StateUpdateResult::CHANGED;
}
};
```
## Building for different devices

View File

@ -1,48 +1,51 @@
; The indicated settings support placeholder substitution as follows:
;
; #{platform} - The microcontroller platform, e.g. "esp32" or "esp8266"
; #{unique_id} - A unique identifier derived from the MAC address, e.g. "0b0a859d6816"
; #{random} - A random number encoded as a hex string, e.g. "55722f94"
[factory_settings]
build_flags =
; WiFi settings
-D FACTORY_WIFI_SSID=\"\"
-D FACTORY_WIFI_PASSWORD=\"\"
; if unspecified the devices hardware ID will be used
; -D FACTORY_WIFI_HOSTNAME=\"esp-react\"
build_flags =
; WiFi settings
-D FACTORY_WIFI_SSID=\"\"
-D FACTORY_WIFI_PASSWORD=\"\"
-D FACTORY_WIFI_HOSTNAME=\"#{platform}-#{unique_id}\" ; supports placeholders
; Access point settings
-D FACTORY_AP_PROVISION_MODE=AP_MODE_DISCONNECTED
-D FACTORY_AP_SSID=\"ESP8266-React\" ; 1-64 characters
-D FACTORY_AP_PASSWORD=\"esp-react\" ; 8-64 characters
-D FACTORY_AP_LOCAL_IP=\"192.168.4.1\"
-D FACTORY_AP_GATEWAY_IP=\"192.168.4.1\"
-D FACTORY_AP_SUBNET_MASK=\"255.255.255.0\"
; Access point settings
-D FACTORY_AP_PROVISION_MODE=AP_MODE_DISCONNECTED
-D FACTORY_AP_SSID=\"ESP8266-React-#{unique_id}\" ; 1-64 characters, supports placeholders
-D FACTORY_AP_PASSWORD=\"esp-react\" ; 8-64 characters
-D FACTORY_AP_LOCAL_IP=\"192.168.4.1\"
-D FACTORY_AP_GATEWAY_IP=\"192.168.4.1\"
-D FACTORY_AP_SUBNET_MASK=\"255.255.255.0\"
; User credentials for admin and guest user
-D FACTORY_ADMIN_USERNAME=\"admin\"
-D FACTORY_ADMIN_PASSWORD=\"admin\"
-D FACTORY_GUEST_USERNAME=\"guest\"
-D FACTORY_GUEST_PASSWORD=\"guest\"
; User credentials for admin and guest user
-D FACTORY_ADMIN_USERNAME=\"admin\"
-D FACTORY_ADMIN_PASSWORD=\"admin\"
-D FACTORY_GUEST_USERNAME=\"guest\"
-D FACTORY_GUEST_PASSWORD=\"guest\"
; NTP settings
-D FACTORY_NTP_ENABLED=true
-D FACTORY_NTP_TIME_ZONE_LABEL=\"Europe/London\"
-D FACTORY_NTP_TIME_ZONE_FORMAT=\"GMT0BST,M3.5.0/1,M10.5.0\"
-D FACTORY_NTP_SERVER=\"time.google.com\"
; NTP settings
-D FACTORY_NTP_ENABLED=true
-D FACTORY_NTP_TIME_ZONE_LABEL=\"Europe/London\"
-D FACTORY_NTP_TIME_ZONE_FORMAT=\"GMT0BST,M3.5.0/1,M10.5.0\"
-D FACTORY_NTP_SERVER=\"time.google.com\"
; OTA settings
-D FACTORY_OTA_PORT=8266
-D FACTORY_OTA_PASSWORD=\"esp-react\"
-D FACTORY_OTA_ENABLED=true
; OTA settings
-D FACTORY_OTA_PORT=8266
-D FACTORY_OTA_PASSWORD=\"esp-react\"
-D FACTORY_OTA_ENABLED=true
; MQTT settings
-D FACTORY_MQTT_ENABLED=false
-D FACTORY_MQTT_HOST=\"test.mosquitto.org\"
-D FACTORY_MQTT_PORT=1883
-D FACTORY_MQTT_USERNAME=\"\"
-D FACTORY_MQTT_PASSWORD=\"\"
; if unspecified the devices hardware ID will be used
;-D FACTORY_MQTT_CLIENT_ID=\"esp-react\"
-D FACTORY_MQTT_KEEP_ALIVE=60
-D FACTORY_MQTT_CLEAN_SESSION=true
-D FACTORY_MQTT_MAX_TOPIC_LENGTH=128
; MQTT settings
-D FACTORY_MQTT_ENABLED=false
-D FACTORY_MQTT_HOST=\"test.mosquitto.org\"
-D FACTORY_MQTT_PORT=1883
-D FACTORY_MQTT_USERNAME=\"\" ; supports placeholders
-D FACTORY_MQTT_PASSWORD=\"\"
-D FACTORY_MQTT_CLIENT_ID=\"#{platform}-#{unique_id}\" ; supports placeholders
-D FACTORY_MQTT_KEEP_ALIVE=60
-D FACTORY_MQTT_CLEAN_SESSION=true
-D FACTORY_MQTT_MAX_TOPIC_LENGTH=128
; JWT Secret
; if unspecified the devices hardware ID will be used
; -D FACTORY_JWT_SECRET=\"esp8266-react\"
; JWT Secret
-D FACTORY_JWT_SECRET=\"#{random}-#{random}\" ; supports placeholders

View File

@ -1,8 +1,8 @@
[features]
build_flags =
-D FT_PROJECT=1
-D FT_SECURITY=1
-D FT_MQTT=1
-D FT_NTP=1
-D FT_OTA=1
-D FT_UPLOAD_FIRMWARE=1
-D FT_PROJECT=1
-D FT_SECURITY=1
-D FT_MQTT=1
-D FT_NTP=1
-D FT_OTA=1
-D FT_UPLOAD_FIRMWARE=1

View File

@ -1,6 +1,7 @@
#ifndef APSettingsConfig_h
#define APSettingsConfig_h
#include <SettingValue.h>
#include <HttpEndpoint.h>
#include <FSPersistence.h>
#include <JsonUtils.h>
@ -8,20 +9,12 @@
#include <DNSServer.h>
#include <IPAddress.h>
#define MANAGE_NETWORK_DELAY 10000
#define AP_MODE_ALWAYS 0
#define AP_MODE_DISCONNECTED 1
#define AP_MODE_NEVER 2
#define DNS_PORT 53
#ifndef FACTORY_AP_PROVISION_MODE
#define FACTORY_AP_PROVISION_MODE AP_MODE_DISCONNECTED
#endif
#ifndef FACTORY_AP_SSID
#define FACTORY_AP_SSID "ESP8266-React"
#define FACTORY_AP_SSID "ESP8266-React-#{unique_id}"
#endif
#ifndef FACTORY_AP_PASSWORD
@ -43,6 +36,14 @@
#define AP_SETTINGS_FILE "/config/apSettings.json"
#define AP_SETTINGS_SERVICE_PATH "/rest/apSettings"
#define MANAGE_NETWORK_DELAY 10000
#define AP_MODE_ALWAYS 0
#define AP_MODE_DISCONNECTED 1
#define AP_MODE_NEVER 2
#define DNS_PORT 53
enum APNetworkStatus { ACTIVE = 0, INACTIVE, LINGERING };
class APSettings {
@ -79,7 +80,7 @@ class APSettings {
default:
newSettings.provisionMode = AP_MODE_ALWAYS;
}
newSettings.ssid = root["ssid"] | FACTORY_AP_SSID;
newSettings.ssid = root["ssid"] | SettingValue::format(FACTORY_AP_SSID);
newSettings.password = root["password"] | FACTORY_AP_PASSWORD;
JsonUtils::readIP(root, "local_ip", newSettings.localIP, FACTORY_AP_LOCAL_IP);

View File

@ -1,17 +0,0 @@
#ifndef ESPUtils_h
#define ESPUtils_h
#include <Arduino.h>
class ESPUtils {
public:
static String defaultDeviceValue(String prefix = "") {
#ifdef ESP32
return prefix + String((unsigned long)ESP.getEfuseMac(), HEX);
#elif defined(ESP8266)
return prefix + String(ESP.getChipId(), HEX);
#endif
}
};
#endif // end ESPUtils

View File

@ -38,9 +38,11 @@ class FSPersistence {
settingsFile.close();
}
// If we reach here we have not been successful in loading the config,
// hard-coded emergency defaults are now applied.
// If we reach here we have not been successful in loading the config and hard-coded defaults are now applied.
// The settings are then written back to the file system so the defaults persist between resets. This last step is
// required as in some cases defaults contain randomly generated values which would otherwise be modified on reset.
applyDefaults();
writeToFS();
}
bool writeToFS() {

View File

@ -5,12 +5,7 @@
#include <HttpEndpoint.h>
#include <FSPersistence.h>
#include <AsyncMqttClient.h>
#include <ESPUtils.h>
#define MQTT_RECONNECTION_DELAY 5000
#define MQTT_SETTINGS_FILE "/config/mqttSettings.json"
#define MQTT_SETTINGS_SERVICE_PATH "/rest/mqttSettings"
#include <SettingValue.h>
#ifndef FACTORY_MQTT_ENABLED
#define FACTORY_MQTT_ENABLED false
@ -33,7 +28,7 @@
#endif
#ifndef FACTORY_MQTT_CLIENT_ID
#define FACTORY_MQTT_CLIENT_ID generateClientId()
#define FACTORY_MQTT_CLIENT_ID "#{platform}-#{unique_id}"
#endif
#ifndef FACTORY_MQTT_KEEP_ALIVE
@ -48,13 +43,10 @@
#define FACTORY_MQTT_MAX_TOPIC_LENGTH 128
#endif
static String generateClientId() {
#ifdef ESP32
return ESPUtils::defaultDeviceValue("esp32-");
#elif defined(ESP8266)
return ESPUtils::defaultDeviceValue("esp8266-");
#endif
}
#define MQTT_SETTINGS_FILE "/config/mqttSettings.json"
#define MQTT_SETTINGS_SERVICE_PATH "/rest/mqttSettings"
#define MQTT_RECONNECTION_DELAY 5000
class MqttSettings {
public:
@ -91,9 +83,9 @@ class MqttSettings {
settings.enabled = root["enabled"] | FACTORY_MQTT_ENABLED;
settings.host = root["host"] | FACTORY_MQTT_HOST;
settings.port = root["port"] | FACTORY_MQTT_PORT;
settings.username = root["username"] | FACTORY_MQTT_USERNAME;
settings.username = root["username"] | SettingValue::format(FACTORY_MQTT_USERNAME);
settings.password = root["password"] | FACTORY_MQTT_PASSWORD;
settings.clientId = root["client_id"] | FACTORY_MQTT_CLIENT_ID;
settings.clientId = root["client_id"] | SettingValue::format(FACTORY_MQTT_CLIENT_ID);
settings.keepAlive = root["keep_alive"] | FACTORY_MQTT_KEEP_ALIVE;
settings.cleanSession = root["clean_session"] | FACTORY_MQTT_CLEAN_SESSION;
settings.maxTopicLength = root["max_topic_length"] | FACTORY_MQTT_MAX_TOPIC_LENGTH;

View File

@ -4,14 +4,9 @@
#include <Features.h>
#include <ArduinoJsonJWT.h>
#include <ESPAsyncWebServer.h>
#include <ESPUtils.h>
#include <AsyncJson.h>
#include <list>
#ifndef FACTORY_JWT_SECRET
#define FACTORY_JWT_SECRET ESPUtils::defaultDeviceValue()
#endif
#define ACCESS_TOKEN_PARAMATER "access_token"
#define AUTHORIZATION_HEADER "Authorization"

View File

@ -1,11 +1,16 @@
#ifndef SecuritySettingsService_h
#define SecuritySettingsService_h
#include <SettingValue.h>
#include <Features.h>
#include <SecurityManager.h>
#include <HttpEndpoint.h>
#include <FSPersistence.h>
#ifndef FACTORY_JWT_SECRET
#define FACTORY_JWT_SECRET "#{random}-#{random}"
#endif
#ifndef FACTORY_ADMIN_USERNAME
#define FACTORY_ADMIN_USERNAME "admin"
#endif
@ -48,7 +53,7 @@ class SecuritySettings {
static StateUpdateResult update(JsonObject& root, SecuritySettings& settings) {
// secret
settings.jwtSecret = root["jwt_secret"] | FACTORY_JWT_SECRET;
settings.jwtSecret = root["jwt_secret"] | SettingValue::format(FACTORY_JWT_SECRET);
// users
settings.users.clear();
@ -103,7 +108,7 @@ class SecuritySettingsService : public SecurityManager {
SecuritySettingsService(AsyncWebServer* server, FS* fs);
~SecuritySettingsService();
// minimal set of functions to support framework with security settings disabled
// minimal set of functions to support framework with security settings disabled
Authentication authenticateRequest(AsyncWebServerRequest* request);
ArRequestFilterFunction filterRequest(AuthenticationPredicate predicate);
ArRequestHandlerFunction wrapRequest(ArRequestHandlerFunction onRequest, AuthenticationPredicate predicate);

View File

@ -0,0 +1,55 @@
#include <SettingValue.h>
namespace SettingValue {
#ifdef ESP32
const String PLATFORM = "esp32";
#elif defined(ESP8266)
const String PLATFORM = "esp8266";
#endif
/**
* Returns a new string after replacing each instance of the pattern with a value generated by calling the provided
* callback.
*/
String replaceEach(String value, String pattern, String (*generateReplacement)()) {
while (true) {
int index = value.indexOf(pattern);
if (index == -1) {
break;
}
value = value.substring(0, index) + generateReplacement() + value.substring(index + pattern.length());
}
return value;
}
/**
* Generates a random number, encoded as a hex string.
*/
String getRandom() {
return String(random(2147483647), HEX);
}
/**
* Uses the station's MAC address to create a unique id for each device.
*/
String getUniqueId() {
uint8_t mac[6];
#ifdef ESP32
esp_read_mac(mac, ESP_MAC_WIFI_STA);
#elif defined(ESP8266)
wifi_get_macaddr(STATION_IF, mac);
#endif
char macStr[13] = {0};
sprintf(macStr, "%02x%02x%02x%02x%02x%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
return String(macStr);
}
String format(String value) {
value = replaceEach(value, "#{random}", getRandom);
value.replace("#{unique_id}", getUniqueId());
value.replace("#{platform}", PLATFORM);
return value;
}
}; // end namespace SettingValue

View File

@ -0,0 +1,14 @@
#ifndef SettingValue_h
#define SettingValue_h
#include <Arduino.h>
#ifdef ESP8266
#include <ESP8266WiFi.h>
#endif
namespace SettingValue {
String format(String value);
};
#endif // end SettingValue

View File

@ -1,15 +1,12 @@
#ifndef WiFiSettingsService_h
#define WiFiSettingsService_h
#include <SettingValue.h>
#include <StatefulService.h>
#include <FSPersistence.h>
#include <HttpEndpoint.h>
#include <JsonUtils.h>
#define WIFI_SETTINGS_FILE "/config/wifiSettings.json"
#define WIFI_SETTINGS_SERVICE_PATH "/rest/wifiSettings"
#define WIFI_RECONNECTION_DELAY 1000 * 30
#ifndef FACTORY_WIFI_SSID
#define FACTORY_WIFI_SSID ""
#endif
@ -19,9 +16,14 @@
#endif
#ifndef FACTORY_WIFI_HOSTNAME
#define FACTORY_WIFI_HOSTNAME ESPUtils::defaultDeviceValue("esp-react-")
#define FACTORY_WIFI_HOSTNAME "#{platform}-#{unique_id}"
#endif
#define WIFI_SETTINGS_FILE "/config/wifiSettings.json"
#define WIFI_SETTINGS_SERVICE_PATH "/rest/wifiSettings"
#define WIFI_RECONNECTION_DELAY 1000 * 30
class WiFiSettings {
public:
// core wifi configuration
@ -55,7 +57,7 @@ class WiFiSettings {
static StateUpdateResult update(JsonObject& root, WiFiSettings& settings) {
settings.ssid = root["ssid"] | FACTORY_WIFI_SSID;
settings.password = root["password"] | FACTORY_WIFI_PASSWORD;
settings.hostname = root["hostname"] | FACTORY_WIFI_HOSTNAME;
settings.hostname = root["hostname"] | SettingValue::format(FACTORY_WIFI_HOSTNAME);
settings.staticIPConfig = root["static_ip_config"] | false;
// extended settings

View File

@ -3,7 +3,7 @@
#include <HttpEndpoint.h>
#include <FSPersistence.h>
#include <ESPUtils.h>
#include <SettingValue.h>
#define LIGHT_BROKER_SETTINGS_FILE "/config/brokerSettings.json"
#define LIGHT_BROKER_SETTINGS_PATH "/rest/brokerSettings"
@ -21,9 +21,9 @@ class LightMqttSettings {
}
static StateUpdateResult update(JsonObject& root, LightMqttSettings& settings) {
settings.mqttPath = root["mqtt_path"] | ESPUtils::defaultDeviceValue("homeassistant/light/");
settings.name = root["name"] | ESPUtils::defaultDeviceValue("light-");
settings.uniqueId = root["unique_id"] | ESPUtils::defaultDeviceValue("light-");
settings.mqttPath = root["mqtt_path"] | SettingValue::format("homeassistant/light/#{unique_id}");
settings.name = root["name"] | SettingValue::format("light-#{unique_id}");
settings.uniqueId = root["unique_id"] | SettingValue::format("light-#{unique_id}");
return StateUpdateResult::CHANGED;
}
};