Giter VIP home page Giter VIP logo

espbootstrap's Introduction

EspBootstrap 2

A simple way to get your ESP8266 or ESP32 project configured and online quickly

arduino-library-badge

Background

A typical journey of an IoT device on a first boot is:

  1. Connect to WiFi network
  2. Check for firmware update (download and install if available)
  3. Connect to all the online services it has to connect to
  4. Start doing whatever it was built to do

The problem is: how would a device know which WiFi to connect to? Or how would it know that configuration of the communication network has changed?

That's when EspBootstrap comes to rescue: you can quickly provision initial configuration parameters, get your device connected to the WiFi, load, parse and store to EEPROM full set of configuration parameters, and be ready for doing whatever amazing things you plan to do.

Concept

EspBootstrap consists of three major components:

Parameters - a helper class that simplifies storing and loading configuration parameters in the EEPROM memory or SPIFFS filesystem of a device.

EspBootstrap - a helper class to quickly collect configuration parameter values from a user via simple web form in an Access Point mode

JsonConfig - a helper class to download/read and parse configuration file (simple JSON format) and populate respective configuration parameters

Both direct structure mapping and use of Dictionary data type is supported. Using dictionary is a simpler way, but may not be suitable for AVR devices with small memory.

Dictionary approach is the default.

DICTIONARY APPROACH:

Typical Implementation Journey

(see example #2)

  1. Create a Dictionary object (Make sure there is a "Title" key created as a first dictionary element for the web form)
  2. Pass dictionary to appropriate ESPBootstrap and JSONConfig objects to load or obtain configuration from a user, device file system and/or from the web

Typical device boot process

  1. Attempt to load parameters from EEPROM or file (fails first time since nothing was ever saved)

  2. Collect initial configuration parameters via web form (http://10.1.1.1) created by ESPBootstrap (typically a WiFi SSID, password and a link to web-based configuration service)

    NOTE: fields with key containing words "password" or "pwd" will have characters masked with a * symbol.

  3. Reboot and connect to WiFi with the recently obtained credentials

  4. Load JSON configuration file from the web service or filesystem using JSONConfig and populate respective configuration parameters into the provided dictionary object

  5. Save new configuration to EEPROM or file

  6. Check for OTA update, download, install and reboot if update is available

  7. Resume normal operation, periodically checking for OTA updates and config changes.

STRUCTURE MAPPING APPROACH:

Typical Implementation Journey

(see example #1)

  1. Define Parameters Structure
  2. Define Parameter "defaults"
  3. Describe Parameter structure layout for ESPBootstrap and JSONConfig
  4. Load or obtain configuration from a user and/or from the web

Typical device boot process

  1. Attempt to load parameters from EEPROM/filesystem (fails first time since nothing was ever saved)
  2. Collect initial configuration parameters via web form (http://10.1.1.1) created by EspBootstrap (typically a WiFi SSID, password and a link to web-based configuration service)
  3. Reboot and connect to WiFi with the recently obtained credentials
  4. Load JSON configuration file from the web service or a file using JsonWebConfig and populate respective configuration parameters
  5. Save new configuration to EEPROM or to the filesystem
  6. Check for OTA update, download, install and reboot if update is available
  7. Resume normal operation, periodically checking for OTA updates and config changes.

Object Types

Types of Parameters, JsonConfig and EspBootstrap objects should be chosen based on where you want to store your configuration and where you want to source your updates from:

Dictionary-based parameters, EEPROM storage, HTTP update

#include <ParametersEEPROM.h>
#include <EspBootstrapDict.h>
#include <JsonConfigHttp.h>

See examples #2 and #3 (3 and 3b) for implementation details.

Memory structure-based parameters, EEPROM storage, HTTP update

#include <ParametersEEPROMMap.h>
#include <EspBootstrapMap.h>
#include <JsonConfigHttpMap.h>

See example #1 for implementation details.

Dictionary-based parameters, SPIFFS filesystem, storage

#include <ParametersSPIFFS.h>
#include <EspBootstrapDict.h>

Parameters object uses JsonConfig file load/save capability to store the parameters.

See examples #4 and #5 for implementation details.

Full list of possible object types:

EspBootstrapDict
EspBootstrapMap

ESPBootstrap - static object to use for processing (correct type is assigned)
JsonConfigHTTP
JsonConfigHTTPMap
JsonConfigSPIFFS
JsonConfigSPIFFSMap
    
JSONConfig - static object to use for processing (correct type is assigned)
ParametersEEPROM
ParametersEEPROMMap
ParametersSPIFFS

If additional storage or update types are required, they could be implemented later (e.g., ParametersSD or JsonConfigFTP)

NOTE: Only one type of storage is supported with the static ESPBootstrap and JSONConfig objects by default. This should cover 99% of the use cases. However, if you need to support multiple storage types, compile the library with _JSONCONFIG_NOSTATIC compile option and create appropriate objects explicitly.

ERROR CODES:

Parameters:

#define PARAMS_OK   0
#define PARAMS_ERR  (-1)
#define PARAMS_LEN  (-2)
#define PARAMS_CRC  (-3)
#define PARAMS_TOK  (-4)
#define PARAMS_MEM  (-98)
#define PARAMS_ACT  (-99)

PARAMS_OK - operation finished successfully

PARAMS_ERR - operation finished with an error, unspecified

PARAMS_LEN - requested size or EPROM size is not sufficient to store parameters

PARAMS_CRC - data inconsistency between parameters in memory and in EEPROM

PARAMS_TOK - parameter tokens do not match

PARAMS_MEM - failed to allocate memory for parameters buffer

PARAMS_ACT - parameters engine was not activated with begin() method (not allocated)

EspBootstrap:

#define BOOTSTRAP_OK        0
#define BOOTSTRAP_ERR      (-1)
#define BOOTSTRAP_TIMEOUT (-99)

BOOTSTRAP_OK - bootstrap was successful. Parameters were entered and stored. No timeouts.

BOOTSTRAP_ERR - bootstrap process ended with errors (e.g., webserver failed to initiate)

BOOTSTRAP_TIMEOUT - bootstrap process ran out of time waiting for user inputs.

JsonConfig:

#define JSON_OK         0
#define JSON_ERR      (-1)
#define JSON_COMMA    (-20)
#define JSON_COLON    (-21)
#define JSON_QUOTE    (-22)
#define JSON_BCKSL    (-23)
#define JSON_MEM      (-24)
#define JSON_FMT      (-25)
#define JSON_HTTPERR  (-97)
#define JSON_NOWIFI   (-98)
#define JSON_EOF      (-99)

JSON_OK - operation finished successfully. All fields were processed as expected.

JSON_ERR - operation finished with unspecified error (reserved for future use and/or testing)

JSON_COMMA - missing comma. (Detected ":" before detecting a value)

JSON_COLON - missing colon. (Detected a comma or a new line before detecting a value)

JSON_QUOTE - missing closing quotation mark

JSON_BCKSL - orphaned back-slash. Back-slash not followed by a "verbatim" character.

JSON_MEM - error allocating memory for dictionary entry

JSON_FMT - incorrect JSON formatting (invalid character)

JSON_HTTPERR - general HTTP error. Cannot initiate a connection to provided URL.

JSON_NOWIFI - device is not connected to WiFi

JSON_EOF - unexpected end of file.

JsonConfig also returns all HTTP error codes in case of HTTP GET operation failure. For instance: 404 - page not found.


EspBootstrap v1 is deprecated, and no longer included in the library.

espbootstrap's People

Contributors

arkhipenko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

espbootstrap's Issues

A few Compiler warnings with platformio

A few compiler warnings while compiling the example ParametersEEPROM.ino with platformio.
Maybe the same is with other examples too.

EBS_example02_ParametersEEPROM.ino:36:0: warning: "_PP" redefined [enabled by default]
 #define _PP(a)
 ^
EBS_example02_ParametersEEPROM.ino:33:0: note: this is the location of the previous definition      
 #define _PP(a) Serial.print(a);
 ^
EBS_example02_ParametersEEPROM.ino:37:0: warning: "_PL" redefined [enabled by default]
 #define _PL(a)
 ^
EBS_example02_ParametersEEPROM.ino:34:0: note: this is the location of the previous definition      
 #define _PL(a) Serial.println(a);
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
 ^

and

In file included from JsonConfigHttp.h:35:0,
                 from EBS_example02_ParametersEEPROM.ino:44:
JsonConfigBase.h: In member function 'virtual int8_t JsonConfigBase::_doParse(size_t, uint16_t)':
JsonConfigBase.h:73:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < aLen; i++) {

and

EBS_example02_ParametersEEPROM.ino: In function 'void setup()':
EBS_example02_ParametersEEPROM.ino:164:23: warning: 'wifiTimeout' may be used uninitialized in this 
function [-Wmaybe-uninitialized]
   if (rc != PARAMS_OK || wifiTimeout) {
                       ^
EBS_example02_ParametersEEPROM.ino:89:8: note: 'wifiTimeout' was declared here
   bool wifiTimeout;
        ^

Adding "captive portal" functionality to `EspBootstrap::run()`

I thought that the functionality that can often be seen on public WiFi networks where you are automatically directed to a login page when connecting to that network would be quite useful for the EspBootstrap use case.

During searching for solutions I found out that this is apparently called a "captive portal" and that it can be implemented by having a DNS server on the Access Point like it is done here: https://github.com/125K/ESP8266-Captive-Portal/blob/master/WiFi_Captive_Portal.ino

I just tried adding the DNS server to EspBootstrapDict::doRun() like so:

#include <Arduino.h>
#include <DNSServer.h> //============================================= ADDED THIS
#include <EspBootstrapBase.h>
#include <Dictionary.h>

/*[...]*/

int8_t EspBootstrapDict::doRun() {

  String ssid(SSID_PREFIX);
  const IPAddress   APIP   (10, 1, 1, 1);
  const IPAddress   APMASK (255, 255, 255, 0);

  WiFi.disconnect();
  WiFi.mode(WIFI_AP);

  ssid += WiFi.macAddress();
  ssid.replace(":", "");
  ssid.toLowerCase();
//#if defined( ARDUINO_ARCH_ESP8266 )
//  ssid += String(ESP.getChipId(), HEX);
//#endif
#if defined( ARDUINO_ARCH_ESP32 )
//  ssid += String((uint32_t)( ESP.getEfuseMac() & 0xFFFFFFFFL ), HEX);
  WiFi.softAP( ssid.c_str());
  delay(50);
#endif

  WiFi.softAPConfig(APIP, APIP, APMASK);
  delay(50);
  WiFi.softAP( ssid.c_str());
  yield();

  DNSServer dnsServer; //============================================= ADDED THIS
  iServer = new WebServer(80);
  if (iServer == NULL) return BOOTSTRAP_ERR;

  dnsServer.start(53, "*", APIP); // DNS spoofing (Only for HTTP) //== ADDED THIS

  iServer->on("/submit.html", __espbootstrap_handlesubmit);
  iServer->onNotFound(__espbootstrap_handleroot);

  iAllDone = false;
  iServer->begin();

  uint32_t timeNow = millis();
  while (!iAllDone && !iCancelAP) {
    dnsServer.processNextRequest(); //================================ ADDED THIS
    iServer->handleClient();
    if ( millis() - timeNow > iTimeout ) {
        iServer->stop();
        iServer->close();
        delete iServer;
        iServer = NULL;
        return BOOTSTRAP_TIMEOUT;
    }
    delay(10);
//    yield();
  }

  iServer->stop();
  iServer->close();
  delete iServer;
  iServer = NULL;
  return (iCancelAP ? BOOTSTRAP_CANCEL: BOOTSTRAP_OK);
}

It worked just fine when connecting with my smartphone! My PC probably didn't do it because it already has a wired connection with another DNS Server answering.

I don't know enough about the structure of the EspBootstrap library but I would be toatally stoked if this feature would find its way into it since I think that it adds a really nice step to streamlining the setup process for devices.

Using `#include <ParametersSPIFFS.h>` and `#include <EspBootstrapDict.h>` in the same .ino file causes redefinition errors

Originally I noticed the compilation to fail and saw the last output of the Arduino build which was the deprecation warning about SPIFFS.

On closer inspection I just noticed that the actual cause of the failure are a lot of redefinition errors like this:

In file included from C:\Users\josts\OneDrive\Documents\Arduino\libraries\EspBootstrap\src/EspBootstrapDict.h:37,
from c:\Users\josts\Downloads\gitClones\HumiTempSensor\HumiTempSensor.ino:8:
C:\Users\josts\OneDrive\Documents\Arduino\libraries\Dictionary\src/Dictionary.h: At global scope:
C:\Users\josts\OneDrive\Documents\Arduino\libraries\Dictionary\src/Dictionary.h:3:8: error: redefinition of 'int8_t node::create(const char*, uint8_t, const char*, uint8_t, node*, node*)'
3 | int8_t node::create(const char* aKey, _DICT_KEY_TYPE aKeySize, const char* aVal, _DICT_VAL_TYPE aValSize, node* aLeft, node* aRight) {
|        ^~~~
In file included from C:\Users\josts\OneDrive\Documents\Arduino\libraries\EspBootstrap\src/ParametersSPIFFS.h:35,
from c:\Users\josts\Downloads\gitClones\HumiTempSensor\HumiTempSensor.ino:7:
C:\Users\josts\OneDrive\Documents\Arduino\libraries\Dictionary\src/Dictionary.h:3:8: note: 'int8_t node::create(const char*, uint8_t, const char*, uint8_t, node*, node*)' previously defined here
3 | int8_t node::create(const char* aKey, _DICT_KEY_TYPE aKeySize, const char* aVal, _DICT_VAL_TYPE aValSize, node* aLeft, node* aRight) {
      |        ^~~~
In file included from C:\Users\josts\OneDrive\Documents\Arduino\libraries\EspBootstrap\src/EspBootstrapDict.h:37,
from c:\Users\josts\Downloads\gitClones\HumiTempSensor\HumiTempSensor.ino:8:
C:\Users\josts\OneDrive\Documents\Arduino\libraries\Dictionary\src/Dictionary.h:58:8: error: redefinition of 'int8_t node::updateValue(const char*, uint8_t)'
58 | int8_t node::updateValue(const char* aVal, _DICT_VAL_TYPE aValSize) {
|        ^~~~
In file included from C:\Users\josts\OneDrive\Documents\Arduino\libraries\EspBootstrap\src/ParametersSPIFFS.h:35,
from c:\Users\josts\Downloads\gitClones\HumiTempSensor\HumiTempSensor.ino:7:
C:\Users\josts\OneDrive\Documents\Arduino\libraries\Dictionary\src/Dictionary.h:58:8: note: 'int8_t node::updateValue(const char*, uint8_t)' previously defined here
   58 | int8_t node::updateValue(const char* aVal, _DICT_VAL_TYPE aValSize) {
      |        ^~~~
[...]

I have the feeling that the Dictionary library has changed since I last successfully built my project (jostsalathe/HumiTempSensor). It appears to me that it has no header guard surrounding its contents which would make it unsuitable for including it in multiple other header files...

Now I don't really know what to do with that information, though...

Would you mind checking if you can reproduce this? Just to be sure that my setup isn't just broken, somehow...

Originally posted by @jostsalathe in #3 (comment)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.