UPnP_Generic Library to auto port-forward and provide access to Local Servers

Working libraries, libraries being ported and related hardware
Post Reply
khoih-prog
Posts: 102
Joined: Thu Feb 27, 2020 7:54 am
Location: Toronto

UPnP_Generic Library to auto port-forward and provide access to Local Servers

Post by khoih-prog »

UPnP_Generic Library

How To Install Using Arduino Library Manager


Why do we need this UPnP_Generic Library

Many of us are manually port-forwarding in Internet Gateway Device (IGD, Router) in order to provide access to local Web Services from the Internet. For example to provide access to your Local Blynk Server from Internet using Dynamic DNS or fixed IP.

This library provides the easier way to automatically port-forward by using the Simple Service Discovery Protocol (SSDP), running on nRF52, SAMD21/SAMD51, STM32F/L/H/G/WB/MP1, Teensy, ESP8266/ESP32, using ESP WiFi, WiFiNINA, Ethernet W5x00, ESP8266/ESP32 AT-command WiFi supporting UDP Multicast.

The SSDP provides a mechanism whereby network clients, with little or no static configuration, can discover network services. SSDP accomplishes this by providing for multicast discovery support as well as server based notification and discovery routing.

The SSDP is used for advertisement and discovery of network services and presence information. It accomplishes the task without assistance of server-based configuration mechanisms, such as Dynamic Host Configuration Protocol (DHCP) or Domain Name System (DNS), and without special static configuration of a network host. SSDP is the basis of the discovery protocol of Universal Plug and Play (UPnP) and is intended for use in residential or small office environments.

This UPnP_Generic Library is created to automatically update your IGDs with the requested port-forward information, using one of the many available boards / shields. See Currently Supported Boards.

The time between checks to update the UPnP Port Mappings is configurable to match your use case, and is set in the examples at 10 minutes. The LEASE_DURATION is also configurable and default to 10hrs (36000s). The Virtual Server Name can also be specified in the sketch and is shown in the IGD, e.g. NRF52-W5X00 or ESP8266-WIFI as in the following picture:

Image

The UPnP_Generic code is very short, can be immersed in your Projects and to be called in the loop() code.

This UPnP_Generic Library is based on and modified from Ofek Pearl's TinyUPnP Library to add support to many boards and shields besides ESP32 and ESP8266.


Initial Releases v3.1.4

1. Initial coding for Generic boards using many different WiFi/Ethernet modules/shields.
2. Add more examples

Currently Supported Boards

 - ESP8266
 - ESP32
 - AdaFruit Feather nRF52832, nRF52840 Express, BlueFruit Sense, Itsy-Bitsy nRF52840 Express, Metro nRF52840 Express, NINA_B302_ublox, NINA_B112_ublox etc..
 - Arduino SAMD21 (ZERO, MKR, NANO_33_IOT, etc.).
 - Adafruit SAM21 (Itsy-Bitsy M0, Metro M0, Feather M0, Gemma M0, etc.).
 - Adafruit SAM51 (Itsy-Bitsy M4, Metro M4, Grand Central M4, Feather M4 Express, etc.).
 - Seeeduino SAMD21/SAMD51 boards (SEEED_WIO_TERMINAL, SEEED_FEMTO_M0, SEEED_XIAO_M0, Wio_Lite_MG126, WIO_GPS_BOARD, SEEEDUINO_ZERO, SEEEDUINO_LORAWAN, SEEED_GROVE_UI_WIRELESS, etc.)
 - STM32 (Nucleo-144, Nucleo-64, Nucleo-32, Discovery, STM32F1, STM32F3, STM32F4, STM32H7, STM32L0, etc.).
 - STM32F/L/H/G/WB/MP1 (Nucleo-64 L053R8,Nucleo-144, Nucleo-64, Nucleo-32, Discovery, STM32Fx, STM32H7, STM32Lx, STM32Gx, STM32WB, STM32MP1, etc.) having 64K+ Flash program memory.
 
Currently Supported WiFi Modules/Shields

 - ESP8266 built-in WiFi
 - ESP32 built-in WiFi
 - WiFiNINA using WiFiNINA or WiFiNINA_Generic library.
 - ESP8266-AT, ESP32-AT WiFi shields using WiFiEspAT or ESP8266_AT_WebServer library.
 
Currently Supported Ethernet Modules/Shields

 - W5x00's using Ethernet, EthernetLarge or Ethernet3 Library.
 - W5x00's using Ethernet2 Library is also supported after applying the fix to add Multicast feature. See Libraries' Patches
 - ENC28J60 using EthernetENC or UIPEthernet library is not supported as UDP Multicast is not available by design.
 - LAN8742A using STM32Ethernet / STM32 LwIP libraries is not supported as UDP Multicast is not enabled by design, unless you modify the code to add support.


Sample Code

This is the nRF52_SimpleServer example

Code: Select all

/*
  Note: This example uses the DDNS_Generic library (https://github.com/khoih-prog/DDNS_Generic)
        You can access this WebServer by either localIP:LISTEN_PORT such as 192.169.2.100:5952
        or DDNS_Host:LISTEN_PORT, such as account.duckdns.org:5952
*/

#include "defines.h"

#define UPNP_USING_ETHERNET     true

#include <UPnP_Generic.h>

#define LISTEN_PORT         5952
#define LEASE_DURATION      36000  // seconds
#define FRIENDLY_NAME       "NRF52-W5X00"  // this name will appear in your router port forwarding section

UPnP* uPnP;

EthernetWebServer server(LISTEN_PORT);

const int led = 13;

void onUpdateCallback(const char* oldIP, const char* newIP)
{
  Serial.print("DDNSGeneric - IP Change Detected: ");
  Serial.println(newIP);
}

void handleRoot()
{
#define BUFFER_SIZE     400
  
  digitalWrite(led, 1);
  char temp[BUFFER_SIZE];
  int sec = millis() / 1000;
  int min = sec / 60;
  int hr = min / 60;
  int day = hr / 24;

  snprintf(temp, BUFFER_SIZE - 1,
           "<html>\
<head>\
<meta http-equiv='refresh' content='5'/>\
<title>%s</title>\
<style>\
body { background-color: #cccccc; font-family: Arial, Helvetica, Sans-Serif; Color: #000088; }\
</style>\
</head>\
<body>\
<h1>Hello from %s</h1>\
<h3>running UPnP_Generic & DDNS_Generic</h3>\
<h3>on %s</h3>\
<p>Uptime: %d d %02d:%02d:%02d</p>\
</body>\
</html>", BOARD_NAME, BOARD_NAME, SHIELD_TYPE, day, hr, min % 60, sec % 60);

  server.send(200, "text/html", temp);
  digitalWrite(led, 0);
}

void handleNotFound() 
{
  digitalWrite(led, 1);
  String message = "File Not Found\n\n";
  
  message += "URI: ";
  message += server.uri();
  message += "\nMethod: ";
  message += (server.method() == HTTP_GET) ? "GET" : "POST";
  message += "\nArguments: ";
  message += server.args();
  message += "\n";
  
  for (uint8_t i = 0; i < server.args(); i++) 
  {
    message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
  }
  
  server.send(404, "text/plain", message);
  digitalWrite(led, 0);
}

void setup(void) 
{
  pinMode(led, OUTPUT);
  digitalWrite(led, 0);
  
  Serial.begin(115200);
  while (!Serial);

  Serial.print("\nStart nRF52_SimpleServer on " + String(BOARD_NAME));
  Serial.println(" with " + String(SHIELD_TYPE));
  
  ET_LOGWARN3(F("Board :"), BOARD_NAME, F(", setCsPin:"), USE_THIS_SS_PIN);

  ET_LOGWARN(F("Default SPI pinout:"));
  ET_LOGWARN1(F("MOSI:"), MOSI);
  ET_LOGWARN1(F("MISO:"), MISO);
  ET_LOGWARN1(F("SCK:"),  SCK);
  ET_LOGWARN1(F("SS:"),   SS);
  ET_LOGWARN(F("========================="));

  #if !(USE_BUILTIN_ETHERNET || USE_UIP_ETHERNET)
    // For other boards, to change if necessary
    #if ( USE_ETHERNET || USE_ETHERNET_LARGE || USE_ETHERNET2  || USE_ETHERNET_ENC )
      // Must use library patch for Ethernet, Ethernet2, EthernetLarge libraries
      Ethernet.init (USE_THIS_SS_PIN);
    
    #elif USE_ETHERNET3
      // Use  MAX_SOCK_NUM = 4 for 4K, 2 for 8K, 1 for 16K RX/TX buffer
      #ifndef ETHERNET3_MAX_SOCK_NUM
        #define ETHERNET3_MAX_SOCK_NUM      4
      #endif
    
      Ethernet.setCsPin (USE_THIS_SS_PIN);
      Ethernet.init (ETHERNET3_MAX_SOCK_NUM);
  
    #elif USE_CUSTOM_ETHERNET
      // You have to add initialization for your Custom Ethernet here
      // This is just an example to setCSPin to USE_THIS_SS_PIN, and can be not correct and enough
      //Ethernet.init(USE_THIS_SS_PIN);
      
    #endif  //( ( USE_ETHERNET || USE_ETHERNET_LARGE || USE_ETHERNET2  || USE_ETHERNET_ENC )
  #endif
  
  // start the ethernet connection and the server:
  // Use DHCP dynamic IP and random mac
  uint16_t index = millis() % NUMBER_OF_MAC;
  // Use Static IP
  //Ethernet.begin(mac[index], ip);
  Ethernet.begin(mac[index]);

  IPAddress localIP = Ethernet.localIP();

  ////////////////
  
  DDNSGeneric.service("duckdns");    // Enter your DDNS Service Name - "duckdns" / "noip"

  /*
    For DDNS Providers where you get a token:
    DDNSGeneric.client("domain", "token");

    For DDNS Providers where you get username and password: ( Leave the password field empty "" if not required )
    DDNSGeneric.client("domain", "username", "password");
  */
  DDNSGeneric.client("account.duckdns.org", "12345678-1234-1234-1234-123456789012");

  DDNSGeneric.onUpdate(onUpdateCallback);

  ////////////////

  uPnP = new UPnP(60000);  // -1 means blocking, preferably, use a timeout value (ms)

  if (uPnP)
  {
    uPnP->addPortMappingConfig(localIP, LISTEN_PORT, RULE_PROTOCOL_TCP, LEASE_DURATION, FRIENDLY_NAME);

    bool portMappingAdded = false;

#define RETRY_TIMES     4
    int retries = 0;

    while (!portMappingAdded && (retries < RETRY_TIMES))
    {
      Serial.println("Add Port Forwarding, Try # " + String(++retries));

      int result = uPnP->commitPortMappings();

      portMappingAdded = ( (result == PORT_MAP_SUCCESS) || (result == ALREADY_MAPPED) );

      //Serial.println("commitPortMappings result =" + String(result));

      if (!portMappingAdded)
      {
        // for debugging, you can see this in your router too under forwarding or UPnP
        //uPnP->printAllPortMappings();
        //Serial.println(F("This was printed because adding the required port mapping failed"));
        if (retries < RETRY_TIMES)
          delay(10000);  // 10 seconds before trying again
      }
    }

    uPnP->printAllPortMappings();

    Serial.println("\nUPnP done");
  }
  
  server.on("/", handleRoot);

  server.on("/inline", []()
  {
    server.send(200, "text/plain", "this works as well");
  });

  server.onNotFound(handleNotFound);

  server.begin();

  Serial.print(F("HTTP EthernetWebServer is @ IP : "));
  Serial.print(localIP); 
  Serial.print(", port = ");
  Serial.println(LISTEN_PORT);
}

void loop(void) 
{
  DDNSGeneric.update(300000);

  uPnP->updatePortMappings(600000);  // 10 minutes

  server.handleClient();
}

Debug Termimal Output Samples

1. This is terminal debug output when running nRF52_SimpleServer example on Adafruit nRF52 NRF52840_FEATHER with W5x00 & Ethernet2 Library.

Code: Select all

Start nRF52_SimpleServer on NRF52840_FEATHER with W5x00 & Ethernet2 Library
Try # 1
[UPnP] IGD current port mappings:
0.   Blynk Server                  192.168.2.110     9443   9443   TCP    0
1.   Blynk WebServer               192.168.2.110     80     80     TCP    0
2.   Blynk Hardware Server         192.168.2.110     8080   8080   TCP    0
3.   Blynk Server                  192.168.2.110     9443   1443   TCP    0
4.   Blynk Secondary Server        192.168.2.112     9443   2443   TCP    0
5.   Blynk Sec. Hardware Server    192.168.2.112     8080   1080   TCP    0
6.   Blynk Server SSL              192.168.2.110     9443   443    TCP    0
7.   MariaDB / MySQL               192.168.2.112     5698   5698   TCP    0
8.   MariaDB / MySQL               192.168.2.112     3306   3306   TCP    0
9.   SAMD-LED-W5X00                192.168.2.85      5999   5999   TCP    25355
10.  SAMD-LED-WIFININA             192.168.2.128     5996   5996   TCP    31330
11.  ESP8266-LED-WIFI              192.168.2.81      8267   8267   TCP    32265
12.  ESP32-LED-WIFI                192.168.2.82      5933   5933   TCP    33995
13.  NRF52-LED-W5X00               192.168.2.88      5953   5953   TCP    34230
14.  SAMD-W5X00                    192.168.2.84      5990   5990   TCP    35450
15.  nRF52-W5X00                   192.168.2.93      5991   5991   TCP    35970

UPnP done
HTTP EthernetWebServer is @ IP : 192.168.2.99
[DDNS] Access whatismyipaddress
[DDNS] httpCode = 200
[DDNS] Current Public IP = aaa.bbb.ccc.ddd
[DDNS] response = aaa.bbb.ccc.ddd
[DDNS] Sending HTTP_GET to duckdns
[DDNS] HTTP_GET = http://www.duckdns.org/update?domains=account.duckdns.org&token=12345678-1234-1234-1234-123456789012&ip=aaa.bbb.ccc.ddd
[DDNS] httpCode = 200
DDNSGeneric - IP Change Detected: aaa.bbb.ccc.ddd
[DDNS] Updated IP = aaa.bbb.ccc.ddd
Image
mrburnette
Posts: 633
Joined: Thu Dec 19, 2019 1:23 am
Answers: 7

Re: UPnP_Generic Library to auto port-forward and provide access to Local Servers

Post by mrburnette »

Outstanding!
khoih-prog
Posts: 102
Joined: Thu Feb 27, 2020 7:54 am
Location: Toronto

Re: UPnP_Generic Library to auto port-forward and provide access to Local Servers

Post by khoih-prog »

@mrburnette

Thanks for your encouraging words.

BTW, I add the STM32 related test results:

1. Terminal output and screenshot of Nucleo-144 F767ZI, running W5500 with Ethernet2 Library

Code: Select all

Start STM32_SimpleServer on NUCLEO_F767ZI with W5x00 & Ethernet2 Library
Add Port Forwarding, Try # 1
[UPnP] IGD current port mappings:
0.   Blynk Server                  192.168.2.110     9443   9443   TCP    0      
1.   Blynk WebServer               192.168.2.110     80     80     TCP    0      
2.   Blynk Hardware Server         192.168.2.110     8080   8080   TCP    0      
3.   Blynk Server                  192.168.2.110     9443   1443   TCP    0      
4.   Blynk Secondary Server        192.168.2.112     9443   2443   TCP    0      
5.   Blynk Sec. Hardware Server    192.168.2.112     8080   1080   TCP    0      
6.   Blynk Server SSL              192.168.2.110     9443   443    TCP    0      
7.   MariaDB / MySQL               192.168.2.112     5698   5698   TCP    0      
8.   MariaDB / MySQL               192.168.2.112     3306   3306   TCP    0      
9.   SAMD-LED-WIFI-ESPAT           192.168.2.134     5991   5991   TCP    30045  
10.  SAMD-WIFI-ESPAT               192.168.2.134     5990   5990   TCP    34800  
11.  ESP32-LED-WIFI                192.168.2.82      5933   5933   TCP    32295  
12.  STM32-W5X00                   192.168.2.85      6032   6032   TCP    35985  

UPnP done
HTTP EthernetWebServer is @ IP : 192.168.2.85, port = 6032
[DDNS] Access whatismyipaddress
[DDNS] httpCode = 200
[DDNS] Current Public IP = 216.154.52.212
[DDNS] response = 216.154.52.212
[DDNS] Sending HTTP_GET to duckdns
[DDNS] HTTP_GET = http://www.duckdns.org/update?domains=account.duckdns.org&token=12345678-1234-1234-1234-123456789012&ip=216.154.52.212
[DDNS] httpCode = 200
DDNSGeneric - IP Change Detected: 216.154.52.212
[DDNS] Updated IP = 216.154.52.212

and we can access the WebServer running locally via DDNS (http://account.duckdns.org:6032/)

Selection_567.png
Selection_567.png (27.22 KiB) Viewed 3096 times


2. Terminal output and screenshot of Nucleo-144 F767ZI, running W5500 with Ethernet3 Library

Code: Select all

Start STM32_SimpleServer on NUCLEO_F767ZI with W5x00 & Ethernet3 Library

Ethernet3 W5500 init, using SPI_CS = 10, number of sockets = 4
**[UPnP] IGD current port mappings:
0.   Blynk Server                  192.168.2.110     9443   9443   TCP    0      
1.   Blynk WebServer               192.168.2.110     80     80     TCP    0      
3.   Blynk Server                  192.168.2.110     9443   1443   TCP    0      
5.   Blynk Sec. Hardware Server    192.168.2.112     8080   1080   TCP    0      
6.   Blynk Server SSL              192.168.2.110     9443   443    TCP    0      
7.   MariaDB / MySQL               192.168.2.112     5698   5698   TCP    0      
9.   SAMD-LED-WIFI-ESPAT           192.168.2.134     5991   5991   TCP    30660  
10.  SAMD-WIFI-ESPAT               192.168.2.134     5990   5990   TCP    35415  
11.  ESP32-LED-WIFI                192.168.2.82      5933   5933   TCP    32910  
12.  STM32-W5X00                   192.168.2.85      6032   6032   TCP    35985   

UPnP done
HTTP EthernetWebServer is @ IP : 192.168.2.85
[DDNS] Access whatismyipaddress
[DDNS] httpCode = 200
[DDNS] Current Public IP = aaa.bbb.ccc.ddd
[DDNS] response = aaa.bbb.ccc.ddd
[DDNS] Sending HTTP_GET to duckdns
[DDNS] HTTP_GET = http://www.duckdns.org/update?domains=account.duckdns.org&token=12345678-1234-1234-1234-123456789012&ip=aaa.bbb.ccc.ddd
[DDNS] httpCode = 200
DDNSGeneric - IP Change Detected: aaa.bbb.ccc.ddd
[DDNS] Updated IP = aaa.bbb.ccc.ddd
Post Reply

Return to “Libraries & Hardware”