NTP, RTC and timezone

Post here first, or if you can't find a relevant section!
Post Reply
Patrick
Posts: 22
Joined: Thu Dec 19, 2019 9:50 am
Answers: 2

NTP, RTC and timezone

Post by Patrick »

I am trying to use the RTC of my STM32F746G-Discovery with standard NTPCLient and STM32Duino RTC libraries.
I use NTP to retrieve the current UTC datetime and to set epoch of the RTC.

The problem is when I want to get the RTC value.
If I set RTC epoch with NTP as local time:
  • localtime from RTC epoch gives UTC datetime.
  • gmtime from RTC epoch gives UTC datetime minus timezone offset.
If I set RTC epoch with NTP as UTC time, same result, with additional timezone offset.

This is the test code:

Code: Select all

// Time zone
setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1); // Europe/Paris
tzset();
// NTP
ntp.setPoolServerName("192.168.2.1");
ntp.begin();
if (ntp.forceUpdate()) {
	Serial.println(ntp.getFormattedTime());
}
ntp.end();

time_t local, utc;
{
	time_t epoch = ntp.getEpochTime();
	char buffer[80];
	// Local
	struct tm ts = *localtime(&epoch);
	strftime(buffer, sizeof(buffer), "LOC %a %Y-%m-%d %H:%M:%S", &ts);
	Serial.print(buffer);
	Serial.println();
	local = mktime(&ts);
	// UTC
	ts = *gmtime(&epoch);
	strftime(buffer, sizeof(buffer), "UTC %a %Y-%m-%d %H:%M:%S", &ts);
	Serial.print(buffer);
	Serial.println();
	utc = mktime(&ts);
}

// RTC
rtc.begin();
{
	rtc.setEpoch(local); // Local time
	time_t epoch = rtc.getEpoch();
	struct tm ts;
	char buffer[80];

	ts = *localtime(&epoch); // Not local, UTC!
	strftime(buffer, sizeof(buffer), "LOC %a %Y-%m-%d %H:%M:%S", &ts);
	Serial.print(buffer);
	Serial.println();

	ts = *gmtime(&epoch); // Not UTC!
	strftime(buffer, sizeof(buffer), "UTC %a %Y-%m-%d %H:%M:%S", &ts);
	Serial.print(buffer);
	Serial.println();
}

{
	rtc.setEpoch(utc); // UTC time
	uint32_t ss = rtc.getSubSeconds();
	time_t epoch = rtc.getEpoch();
	struct tm ts;
	char buffer[80];

	ts = *localtime(&epoch); // Not local, not UTC!
	strftime(buffer, sizeof(buffer), "LOC %a %Y-%m-%d %H:%M:%S", &ts);
	Serial.print(buffer);
	Serial.println();

	ts = *gmtime(&epoch); // Not local, not UTC!
	strftime(buffer, sizeof(buffer), "UTC %a %Y-%m-%d %H:%M:%S", &ts);
	Serial.print(buffer);
	Serial.println();
}
And this is the output:

Code: Select all

15:47:50

LOC Mon 2021-11-15 16:47:50
UTC Mon 2021-11-15 15:47:50

LOC Mon 2021-11-15 15:47:50
UTC Mon 2021-11-15 14:47:50

LOC Mon 2021-11-15 14:47:50
UTC Mon 2021-11-15 13:47:50
May be I am missing something.
by Patrick » Wed Nov 17, 2021 2:14 pm
There is a follow up to this topic.

I need to monitor things for a long time (12 hours). So I wanted to setup the RTC, set an alarm and wait for alarm match.
First part is achieved with NTP and RTC, but with localtime. The problem is if there is overlap between the alarm boundaries and the timezone change (standard from/to DST). Operation can lasts +/- 1 hour in this case.
So I changed everything to setup the RTC with UTC and set timezone to Europe/Paris only to convert UTC to local for IHM and then back to UTC timezone.

This is the code for this last function:

Code: Select all

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	unsetenv("TZ");
	tzset();
}
Everything works well till after the first call to time_local. After, I had the same behavior encountered at first: datetimes are shifted:

Code: Select all

[INFO] UTC 17/11/2021 13:49:01 <-- OK
[INFO] LOC 17/11/2021 14:49:01 <-- OK
[INFO] UTC 17/11/2021 12:49:05 <-- KO
[INFO] LOC 17/11/2021 13:49:05 <-- KO
The solution is to explicitely specify UTC timezone. If TZ is not defined, tzset doesn't reset completely timezone parameters (used in mktime, called by rtc.getEpoch):

Code: Select all

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	setenv("TZ", "UTC0", 1);
	tzset();
}
And now, everything is OK and alarm is triggered:

Code: Select all

[INFO] UTC 17/11/2021 13:55:47
[INFO] LOC 17/11/2021 14:55:47
[INFO] UTC 17/11/2021 13:55:51
[INFO] LOC 17/11/2021 14:55:51
[INFO] alarm +12h set to 18/11/2021 01:55:51
[INFO] alarm +1m set to 17/11/2021 13:56:51
[INFO] alarm match
This is the complete code if you have the same needs:

Code: Select all

void time_init() {
	// NTP
	ntp.setPoolServerName("192.168.2.1");
	ntp.begin();
	ntp.forceUpdate();
	ntp.end();
	// RTC
	rtc.begin();
	rtc.setEpoch(ntp.getEpochTime());
}

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	setenv("TZ", "UTC0", 1);
	tzset();
}

void time_utc(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	gmtime_r(&epoch, &time);
}

void time_print(struct tm &time) {
	char buffer[20];
	strftime(buffer, sizeof(buffer), "%d/%m/%Y %H:%M:%S", &time);
	Serial.println(buffer);
}

void time_alarm(const unsigned long delta, void *data) {
	struct tm time = { 0 };
	time_t alarm = rtc.getEpoch() + delta;
	gmtime_r(&alarm, &time);
	rtc.attachInterrupt(time_match, data);
	rtc.setAlarmDay(time.tm_mday);
	rtc.setAlarmTime(time.tm_hour, time.tm_min, time.tm_sec);
	rtc.enableAlarm(rtc.MATCH_DHHMMSS);
	time_print(time);
}

void time_match(void *data) {
	UNUSED(data);
	Serial.println("[INFO] alarm match");
}

void time_test() {
	struct tm time;
	Serial.print("[INFO] UTC ");
	time_utc(time);
	time_print(time);
	Serial.print("[INFO] LOC ");
	time_local(time);
	time_print(time);
	// Wait 5 seconds
	delay(5000);
	Serial.print("[INFO] UTC ");
	time_utc(time);
	time_print(time);
	Serial.print("[INFO] LOC ");
	time_local(time);
	time_print(time);
	// Alarms
	Serial.print("[INFO] alarm +12h set to ");
	time_alarm(12 * 60 * 60, NULL); // 12 hours
	Serial.print("[INFO] alarm +1m set to ");
	time_alarm(1 * 60, NULL); // 1 minutes
}
Go to full post
Patrick
Posts: 22
Joined: Thu Dec 19, 2019 9:50 am
Answers: 2

Re: NTP, RTC and timezone

Post by Patrick »

I found a way to have right UTC and local datetime.
Basically, I set the RTC with functions setTime and setDate and I don't use setEpoch anymore.

This is the code:

Code: Select all

//////////
// Time //
//////////

void time_init() {
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	// NTP
	ntp.setPoolServerName("192.168.2.1");
	ntp.begin();
	if (ntp.forceUpdate()) {
		Serial.print("NTP ");
		Serial.println(ntp.getFormattedTime());
	}
	ntp.end();
	// RTC
	rtc.begin();
	struct tm time = { 0 };
	time_t now = ntp.getEpochTime();
	time = *localtime(&now);
	rtc.setTime(time.tm_hour, time.tm_min, time.tm_sec);
	rtc.setDate(time.tm_mday ? time.tm_mday : 7, time.tm_mon + 1, time.tm_year - 100);
}

void time_local(struct tm& time) {
	time_t epoch = rtc.getEpoch();
	localtime_r(&epoch, &time);
}

void time_utc(struct tm& time) {
	time_t epoch = rtc.getEpoch();
	gmtime_r(&epoch, &time);
}

void time_print(struct tm& time) {
	char buffer[20];
	strftime(buffer, sizeof(buffer), "%d/%m/%Y %H:%M:%S", &time);
	Serial.println(buffer);
}
This is the test code:

Code: Select all

// Time
time_init();
struct tm time;
// Local
Debug("LOC ");
time_local(time);
time_print(time);
// UTC
Debug("UTC ");
time_utc(time);
time_print(time);
And this is the output:

Code: Select all

NTP 13:23:40
LOC 16/11/2021 14:23:40
UTC 16/11/2021 13:23:40
A possible explanation of this behavior is that setEpoch uses gmtime so set the RTC to UTC datetime and getEpoch uses mktime so produces a local datetime.

Now I set RTC with a local datetime and get also a local datetime so everything is coherent.

This page was very useful.
Patrick
Posts: 22
Joined: Thu Dec 19, 2019 9:50 am
Answers: 2

Re: NTP, RTC and timezone

Post by Patrick »

There is a follow up to this topic.

I need to monitor things for a long time (12 hours). So I wanted to setup the RTC, set an alarm and wait for alarm match.
First part is achieved with NTP and RTC, but with localtime. The problem is if there is overlap between the alarm boundaries and the timezone change (standard from/to DST). Operation can lasts +/- 1 hour in this case.
So I changed everything to setup the RTC with UTC and set timezone to Europe/Paris only to convert UTC to local for IHM and then back to UTC timezone.

This is the code for this last function:

Code: Select all

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	unsetenv("TZ");
	tzset();
}
Everything works well till after the first call to time_local. After, I had the same behavior encountered at first: datetimes are shifted:

Code: Select all

[INFO] UTC 17/11/2021 13:49:01 <-- OK
[INFO] LOC 17/11/2021 14:49:01 <-- OK
[INFO] UTC 17/11/2021 12:49:05 <-- KO
[INFO] LOC 17/11/2021 13:49:05 <-- KO
The solution is to explicitely specify UTC timezone. If TZ is not defined, tzset doesn't reset completely timezone parameters (used in mktime, called by rtc.getEpoch):

Code: Select all

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	setenv("TZ", "UTC0", 1);
	tzset();
}
And now, everything is OK and alarm is triggered:

Code: Select all

[INFO] UTC 17/11/2021 13:55:47
[INFO] LOC 17/11/2021 14:55:47
[INFO] UTC 17/11/2021 13:55:51
[INFO] LOC 17/11/2021 14:55:51
[INFO] alarm +12h set to 18/11/2021 01:55:51
[INFO] alarm +1m set to 17/11/2021 13:56:51
[INFO] alarm match
This is the complete code if you have the same needs:

Code: Select all

void time_init() {
	// NTP
	ntp.setPoolServerName("192.168.2.1");
	ntp.begin();
	ntp.forceUpdate();
	ntp.end();
	// RTC
	rtc.begin();
	rtc.setEpoch(ntp.getEpochTime());
}

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	// Time zone Europa/Paris
	setenv("TZ", "CET-1CEST,M3.5.0,M10.5.0/3", 1);
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	setenv("TZ", "UTC0", 1);
	tzset();
}

void time_utc(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	gmtime_r(&epoch, &time);
}

void time_print(struct tm &time) {
	char buffer[20];
	strftime(buffer, sizeof(buffer), "%d/%m/%Y %H:%M:%S", &time);
	Serial.println(buffer);
}

void time_alarm(const unsigned long delta, void *data) {
	struct tm time = { 0 };
	time_t alarm = rtc.getEpoch() + delta;
	gmtime_r(&alarm, &time);
	rtc.attachInterrupt(time_match, data);
	rtc.setAlarmDay(time.tm_mday);
	rtc.setAlarmTime(time.tm_hour, time.tm_min, time.tm_sec);
	rtc.enableAlarm(rtc.MATCH_DHHMMSS);
	time_print(time);
}

void time_match(void *data) {
	UNUSED(data);
	Serial.println("[INFO] alarm match");
}

void time_test() {
	struct tm time;
	Serial.print("[INFO] UTC ");
	time_utc(time);
	time_print(time);
	Serial.print("[INFO] LOC ");
	time_local(time);
	time_print(time);
	// Wait 5 seconds
	delay(5000);
	Serial.print("[INFO] UTC ");
	time_utc(time);
	time_print(time);
	Serial.print("[INFO] LOC ");
	time_local(time);
	time_print(time);
	// Alarms
	Serial.print("[INFO] alarm +12h set to ");
	time_alarm(12 * 60 * 60, NULL); // 12 hours
	Serial.print("[INFO] alarm +1m set to ");
	time_alarm(1 * 60, NULL); // 1 minutes
}
Patrick
Posts: 22
Joined: Thu Dec 19, 2019 9:50 am
Answers: 2

Re: NTP, RTC and timezone

Post by Patrick »

Third episod!
I am now displaying the current local datetime live but the card is always freezing.
I found out that the problem is that setenv causes memory leak. It is a well known behavior, perfectly documented.

As I call it twice each time I want to get the local datetime, I quickly run out memory.

The solution is to avoid calling setenv. This is the code for the latest time_local functions:

Code: Select all

void time_local(struct tm &time) {
	time_t epoch = rtc.getEpoch();
	time_local(time, epoch);
}

void time_local(struct tm &time, time_t &epoch) {
	const char *LOCAL = "CET-1CEST,M3.5.0,M10.5.0/3";
	const char *UTC = "UTC0";
	// Time zone Europa/Paris
	char *tz = getenv("TZ");
	if (tz) {
		strcpy(tz, LOCAL);
	} else {
		setenv("TZ", LOCAL, 1);
		tz = getenv("TZ");
	}
	tzset();
	localtime_r(&epoch, &time);
	// Time zone UTC
	strcpy(tz, UTC);
	tzset();
}
getenv returns the allocated buffer for environnment variable TZ.
If getenv returns null this is because the variable doesn't exist. In this case setenv is called only once to set TZ with the largest timezone string. It will allocate the buffer for TZ.
The next calls will return the buffer large enought to accomodates all the timezone values used.
It is enough to copy the desired value in the buffer, before calling tzset and localtime_r.
This way, there is no more memory leak.

Here are some interesting links that led me to the solution, here, here and here.
Bingo600
Posts: 86
Joined: Sat Dec 21, 2019 3:56 pm

Re: NTP, RTC and timezone

Post by Bingo600 »

Thanx for sharing
mcu8484
Posts: 13
Joined: Tue Oct 20, 2020 1:11 am

Re: NTP, RTC and timezone

Post by mcu8484 »

This is an old thread but this problem is due to a bug in the stm32duino RTC library method getEpoch(). It's still there in the latest version as of today. The getEpoch() method incorrectly uses the mktime() function which assumes its input is in local time and makes relevant adjustments according to time zone settings which is not correct for the RTC which stores UTC/GM time. The mktime() is meant to complement localtime(). The proper fix for the problem is for the library to implement timegm() which is complements gmtime().
Manupilla
Posts: 1
Joined: Mon Feb 27, 2023 11:15 am

Re: NTP, RTC and timezone

Post by Manupilla »

NTP does not regconize time zones, instead it manages all time informations based on UTC. In general the handling of time zones is a job of a computer's operating system.









https://anonigstalk.com/
https://bingenerator.one/
Last edited by Manupilla on Mon Mar 20, 2023 6:42 pm, edited 1 time in total.
dannyf
Posts: 447
Joined: Sat Jul 04, 2020 7:46 pm

Re: NTP, RTC and timezone

Post by dannyf »

The proper fix for the problem is for the library to implement timegm() which is complements gmtime().
check the time zone setting.

what I usually do is to write a set of routines that turn RTC time to the unix time and vice versa. So everything is done in terms of time_t.

Much easier to port the code from one platform to another.

for stm32f1, my two pieces of code would be this, with helper functions RTCGetCNT() and RTCSetCNT().

Code: Select all

//read rtc
time_t RTC2time(time_t *t) {
	uint32_t tmp=RTCGetCNT();


	if (t==NULL) return (tmp==0)?(-1):tmp; 			//just return seconds since epoch time if pointer is NULL
	return *t=(tmp==0)?(-1):tmp;						//change pointed value + return seconds since epoch time
}

//initialize rtc with time_t
time_t time2RTC(time_t t) {
	t=(t<946684800ul)?946684800ul:t;					//minimum time is 1/1/2000 for the hardware rtc
	RTCSetCNT(t);
	return t;
}
Post Reply

Return to “General discussion”