NTP service provides incorrect time

All related to the Kernel, packages system and libraries
Post Reply
wep_hkpu
Posts: 25
Joined: Thu Mar 10, 2016 2:24 pm

NTP service provides incorrect time

Post by wep_hkpu » Wed Jan 18, 2017 8:28 am

Dear libelium-dev,

I have a system which requires Meshliums to be operated for 24-7.
Due to the device clock issues, I have configured the device based on your suggestions in previous posts.

See below for the list of commands I used:

Code: Select all

ntpdate pool.ntp.org
ntpd
remountrw
hwclock --systohc
remountro
Also, I added a line to the file rc.local

Code: Select all

/usr/sbin/ntpd
According to my observation, it seems like the devices have pooled the wrong time from ntp server.
I have several devices and this issue occurred with different devices.
The below picture shows that the ntp socket is still in used but the time is wrong.
https://drive.google.com/open?id=0B6r8l ... UZ2WjhFSWs

I'm not certain about the cause of such issue. Do you have any idea how to deal with this?

Regards.

libelium-dev
Posts: 27967
Joined: Mon Sep 28, 2009 1:06 pm

Re: NTP service provides incorrect time

Post by libelium-dev » Wed Jan 18, 2017 9:24 am

HI wep_hkpu,

Please type the commands you said but in this order, then please tell us your thoughts:

Code: Select all

remountrw
ntpdate pool.ntp.org
hwclock --systohc
remountro
Regards

wep_hkpu
Posts: 25
Joined: Thu Mar 10, 2016 2:24 pm

Re: NTP service provides incorrect time

Post by wep_hkpu » Wed Mar 01, 2017 8:44 am

Dear libelium-dev,

I have been inspected the situation for a month.
Although the sequence of commands has changed as you suggested, the NTP service still pools incorrect time occasionally.
For your information, we have several devices.
About 1/3 of the devices perform very well with out the unusual NTP problem.
Some of them encounter the problem once a week, while some of them have the problem several times in a week.

According to the incorrect time, I cannot adjust the clock by executing the command

Code: Select all

ntpdate pool.ntp.org
again since the ntp socket is in used.
The only way to adjust the time is restarting the device.
It seems like the device will forget the old time and could perform the command again after restarted (due to the command /usr/sbin/ntpd.)

Do you have any idea how to solve this problem?
Our devices have to operate in 24-7. It is a serious problem if the ntp pools the incorrect time.
Ideally, the device clock should be correct after executed the sequence of commands that you provided. I'm not sure what is the problem.

By the way, I have a question about time synchronization processes.
I understand that the device will periodically perform the NTP service with the command.

Code: Select all

/usr/sbin/ntpd
Is the command also set the device time to the time from NTP?
If no, will the two time be synchronized periodically?
Because the sequence you provided includes

Code: Select all

ntpdate pool.ntp.org
hwclock --systohc



Regards

libelium-dev
Posts: 27967
Joined: Mon Sep 28, 2009 1:06 pm

Re: NTP service provides incorrect time

Post by libelium-dev » Wed Mar 01, 2017 9:53 am

Hi,

It looks like a problem connecting with the time synchronization servers. Please, check the response time with the ntp servers with a ping command from the Meshliums:

Code: Select all

ping 0.debian.pool.ntp.org
ping 1.debian.pool.ntp.org
ping 2.debian.pool.ntp.org
ping 3.debian.pool.ntp.org
Please, reply with the output.

Regards

wep_hkpu
Posts: 25
Joined: Thu Mar 10, 2016 2:24 pm

Re: NTP service provides incorrect time

Post by wep_hkpu » Fri Mar 03, 2017 8:33 am

Dear libelium-dev,

I waited for the incorrect time from NTP services.
The picture in the link below shows the results when the time was about 6 minutes in advance the actual time.
https://drive.google.com/open?id=0B6r8l ... 0w1Uk5VV00
The response time seems to be similar.

Then, I waited for another half day.
The picture in the link below shows the results when the time was about 90 minutes in advance the actual time.
https://drive.google.com/open?id=0B6r8l ... ThGdkFCX1E
You can see the difference from 3.debian.pool.ntp.org.

However, I did the ping several times on the same device for 3.debian.pool.ntp.org.
Sometimes it had 100% packet loss.
Sometimes the response time became normal (about 3 ms).

So, the ping test may be inconclusive.
Please suggest some solutions.

Regards.

libelium-dev
Posts: 27967
Joined: Mon Sep 28, 2009 1:06 pm

Re: NTP service provides incorrect time

Post by libelium-dev » Tue Mar 07, 2017 10:53 am

Hi wep_hkpu,

Your connectivity works okay,

Something is wrong whith this ntp servers, or with meshlum ntp agent, so please try with:

Code: Select all

0.asia.pool.ntp.org
1.asia.pool.ntp.org
2.asia.pool.ntp.org
3.asia.pool.ntp.org
Anyway, try this:

Set time manually approximate time (not exactly, for testing):

Code: Select all

date --set="10:00:00"
Ensure this time is set ok:

Code: Select all

date
Force ntp update manually:

Code: Select all

ntpdate -s 0.asia.pool.ntp.org
Ensure ntp update from server ok:

Code: Select all

date
Now, schedule a cronjob every hour to force this sync:

Code: Select all

remountrw && crontab -e
Add a line like this:

Code: Select all

0 * * * * /usr/sbin/ntpdate -s 0.asia.pool.ntp.org

Best regards

wep_hkpu
Posts: 25
Joined: Thu Mar 10, 2016 2:24 pm

Re: NTP service provides incorrect time

Post by wep_hkpu » Tue Mar 14, 2017 7:58 am

Dear libelium-dev,

The address 0.asia.pool.ntp.org is unreachable.
However, I have tried your alternative solution, to set the time and assign a job in the crontab using 1.asia.pool.ntp.org.
The time is still incorrect.
Do you have any idea?

By the way, do you have some commands to restart the ntp service?
I can't let the device to capture data with the wrong timestamp.
So, I have to manually adjust the time day by day before we can solve the problem.
But I don't want to restart the Meshlium to sync the time because there will be another issue.

Sometimes, the Meshlium doesn't operate again after restart via Manager System.
It happened several times before and it is really difficult to unplug the device and plug it in again (due to the location of installation).

So, before we can permanently solve the time synchronization problem, it would be nice if we can restart the ntp service with some commands.
Also, please suggest if you have any idea to solve the incorrect time in the long run.

Regards.

libelium-dev
Posts: 27967
Joined: Mon Sep 28, 2009 1:06 pm

Re: NTP service provides incorrect time

Post by libelium-dev » Tue Mar 14, 2017 12:40 pm

Hi wep_hkpu,

We think you don't have the same time zone set in both devices. Please, check if the timezone in Meshlium is the same that you have on your computer (typing date in the terminal, the timezone will be printed). Then you can synchronise date and time using ntp. Remember that you can change the timezone using the Manager System, and after changing it you will need to restart the Meshlium in order to make the changes effective.

Best regards,

wep_hkpu
Posts: 25
Joined: Thu Mar 10, 2016 2:24 pm

Re: NTP service provides incorrect time

Post by wep_hkpu » Wed Mar 15, 2017 9:16 am

Dear libelium-dev,

I have checked the time zone again as you suggested.
All the devices are using the same time zone (HKT).

Regards.

libelium-dev
Posts: 27967
Joined: Mon Sep 28, 2009 1:06 pm

Re: NTP service provides incorrect time

Post by libelium-dev » Wed Mar 15, 2017 3:23 pm

Hi wep_hkpu,

We made some tests in order to find an issue, but it works everything we made.

Could you do the following steps, writing the results in a new post?

1. Set a fake date and time

Code: Select all

date --set="10:00:00"
The system print the new date
Wed Mar 15 10:00:00 HKT 2017
You can check typing

Code: Select all

date
2. Force ntp to update it manually

Code: Select all

ntpdate -s 0.asia.pool.ntp.org
And then check the output and the system log. There is no output in our case. But you can check if data and time have been synchronized typing

Code: Select all

date

Code: Select all

Wed Mar 15 22:12:31 HKT 2017
You can also check the system log

Code: Select all

tail -n 50 /var/log/syslog
...
Mar 15 22:06:24 localhost ntpdate[20480]: adjust time server 80.241.0.72 offset 0.011684 sec
If you get a message like this one "ntpdate: the NTP socket is in use, exiting", probably one ntp process is in the list trying to synchronise. In this case, please let us know the following information

Code: Select all

netstat -ntulp
ps aux | grep ntp
And if it exists a ntp process, kill it and repeat the process again.

If 0.asia.pool.ntp.org doesn't work, you can try using one from the list
0.asia.pool.ntp.org
1.asia.pool.ntp.org
2.asia.pool.ntp.org
3.asia.pool.ntp.org
There is no ntp service to restart in Meshlium.

Please, let us know the results of this procedure in order to find a solution.

Best regards,

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest