Our full technical support staff does not monitor this forum. If you need assistance from a member of our staff, please submit your question from the Ask a Question page.


Log in or register to post/reply in the forum.

Incomplete TOA5 Files after change from index suffix to timestampsuffix

scholse1 Jan 27, 2025 01:44 PM

After Changing the File suffix from Index to a Timestamp on files which are created using the FTPClient(), some files have incomplete last rows.

I am using a CR1000X to measure 10Hz 3D Winddata using a Thies Anemometer.

Additionally to writing Data to the SD-Card we transfere the Data to a FTP Server.

For CRD Files the Tablefile created like this:

 

  TableFile ("CRD:"&Status.SerialNumber(1,1)&"_CRD_TimeSeries_",64, -1, 0, NMBR_MIN_TABLES_CRD, Min, 0, 0)

 

Initially the file on the FTP server was created like this:

 

   FTPResult=FTPClient(FTP_SERVER_ADDR,FTP_SERVER_USER,FTP_SERVER_PASS,"TimeSeries",Status.SerialNumber(1,1)&"_TimeSeries_",12,0,0,min,8)

 

But after a reboot of the logger it lost its current indexposition and restarted at zero. Therefore it always started to overwrite the files on the ftp server. Which is not ideal since after a possible Poweroutage data will be lost.

I then changed the output file to use a timestamp as suffix.

 

      FTPResult=FTPClient(FTP_SERVER_ADDR,FTP_SERVER_USER,FTP_SERVER_PASS,"TimeSeries",Status.SerialNumber(1,1)&"_TimeSeries_YYYY-MM-DD_HH-MM-SS.dat",12,0,0,min,8)

 

This introduced the current issue where sometimes, the Files written to the server are incomplete, meaning the last row is missing characters.

 

"2024-10-15 12:23:29.6",5273842,"04240470","NAN","NAN","NAN","NAN","02240266","NAN","NAN","NAN"
"2024-10-15 12:23:29.7",5273843,"04240470","NAN","NAN","NAN","NAN","02240266","NAN","NAN","NAN"
"2024-10-15 12:23:29.8",527384

 

Luckily no data is lost since when this happens, all the data is carried over to the next file but it introduced inconsistencies and might introduce new issues in the future.

What could be the underlying problem here?

Could it help if i change the TableFile option to 64 so at least the writing process for the CRD Files is quicker and therefore open up resources which might mitigate the issue?

I suspect it has something to do with the overhead that is introduced with converting everything to ASCII but at least for the FTP Files this is needed because of server side automated processing of data.

Log in or register to post/reply in the forum.