10.4. Frequently Asked Questions

Here some answers to questions which give a bit more detail (they might get absorbed into the main documentation over time).

10.4.1. Synchronization of logged Packages

Question:

> It is said in the Version Information section in chapter Logging Messages in Links and Nodes: “Both lnrecorder and the lnrdb Python library require at least LN version 2.1.0. Prior versions have bugs which, among other things, cause that metadata is not written correctly when the lnrecorder is terminated by a signal. It works together with older versions of LN.”

>Does the ln manager version affects the logging capability? (we recorded data using lnrecorder from links_and_nodes_manager/2.1.5-rev0@common/stable but it was connecting to a 1.x ln manager)

> If not, it is expected to have data of different length despite specifying the same rate for all topics?

Answer:

> Does the ln manager version affects the logging capability?

No, that comment does not address the manager version, but the version of the lnrecorder and the associated library.

The older lnrecorder executable did not “flush” all necessary files when stopped via Ctrl-C or similar. This can cause loss of trailing data when the lnrecorder is shut down.

> We recorded …connecting to a 1.x ln manager

I think this should be ok.

> If not, it is expected to have data of different length despite specifying the same rate for all topics?

Yes, i’d say this is to be expected.

lnrecorder can only record what was published, and when it was published.

Different topics can originate from different processes which are very unlikeley to publish at the exact same rate. (And the rate-limiter does not interpolate/extrapolate data: It acts more like a gate: either an update is passed to the subscriber or not).

And if you publish all recorded topics from the same thread then there will always be time between the topic writes.

And when the recording was stopped “in between”, you will not get the last topic-writes in your log. Much more likely is however, that the different number of packets is caused by the time it takes to create the subscriber-ports. So when you start lnrecorder, it subscribes to the existing topics. But the subscription can easily take ~0.5s per topic.

Different publishers are in no way automatically synchronized by LN to each other.

The only synchronization guarantee is that the fields within the message/update of a single topic were published by the same write (no currupt topic-packets).

If you want to compare recordings of multiple topics to each other on a common time-axis, then you should use the “source timestamp” from when that packet was published. If you know that all your topics are published from the same thread, and always all of them together, then it is usually sufficient to decide via the “source timestamp” how many samples from the beginning of the recording for each topic to ignore. and then only look at the common number of samples (more precisely, minimal number of samples left) this source-timestamp is usually the “wall clock” when the publisher did the call to ln.outport.write().

That way you can safely relate messages from different topics originating from the same host (because they see the same clock).

If you want to relate messages from topics originating from different publisher-hosts, then you need to ensure that those host’s wall-clocks are sufficiently synchronized (e.g. via (NTP) aka Network Time Protocol, PTP or other means).

lnrecorder will preserve / record this “source timestamp” in the recording in column/field publisher_timestamp.