Data processing

When blocks of samples arrive in the PingDataMetrics server, they are queued on disk and loaded into the database.

Samples from a single server are processed in time-order, so that sample blocks with older data are always processed before a sample block containing newer data. The PingDataMetrics server does not do time-correlation between blocks coming from different servers. So, server A samples from two hours ago can be loaded immediately after server B samples from two minutes ago. This flexibility enables servers to be unavailable to the PingDataMetrics server, without affecting the overall system monitoring. Also, a query for data from server A and B can return data for server B but not server A, until the data queued for server A has been collected and imported. Samples collected from the PingDataMetrics server itself are processed ahead of all other servers.