This document describes Kronometrix Data Recorders.
Introduction
Kronometrix connects to a wide variety of data sources: everything from IoT devices, ICT enterprise to weather and environment sensors. In addition to multifaceted data ingress, the distributed data fabric provides high-speed transport for data consolidation, analysis and visualization in real-time.
All recorded observations are stored as raw data. Raw data is produced by a recorder, which fetches data from a system, device or sensor, data which has not been modified, altered or changed in any way. All collected metrics are variable measured sequentially in time, called time series. All these observations collected over fixed sampling intervals create a historical time series. To easy the access to all this set of data we store the observations on commodity disk drives, compressed, in text format, like CSV format.
Time series let us understand what has happened in past and look in the future, using various statistical models
Data Message
All collected metrics over time are combined as a data message. There can be many types of data messages: metrics regarding computer system utilization cpu or memory utilization, or weather data from a meteorological station, or water cubic meters per hour from an water pump. A data message is in direct relation to a data source.
Data Source
A data source, is described as any system connected to a public or private network with a valid IPv4 or IPv6 address. Example: a server, a logger, a graphic workstation, an iPad or an IoT sensor capable to send and receive data. There can be many types of data sources, each having one or many of data messages:
-
Computer system: overall cpu utilization, disk and network IO, per device metrics (Linux, FreeBSD, Windows)
-
HTTP server: throughput and utilization along with its inventory data (Nginx, Apache, Tomcat)
-
Enterprise service: response time performance and availability (SMTPS, IMAP, HTTP, LDAP, NTP, AD)
-
Automatic weather station: air temperature and pressure, humidity, wind speed and direction
What is a data recorder?
Kronometrix uses data recorders to connect to various data sources and collect data. Some other systems are using software based agents to fetch data and transport it forward for analysis.
Recorder
A light software probe, designed to connect and extract data from different sources: operating systems, web applications, weather and environment sensors, or industrial equipment, using several technologies and data communication protocols. The recorder does not offer data transport capabilities within.
Agent
A software module that resides on a certain system or device, which does not require user’s interaction, and have internal transport capabilities. Usually, a proprietary software application installed to collect performance data from a computer system, or network equipment.
Features
Light software probe
The recorders are designed as very simple standalone command-line interface utilities, which can run interactively or continnous in the background, to collect various data.
Data-Driven Programming
Recorders are designed based on the data-driven paradigm, implemented by a main loop, and focusing on structured data, pattern matching and resulting processing. This way the recorders can easily handle and process different data, without massive computing capabilities.
System Programming Language
All recorders are developed using a high-leve programming language, using C and Perl to easy and speed-up the development process and be fast.
Rapid Prototyping
It is easy to start and develop a new recorder for any technology or industry, using Kronometrix Data Recording. Based on programming languages like Perl and C there are available several level of functions: from data communication protocols, asynchronous operations to very efficient string processing.
Easy to Customize
Add or remove any metrics you want, in a matter of minutes. The recorders offer a very simple access to add new monitoring metrics.
Raw Data Support
The recorder is collecting all observations and metrics, and stores them as simple text flat files, on commodity disk drives. Data is structured as CSV records, to be universal compatible with any software or 3rd parties applications.
Data Ontology
You don’t need to record all sort of metrics and parameters. To help you, we have carefully selected and analyzed, for each industry, the most needed metrics for different business cases. The metrics have been grouped and classified, to build a very efficient data analysis process.
Timeseries Compatible
Raw data is produced by a recorder, which fetches data from a system, device or sensor, data which has not been modified, altered or changed in any way. All collected metrics are variable measured sequentially in time, called time series. All these observations collected over fixed sampling intervals create a historical time series.
IoT Readiness
Recorders are designed to connect and fetch data from various data sources, using different communication protocols. Being simple, without a monolithic architecture of a software agent, the recorders can easily adapt to any usage, from industrial MODBUS devices to healthcare or other industries. The recorders don’t offer any data transport capabilities, focusing entirely on the data collection.
Requirements
- CPU: x64, ARMv8
- Memory: 32 MB RAM
- Storage: 128 MB, 750KB per day / data source
- Protocols: SERIAL COMMUNICATION, MODBUS(RTU,TCP,ASCII), BACnet, HTTP(S)
- SBC: Raspberry PI 3B, 3B+
Key Features
- Support for raw data
- Industrial IoT readiness
- Conservative in CPU and memory usage on different architectures
- Easy to change or add new data recorders to collect new data
- Runs without human intervention, easy to detect data transmission problems
- Fetch data from any system, device, sensor with support for multi-industry
- Full control with no complicated licenses for further developments
Supported Industries
Information and Communications Technology
- sysrec – Overall system performance data recorder
- cpurec – Per CPU statistics data recorder
- nicrec – Per NIC statistics data recorder
- diskrec – Per disk statistics data recorder
- hdwrec – System inventory data recorder
- faultrec – Fault Management data recorder
- jvmrec – Java VM statistics data recorder
- snmprec – Ethernet and SAN Switch, SNMP data recorder
- netrec – TCP, UDP data recorder
- httprec – HTTP server statistics: NGINX, Apache, Tomcat, PHP-FPM
- dbrec – database data recorder: MariaDB, MySQL, PostgreSQL
- certrec – X.509 security certificate statistics data recorder
- svcrec – service performance,availability: IMAP,SMTP,POP3,LDAP,DNS,TCP(Any)
- direc – filesystem directory statistics data recorder
- ntprec – NTP server statistics data recorder
- smtprec – SMTP server data recorder
- imaprec – IMAP/POP server data recorder
- procrec – process statistics data recorder
- webrec – web application performance,availability data recorder
- wprec – WordPress security, performance and availability data recorder
Public Cloud Providers
- awsrec – Amazon Web Services data recorder
- azurec – Microsoft Azure data recorder
- gcprec – Google Computing Platform data recorder
Industrial IoT, Weather and Environment
- axisrec – AXIS security and video surveillance data recorder
- bacrec – Building management BACnet data recorder
- blrec – Bluetooth Low Energy data recorder
- rs485rec – Industrial RS-232, RS-485, MODBUS data recorder
- wsrec – General weather station recorder RS-232/USB
Getting Started
Please visit and get familiar how to install, setup and run Kronometrix Data Recording for your own business and applications.
Commercial products
Kronometrix Industrial IoT Gateway
For more information about Kronometrix products and services, please contact our support.