DMPACK | DABAMOS

Deformation Monitoring Package (DMPACK)

The Deformation Monitoring Package (DMPACK) is a free and open source IoT middleware for wireless sensor networks, automated sensor control, and real-time time series processing in geodesy and geotechnics. The package consists of a library libdmpack and additional programs based on it which serve as a reference implementation of solutions to various problems in modern deformation monitoring, among them:

sensor control (RS-232/422/485, 1-Wire, named pipe, sub-process),
Leica GeoCOM API for Fortran,
SQLite database access,
message passing via POSIX message queues,
inter-process communication via POSIX semaphores,
data serialisation (ASCII, CSV, JSON, JSON Lines, HDF5, Fortran 95 Namelist),
compression (deflate, zstd),
HTTP-RPC API for remote procedure calls,
data synchronisation between client and server,
distributed logging,

client status messages (heartbeats),
server-side web applications,
time series plotting,
MQTT connectivity,
Lua scripting,
HTML5 reports,
Atom XML web feeds.
e-mail,
regular expression matching,
command-line argument parsing,
configuration file evaluation.

DMPACK is a scientific monitoring and early warning system developed for risk mitigations through autonomous control measurements of buildings, infrastructure, terrain, geodetic nets, and other objects, like:

bridges, tunnels, dams;
landslides, cliffs, glaciers;
construction sites, mining areas;
churches, monasteries, and other heritage buildings.

The software runs on sensor nodes, usually industrial embedded systems or single-board computers, and obtains observation data from arbitrary sensors, like robotic total stations, digital levels, inclinometers, weather stations, or GNSS receivers. The raw sensor data is then processed, stored, and optionally transmitted to a server.

The package is suitable for a wide range of scenarios, such as:

bridge monitoring,
slope monitoring,
land and rock slide prediction,
structural health monitoring,
environmental monitoring,
geo-hazard assessment.

DMPACK is built around the relational SQLite database for time series and log storage on client and server. The server component is optional. It is possible to run DMPACK on sensor clients only, without data distribution. The client-side inter-process communication (IPC) is based on POSIX message queues and POSIX semaphores.

The software is written in Fortran 2018 and released under ISC licence.

DMPACK client–server architecture — Schematic view of the **DMPACK** client–server architecture

Download

DMPACK is written in Fortran 2018 and has to be compiled from source:

Source Code Repository (GitHub)

Please see the User Guide for instructions on how to build from source on Linux or FreeBSD.

Documentation

The online documentation is available in HTML format:

Additional man pages are included in the source code repository.

Requirements

At the current development stage, the following operating systems and 64-bit platforms are supported:

GNU/Linux (x86-64, AArch64)
FreeBSD (x86-64, AArch64)

The software package is compatible to 64-bit single-board computers only, such as Raspberry Pi 3/4/5.

A Fortran 2018 and an ANSI C compiler are required for building DMPACK from source.

Software Stack

DMPACK stack

Programs

DMPACK includes software programs for sensor I/O, database management, observation processing, and other tasks related to automated control measurements.

dmapi: A FastCGI-based HTTP-RPC service that provides an API for node, sensor, target, observation, and log synchronisation, as well as heartbeat transmission. Clients may either send records to be stored in the server database, or request data of a given time range. Depending on the HTTP Accept header, the server returns data in CSV, JSON, JSON Lines or Namelist format. Requires a FastCGI-compatible web server, such as lighttpd(1).
dmbackup: Creates an online backup of a database by either using the SQLite backup API or VACUUM INTO.
dmbeat: Sends short status messages (heartbeats) periodically to a remote dmapi instance.
dmdb: Stores observations received from POSIX message queue in a SQLite database.
dmdbctl: A command-line interface to the DMPACK observation database, to read, add, update, or delete nodes, sensors, and targets.
dmexport: Exports beats, nodes, sensors, targets, observations, and logs from database to file, either in CSV, JSON, or JSON Lines format.
dmfeed: Creates an Atom syndication feed in XML format (RFC 4287) from logs of given sensor node and log level. If the feed is served by a web server, clients can subscribe to it by using a feed reader or news aggregator. The program may be executed periodically as a cron job.
dmfs: Reads sensor data from virtual file system, file, or named pipe. The program be used to read values from sensors connected via 1-Wire (OWFS). Observations are forwarded via POSIX message queue and/or written to file.
dmgrc: Creates log messages from GeoCOM return codes.
dmimport: Imports nodes, sensors, targets, observations, and logs from CSV file into database.
dminfo: Prints system and database information as key–value pairs to standard output.
dminit: Creates and initialises SQLite observation, log, and beat databases.

dmlog: A utility program to send log messages from command-line or shell script to the POSIX message queue of a dmlogger process, to be stored in the log database.
dmlogger: Stores logs received from POSIX message queue in a SQLite database.
dmlua: Runs a custom Lua script to process an observation and forward it to the next specified receiver.
dmpipe: Executes a program as a sub-process connected through an anonymous pipe and forwards the output via POSIX message queue. Optionally, observations are written to file or stdout.
dmplot: Creates line plots of time series read from database, with output to file, terminal, or X11 window. Uses gnuplot(1) internally as plotting backend.
dmrecv: Receives logs or observations from POSIX message queue and writes them to stdout, file, or named pipe.
dmreport: Creates HTML reports containing plots and optionally log messages.
dmsend: Sends observations or logs from file to a DMPACK application via POSIX message queue.
dmserial: Connects to a TTY/PTY serial port for sensor communication. The program sends requests to a connected sensor to receive responses. The program pre-processes the response data using regular expressions and forwards observations via POSIX message queue.
dmsync: Synchronises nodes, sensors, targets, observations, and log messages between client and dmapi server. Only uni-directional synchronisation from client to server is supported.
dmuuid: A command-line tool to generate UUID4 identifiers (by default in hex format without hyphens).
dmweb: A CGI-based web user interface for DMPACK database access on client and server. Requires a web server and gnuplot(1).