DMPACK

Download Documentation Programs Screen Shots Dependencies

The Deformation Monitoring Package (DMPACK) is a free and open source IoT middleware for wireless sensor networks, automated sensor control, and real-time time series processing in engineering geodesy and geotechnics. The package consists of a library libdmpack and additional programs based on it which serve as a reference implementation of solutions to various problems in modern deformation monitoring, among them:

sensor control (RS-232/422/485, TTL/UART, Modbus RTU/TCP, 1-Wire, sub-process, file system),
SQLite database access,
message passing via POSIX message queues,
process synchronisation via POSIX semaphores,
data serialisation (ASCII, CSV, GeoJSON, HDF5, JSON, JSON Lines, Namelist),
compression (deflate, zstd),
HTTP-based remote procedure call API,
concurrent client–server synchronisation,
FTP file transfer,
distributed logging,
client status messages (heartbeats),
server-side web applications,

time series plotting,
MQTT and XMPP connectivity,
Leica GeoCOM API for Fortran,
Deutscher Wetterdienst (DWD) API,
camera access (V4L2, RTSP),
MPPT and battery monitoring (VE.Direct),
report generation (HTML5, PDF),
Atom XML web feeds,
Lua scripting,
e-mail,
regular expression matching,
command-line argument parsing,
configuration file evaluation.

DMPACK is a scientific monitoring and early warning system developed for risk mitigations through autonomous control measurements of buildings, infrastructure, terrain, geodetic nets, and other objects, like:

roads, bridges, tunnels, dams;
landslides, cliffs, glaciers;
construction sites, mining areas;
churches, monasteries, and other heritage buildings.

The software runs on sensor nodes, usually industrial embedded systems or single-board computers, and obtains observation data from arbitrary sensors, like robotic total stations, digital levels, inclinometers, weather stations, or GNSS receivers. The raw sensor data is then processed, stored, and optionally transmitted to a server.

DMPACK Tech Stack

The package is suitable for a wide range of scenarios, such as:

bridge monitoring,
slope monitoring,
land and rock slide prediction,
structural health monitoring,
environmental monitoring,
geo-hazard assessment.

DMPACK is built around the relational SQLite database for time series and log storage on client and server. The server component is optional. It is possible to run DMPACK on sensor clients only, without data distribution. The client-side inter-process communication (IPC) is based on POSIX message queues and POSIX semaphores.

The software is written in Fortran 2018 and released under ISC licence.

DMPACK client–server architecture — Schematic view of the **DMPACK** client–server architecture

Download

DMPACK is written in Fortran 2018 and has to be compiled from source:

Please see the User Guide for instructions on how to build the software from source on Linux or FreeBSD.

Documentation

The software and source code documentation is available in HTML format:

Additional man pages are included in the source code repository.

Requirements

At the current development stage, the following operating systems and architectures are supported:

GNU/Linux (x86-64, AArch64)
FreeBSD (x86-64, AArch64)

The software package is compatible to 64-bit single-board computers only, such as Raspberry Pi 3/4/5.

A Fortran 2018 and an ANSI C compiler are required for building DMPACK from source.

Software Stack

DMPACK stack

Programs

DMPACK includes software programs for sensor I/O, database management, observation processing, and other tasks related to automated control measurements.

dmapi: A FastCGI-based HTTP-RPC service that provides an API for node, sensor, target, observation, and log synchronisation, as well as heartbeat transmission. Clients may either send records to be stored in the server database, or request data of a given time range. Depending on the HTTP Accept header, the server returns data in CSV, JSON, JSON Lines or Namelist format. Requires a FastCGI-compatible web server, such as lighttpd(1).
dmbackup: Creates an online backup of a database by either using the SQLite backup API or VACUUM INTO.
dmbeat: Sends short status messages (heartbeats) periodically to a remote dmapi instance.
dmbot: XMPP chat bot that answers to commands from authorised users.
dmdb: Stores observations received from POSIX message queue in a SQLite database.
dmdbctl: A command-line interface to the DMPACK observation database, to read, add, update, or delete nodes, sensors, and targets.
dmdwd: Reads weather reports from the open data API of Deutscher Wetterdienst.
dmexport: Exports beats, nodes, sensors, targets, observations, and logs from database to file, either in CSV, JSON, or JSON Lines format.
dmfeed: Creates an Atom syndication feed in XML format (RFC 4287) from logs of given sensor node and log level. If the feed is served by a web server, clients can subscribe to it by using a feed reader or news aggregator. The program may be executed periodically as a cron job.
dmfs: Reads sensor data from virtual file system, file, or named pipe. The program be used to read values from sensors connected via 1-Wire (OWFS). Observations are forwarded via POSIX message queue and/or written to file.
dmgrc: Creates log messages from GeoCOM return codes.
dmimport: Imports nodes, sensors, targets, observations, and logs from CSV file into database.
dminfo: Prints system and database information as key–value pairs to standard output.
dminit: Creates and initialises SQLite observation, log, and beat databases.
dmlog: A utility program to send log messages from command-line or shell script to the POSIX message queue of a dmlogger process, to be stored in the log database.

dmlogger: Stores logs received from POSIX message queue in a SQLite database.
dmlua: Runs a custom Lua script to process an observation and forward it to the next specified receiver.
dmmb: Sensor control program for Modbus RTU/TCP. Loads the observations to perform from a configuration file and either reads from or writes to Modbus registers.
dmmbctl: A command-line utility to read a value from or write a value to a register of a Modbus RTU/TCP device.
dmpipe: Executes a program as a sub-process connected through an anonymous pipe and forwards the output via POSIX message queue. Optionally, observations are written to file or stdout.
dmplot: Creates line plots of time series read from database, with output to file, terminal, or X11 window. Uses gnuplot(1) internally as plotting backend.
dmrecv: Receives logs or observations from POSIX message queue and writes them to stdout, file, or named pipe.
dmreport: Generates reports in HTML and PDF format, containing plots and optionally log messages.
dmsend: Sends observations or logs from file to a DMPACK application via POSIX message queue.
dmserial: Connects to a TTY/PTY serial port for sensor communication (RS-232/422/485). The program sends requests to a connected sensor to receive responses. The program pre-processes the response data using regular expressions and forwards observations via POSIX message queue.
dmsync: Synchronises nodes, sensors, targets, observations, and log messages between client and dmapi server. Only uni-directional synchronisation from client to server is supported.
dmsystem: Monitors system parameters (free disk space, database sizes, load average, …).
dmuuid: A command-line tool to generate UUIDv4 identifiers (by default in hex format without hyphens).
dmved: Reads status information of a connected Victron Energy MPPT solar charge controller or SmartShunt battery monitor, using the VE.Direct protocol.
dmweb: A CGI-based web user interface for DMPACK database access on client and server. Requires a web server and gnuplot(1).