TSDF (Time Series DataFrame) – A data storage architecture for scalable processing of heterogeneous and geospatial time series

TSDF (Time Series DataFrame) - A data storage architecture for scalable processing of heterogeneous and geospatial time series

Category

Contribute

Institutions

EPFL

Data type

Geospatial time series

Field

Environmental Engineering

Researchers

Martí Bosch

Abstract

Geospatial time series data are pivotal in environmental sciences, from meteorological stations to mobile sensors. However, the absence of standardized practices due to data heterogeneity and irregularity hinders their processing and storage. This proposal introduces TSDF, a data specification addressing these issues. It offers a flexible framework for accessing, processing, storing, and sharing geospatial time series data. TSDF utilizes Apache Parquet for a binary format, enabling hierarchical structuring of measurements. A Python package will be developed for easy data loading into pandas data frames, simplifying operations like filtering, reducing, and combining data. This work streamlines geospatial time series data processing, enhancing interoperability, reusability, and collaboration.

Scroll to Top