Uploading time series data
The Certify API allows you to upload time series data for your carbon removal projects. Time series data consists of sensor measurements collected over time, such as temperature, pressure, flow rates, and chemical concentrations. This data is critical for monitoring and verifying carbon removal operations.
Time series data is different from measurement samples. While measurement samples represent discrete laboratory measurements at specific points in time, time series data represents continuous or regularly sampled sensor readings over extended periods.
Time series data can currently be associated with either a Direct Air Capture (DAC) capture facility or a DAC storage location (saline aquifer).
Upload Process Overview
Uploading time series data involves several steps:
Create sensors
First call the POST Sensor
endpoint to create sensor records that define the metadata about the sensor, including what physical parameter it measures, its units, and the facility or storage location it is linked to.
Get upload URL
Get a signed upload URL by creating a file upload resource using the POST File Uploads
endpoint.
Upload file to cloud storage
Use the signed URL to upload your parquet file containing time series data to cloud storage. The content-type header in the PUT request to cloud storage must match the file type supplied to the file upload resource.
Submit for processing
Create a data upload submission with the POST Data upload submission
. Isometric will process the files asynchronously.
Monitor processing
The GET Data upload submissions
endpoint allows you to monitor the status of your data upload submissions. Any errors in processing will be reported here.
Measurement Properties
A measurement property consists of:
- Quantity Kind: The physical parameter being measured (e.g., temperature, mass flow rate)
- Qualifier (optional): Additional specification for the measurement (e.g., CO2 mass fraction)
The allowed measurement properties depend on the type of facility or storage location your sensor is linked to.
DAC Capture Facility
DAC Capture Facility measurement properties
DAC Capture Facility measurement properties
Property | Quantity Kind | Qualifier | Example Units |
---|---|---|---|
ELECTRIC_CURRENT | ELECTRIC_CURRENT | — | amp |
MASS_DENSITY | MASS_DENSITY | — | kg / m^3 |
MASS_FLOW_RATE | MASS_FLOW_RATE | — | tonne / hour |
MASS_FRACTION (COMPOUND_CO2) | MASS_FRACTION | COMPOUND_CO2 | mg / kg |
MASS_FRACTION (COMPOUND_H2O) | MASS_FRACTION | COMPOUND_H2O | mg / kg |
MASS_FRACTION (COMPOUND_H2S) | MASS_FRACTION | COMPOUND_H2S | mg / kg |
MASS_FRACTION (ELEMENT_O) | MASS_FRACTION | ELEMENT_O | mg / kg |
PRESSURE | PRESSURE | — | bar |
TEMPERATURE | TEMPERATURE | — | degC |
VOLTAGE | VOLTAGE | — | volt |
VOLUME_FLOW_RATE | VOLUME_FLOW_RATE | — | m^3 / hour |
DAC Storage Location (Saline Aquifer)
DAC Storage Location (Saline Aquifer) measurement properties
DAC Storage Location (Saline Aquifer) measurement properties
Property | Quantity Kind | Qualifier | Example Units |
---|---|---|---|
MASS_DENSITY | MASS_DENSITY | — | kg / m^3 |
MASS_FLOW_RATE | MASS_FLOW_RATE | — | tonne / hour |
MASS_FRACTION (COMPOUND_CO2) | MASS_FRACTION | COMPOUND_CO2 | mg / kg |
MASS_FRACTION (COMPOUND_H2O) | MASS_FRACTION | COMPOUND_H2O | mg / kg |
MASS_FRACTION (COMPOUND_H2S) | MASS_FRACTION | COMPOUND_H2S | mg / kg |
MASS_FRACTION (ELEMENT_O) | MASS_FRACTION | ELEMENT_O | mg / kg |
PRESSURE | PRESSURE | — | bar |
TEMPERATURE | TEMPERATURE | — | degC |
VOLUME_FLOW_RATE | VOLUME_FLOW_RATE | — | m^3 / hour |
Data Format Requirements
Parquet file structure
We currently support upload of time series as Parquet files.
The Parquet file must contain the following columns:
start_timestamp
: A timestamp[ns] column representing the start timestamp of the measurement in UTC.end_timestamp
: A timestamp[ns] column representing the end timestamp of the measurement in UTC.sensor_reference
: A string column containing the reference of the sensor that recorded the measurement.min_value
: A double column containing the minimum value recorded by the sensor during the measurement period.max_value
: A double column containing the maximum value recorded by the sensor during the measurement period.mean_value
: A double column containing the mean value recorded by the sensor during the measurement period.count_value
: An integer column indicating how many measurements were taken during this period.stddev_value
: A double column indicating the standard deviation of the measurements taken during this period.
We will ignore any extra columns in the Parquet file that are not listed above.
File Size and Performance
- Maximum file size: 100 MB per upload.
- Processing time: Processing a file upload can take up to an hour.