Devices, developers, applications, and services produce and utilize enormous amounts of JSON data every day. A portion of this data consists of time-stamped events or metrics that are a perfect match for storing and analyzing in InfluxDB. To help developers build the applications of the future, InfluxDB provides several ways to get JSON data into InfluxDB easily.
This blog post demonstrates how to take JSON data, translate it into line protocol, and send it to InfluxDB using the InfluxDB Client libraries.
What are JSON and line protocol? Link to heading
Below is an example of a fictional JSON data point from a weather sensor containing a variety of sensor readings:
To get this data into InfluxDB, the following tools show how to translate these rows into line protocol. Line protocol consists of the following items:
- Measurement name: the name data to store and later query
- Fields: The actual data that will be queried
- Tags: optional string fields that are used to index and help filter data
- Timestamp: optional; if not provided, the current timestamp is used
Using the JSON example above, the goal state is to translate the data into the following line format:
Client Libraries Link to heading
The client libraries provide developers with the tools they need to quickly send read and transformed data into InfluxDB. This way developers can work in the specific language they are most comfortable with or with what the problem space requires. And at the same time have a direct method to inject data into InfluxDB.
Python Link to heading
The Python language and ecosystem have demonstrated a phenomenal way to parse and clean data. Users build Python-based tooling to manipulate data in many different use cases, from IoT devices to enterprise applications. The simple syntax and powerful libraries make it a great first option for getting data into InfluxDB. We have a video series introducing the Python client library here.
The following example demonstrates reading the weather JSON data as a file into a Python dictionary, generating a data point, and sending that data point to InfluxDB:
This example uses the
Point helper. This helper allows developers to quickly
and easily generate data in line protocol with the provided helper functions.
This creates the weather data point, then sets a single tag based on the
station name, uses the data’s timestamp as the timestamp, and then takes all
the sensor readings and makes them fields.
Using the Point helper is one of many ways to write data. Developers could have data to send in the form of a Python Dictionary that specifies the metric name, tags, fields, and timestamp or generates a simple string. For more examples of these, check out the Deep Dive into the WriteAPI blog post.
Go established itself as a high-performance, readable, and efficient language. While it gained initial popularity from server-side, backend development, Go is used across CLI applications, IoT devices, and scientific applications.
In this example, a
Weather struct is created to unmarshal the Weather station
data easily. Then, use the
NewPoint helper to create the data point and,
finally, send it to InfluxDB.
This example reads the JSON file but then unmarshals the data and parses the timestamp. This occurs using the internal parse method along with the format using the Go standard time to explain the format.
NewPoint helper, in this case, allows the user to quickly and easily
specify the various parts of line protocol sent to InfluxDB with the
WritePoint() function. Developers could also write a string of line protocol
This example is a Node.js script that, similar to the other examples, reads in a file, creates a Point, and sends it to InfluxDB. You could adapt this to meet many other scenarios:
Get your JSON data into InfluxDB today Link to heading
This post shows how quick, easy, and flexible the InfluxDB client libraries are. While the above only demonstrates JSON data, it starts to illustrate the great power developers have when sending data to InfluxDB. Combined with the other APIs, developers have even more options and potential.