Skip to main content

Overview

Use Data Engine to turn device readings into data your application can use. A DataStream identifies the data you want to read or process; an Analyzer derives new values; a Data Stream Bridge forwards selected values to your MQTT broker.

Start with the data you need. For example, a temperature reading can be displayed directly, calibrated before use, or combined with other readings for a more complex calculation.

How Data Flows Through WEDA​

Both device-originated DataStreams and Analyzer output DataStreams can supply downstream features. Choose the path that fits your task:

  • Use the reading as received: Query the DataStream, forward it through a Bridge, or attach it to an Alert Policy.
  • Derive a value first: Configure an Analyzer to read existing DataStreams and publish its result to an output DataStream.
  • Process in stages: Use an output DataStream as the input to another JavaScript Analyzer.

Understand DataStreams​

A DataStream is a named stream of values associated with a device. Each value includes its timestamp. For example, a temperature DataStream identifies which device reading you are querying, while the stream's data type describes the value format.

Keep three things separate:

  • The device identifies the source or owner of the data. One device can have several DataStreams.
  • The DataStream has its own ID, name, data type and source information. Use its ID to select the data for a query, Analyzer or Bridge.
  • The values are the readings received over time. Creating a DataStream defines a destination; a source still needs to supply its values.

How the DataStream is prepared depends on the source:

  • WEDA Node and WEDA SubNode: WEDA synchronizes reported sensor capabilities into system-managed DataStreams.
  • Custom Device: Create the required DataStreams and configure the Data Source and Data Parser to map incoming data to them.
  • Analyzer output: Prepare a user-managed DataStream and bind it as the Analyzer's output.

See DataStreams for discovery, creation, source relationships and value queries.

Process Data with Analyzers​

An Analyzer runs in WEDA Cloud. It reads input DataStreams, applies the configured expression or script, and publishes derived values to output DataStreams. Your application can query or forward those outputs in the same way it uses other DataStreams.

WEDA v1.2 adds JavaScript Analyzers and multi-layer processing to the existing Transfer Function approach. Choose the computation according to the work:

ApproachWhen to use itTemperature example
Transfer FunctionApply a mathematical expression to one input and produce one output.Convert Celsius to Fahrenheit.
JavaScript AnalyzerApply script logic to one to ten inputs and produce one or more outputs.Apply conditional calibration or calculate several derived values.
Multi-layer processingChain JavaScript Analyzers through their input and output DataStreams.Calibrate a reading, then classify the calibrated temperature.

Multi-layer describes the connection between Analyzers. Configure a downstream JavaScript Analyzer to use an upstream Analyzer's output DataStream as its input:

The second Analyzer can publish its own result to another output DataStream. The intermediate DataStream remains available for queries and other consumers. A chain can be at most five Analyzers deep and cannot loop back to an earlier stage.

For multiple inputs, values may arrive at different times. Data Assembly settings control how those inputs are collected into a set before evaluation, including how long to wait for missing inputs. Configure these settings on the Analyzer. A single-input chain does not need a multi-input assembly step.

Note: JavaScript execution limits

JavaScript evaluations have limits on script length, execution time, memory and statement count. Scripts cannot make network requests. Each evaluation starts with fresh script state. Review the runtime limits and input/output rules in the Analyzers guide when preparing your script.

The Analyzers guide also covers Data Assembly policies, enabling an Analyzer, chaining constraints and troubleshooting.

Query and Forward Your Data​

Choose how your application will use the DataStream:

Your taskUseContinue with
Display the most recent reading received within the freshness window, 60 minutes by defaultA latest-value queryDataStreams
Inspect readings over a past intervalA historical-value queryDataStreams
Calculate a summary over a time range, such as average temperatureAn aggregate queryDataStreams
Send selected values to your own system over MQTTA Data Stream BridgeData Stream Bridge
Evaluate values against trigger and recovery conditionsAn Alert PolicyAlert Policies

An aggregate query calculates a result when you request it. An Analyzer publishes derived values to output DataStreams as it processes incoming data. Choose an aggregate query for a time-range summary and an Analyzer when you need a derived stream for further processing or consumption.

Historical and aggregate queries return only recorded values. Before expecting a past time range, check that the stream's recordToDataLake setting is enabled and that the range falls within the data retention policy.