Streaming Data Asset
Project Description
As live data from IoT and sensor sources becomes more common we would like to see the Ocean Market support streaming data. This will require a different product type than static assets with datasets. Some examples of data streams include the public transport and weather data being provisioned by cities like Helsinki’s transport APIs: https://www.hsl.fi/en/hsl/open-data 3
Grant Scope
We intend to write an extension for Ocean Market which will make streaming data listings possible. With live data the asset will be a stream and its listing on Ocean Market will have additional metadata fields. Instead of downloading a dataset a user will access a data stream at a given URL for a specified amount of time.
The pricing in the initial implementation will be a one-time payment, the same as Ocean Market supports now. The price will be keyed to a time limit for the subscription. The limit selection will be defined by the provider (e.g. 1h, 4h, 8h, 1D, 1W). When the time limit is reached the stream will stop and the user will need to purchase a new subscription. (In a future iteration we’d like to implement streaming payments, using Superfluid or LlamaPay, which will run continuously in the background.)
Project Scope
Our intention is to foster a data market that supports multiple types of data and payments, and can be used by individuals or enterprises in any industry with ease.
The below Process Flow Diagram shows how the Ocean Market, which acts as the proxy-host between the source/provider and the consumer for static data, morphs into a facilitator between the two parties when it comes to streaming data. The market enables the dealmaking and chaneling of the payment, while the fulfillment happens in a peer-to-peer fashion between the data stream provider and the consumer.
This is similar to ecommerce sites which include streaming products in their catalog. For example, a customer finds a streaming product, such as an audio book or podcast, and pays for it. Instead of downloading a file from the ecommerce site, the customer directly connects to a stream provided by the publisher.
As shown in the below example, the podcast provider (the data source in our case) deals with the consumer directly for delivery/fulfillment.
In the center is the Ocean Market App with two extensions extending to the right. The upper one, Streaming Data Extension, is the subject of this grant. In a future iteration we would like to implement the Streaming Payment Extension.
As an additional benefit, we’ll also be enabling an integration with Tableland (a web3-SQL app), thereby SQL-izing the DDO and metadata. The idea is to let the Market users access the Tableland instance in order to perform analytics on dataset-level metadata and metrics such as prices.
Final Product
A Data Market that supports multiple types of data and payments. Static datasets and dynamically generated analytic data will be procured with a single transaction and streaming data will be provisioned as a streaming transaction.
Value Add Criteria
1. This project will enhance the functionality and usage of the Ocean Market so that it handles streaming data assets and ultimately (in a future iteration) streaming payments.
2. Users will be able to work with live data from IoT, sensor, and monitor sources.
3. Integration with Tableland (a web3-SQL app: https://docs.tableland.xyz/) will enhance users' analytics capabilities.
This will boost the usage and adoption of the Ocean platform.
