Space and Time Introduces Python Data Jobs

What it is and Why Space and Time built it

Space and Time built the first and only ZK proof for SQL. This is an incredibly powerful tool that allows a smart contract to retrieve and process data with SQL in a cryptographically proven way.

This solution opens up a wealth of new use cases for blockchain technology. Although SQL is a powerful and near-Turing-complete language, it has its deficiencies in the sense that it doesn’t cover 100% of business use cases. 

For instance, to create a custom business login, developers will eventually run into the need to deploy arbitrary code. Chainlink built an impressive solution to this: Chainlink Functions. Functions allow Javascript to run redundantly on Chainlink nodes, which come to a consensus on the output. Now, smart contracts can access ZK-proven analytics and data processing with Proof of SQL, as well as fast-running scripts with Functions.

Despite the importance of Chainlink Functions, there is still another class of use cases that haven’t been solved in Web3: long-running Python jobs. The businesses and developers working with Space and Time are working with data and data engineers use Python. Because of this, Space and Time knew they had to solve two things. First: enable users to leverage Python to extract data from their existing database, transform it, and load it into Space and Time in the easiest and fastest way possible, without actually writing code. Second: connect Python jobs to smart contracts in a cryptographically guaranteed way. 

This is where the Space and Time Python Data Jobs comes in, now available in beta on the Space and Time Studio. 

How it works

Getting Data into Space and Time

Python Data Jobs makes the process of getting data into Space and Time from any offchain source without ever writing code faster. A few days ago, Space and Time released AI SQL, an OpenAI-powered service that allows a user to write a natural language prompt, converts it into an SQL query, and returns the result. 

The team has now revealed that Houston, the AI chatbot in the Space and Time Studio, can now be used to generate simple ETL (extract, transform, load) scripts to grab data from source from Web2 databases or Web3 decentralized storage platforms, prep it, and load it into Space and Time. 

This AI chatbot creates a script that connects to PostgreSQL (or Snowflake or IPFS, as examples), understands what's in the database, transforms it, creates tables in SxT, and loads one row at a time out of PostgreSQL and into SxT. 

Usually, database migration is a long, expensive, and tedious job that requires Python. However, developers can now do it with natural language in a single pass. 

Getting Data Out of Space and Time

According to the team, Python Data Jobs can also be used to get data out of Space and Time, process it, and send it to a smart contract. Why this problem persists is because Python jobs often run for a long time.

For instance, a developer has a script to calculate the probability that BTC remains above $40k for the rest of the year, that script has to capture data from the markets, process it, and run a Monte Carlo simulation against it in Python, which altogether might take around 20 seconds. 

Furthermore, if the developer is connecting the result to a smart contract, they need to ensure that it’s tamperproof. Consensus-based proving is perfect for fast-running scripts, but it doesn’t work well for a script running that long. 

If the developer is running the computation redundantly across, say, 30 nodes, node 1 might finish the job in 18 seconds, while node 5 finishes in 25, and node 15 finishes in 21. A new architecture is required.

During the Python Data Jobs beta, Space and Time is building to do this with ZK: a ZK proof for Python. This all changes. When developers run a Python Data Job in SxT, the inputs, outputs, and code itself are all hashed to a major chain. The script is only run once, and if the outcome isn’t as expected, the user can request a proof and SxT cryptographically proves what was run. 

Rather than proving it in real-time with redundant computation and consensus, Space and Time runs it once and hash all the metadata to create a tamperproof audit trail to incentivize node operators not to tamper with the execution. 

Here is more information about the ZK solution that Space and Time is building to enhance the real-time security of Python Data Jobs.

What it enables

Seamless database migrations

Using Space and Time’s Python Data Job for database migration is easy. As a developer, simply tell Houston what migration you want to do, give it access to the source database, and Houston uses the already-built prompt-to-SQL framework to retrieve information about the database. 

Here are two example use cases. 

• Example use case 1: Truflation ingests massive volumes of real-time inflation data across dozens of different data feeds (commodities, bond rates, housing, etc.) into storage, then builds aggregations (inflation indexes) to be exposed onchain via oracles. With Python Data Jobs, these large volumes of data can be efficiently processed and prepped for aggregations. 

• Example use case 2: dClimate periodically ETLs weather data from multiple weather feeds and loads this data into IPFS. Python Data Jobs can streamline this process by automating the extraction and transformation of the weather data.

Complex Calculations for DeFi

Another major benefit of Python Data Jobs is that it enables developers to integrate sophisticated financial models, like those used for predicting price movements or assessing risk factors, into smart contracts with optimistic security.

This feature enables DeFi protocols to leverage more sophisticated business logic beyond what Proof of SQL enables. Here are two example use cases. 

• Example use case 1: dYdX executes calculations for perpetual options/futures pricing offchain because they require historical pricing input data and complex compute that cannot be executed by smart contracts onchain. Python Data Jobs allows these calculations to be done in a tamperproof way.

• Example use case 2: 3Commas executes offchain machine learning models for DeFi/CeFi decisioning (swaps, futures, bot trades, etc.) in a centralized compute container environment. Python Data Jobs provides a Web3-native alternative.

Get Started

Developers can start building Python Data Jobs with Houston on the Space and Time Studio. To celebrate the beta release of their new product, Space and Time is providing Python Data Jobs for free to all users for one month.