Journal article · 2025
Optimization Techniques for SQL+ML Queries: A Performance Analysis of Real-Time Feature Computation in OpenMLDB
International Journal of Database Management Systems (IJDMS), vol. 17 (3/4/5) ·
Abstract
This study tunes SQL+ML query execution in OpenMLDB, an open-source database that serves both offline and online feature computation. Using synthetic data in Docker to reproduce production batches and parallel requests, it reaches about 12,500 queries per second at under one millisecond of latency — roughly 23 times SparkSQL and ClickHouse and 3.57 times PostgreSQL and MySQL. Query-plan optimization, caching and parallel processing account for about 35%, 25% and 20% of the gain. The results suit time-sensitive ML applications such as fraud detection and recommendation, and argue for SQL optimization designed for ML workloads.
Department of Software Engineering, Nankai University, Tianjin, and Department of Control Science and Engineering, Tianjin University. Listed on arXiv under Databases (cs.DB).
What we measured
- Throughput and latency of real-time feature computation under production-like batch and concurrent workloads.
- The contribution of each optimisation: query-plan rewriting, caching and parallel execution.
- OpenMLDB against SparkSQL, ClickHouse, PostgreSQL and MySQL on the same queries and data.