Rust vs Python Apache Iceberg Clients: Rust Up to 4x Faster on Small Calls

·2 views

A developer benchmarked two Apache Iceberg REST catalog clients — the Rust client (iceberg-catalog-rest 0.10.1) and the Python client (pyiceberg 0.12.0) — across six shared read operations on local and cloud catalogs. The Rust client was 1.90x to 4.29x faster on lightweight namespace operations, while the gap narrowed to 1.14x–1.38x on heavier table-level calls where server processing dominates. Testing was conducted against Apache Polaris running locally in Docker, as well as Google BigLake and Microsoft OneLake over the internet. The benchmark measured speed only and did not validate correctness; notably, the Python client supports 21 of 25 tested catalog endpoints versus just 13 for the Rust client. The author cautions that a faster client lacking required operations is still the wrong choice, and that timing results varied significantly between runs.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Rust Iceberg Client Outpaces Python by Up to 4x in REST Catalog Benchmark

A developer benchmarked two Apache Iceberg REST catalog clients — the Rust client (iceberg-catalog-rest 0.10.1) and the Python client (pyiceberg 0.12.0) — across identical operations on the same tables. Tests were run against a local Apache Polaris instance in Docker, as well as Google BigLake and Microsoft OneLake over the internet. On four lightweight read operations, the Rust client was 1.90x to 4.29x faster than pyiceberg, while the gap narrowed to 1.14x–1.38x on heavier table-level calls where the server does more processing. However, the Rust client supports only 13 of 25 tested catalog endpoints, compared to 21 for pyiceberg, making feature coverage a key consideration alongside speed. The benchmark focused solely on performance and did not validate the correctness of responses from either client.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Vite+ Chapter 3: How to Build and Run a Real Project Using the vp CLI

Vite+ is a unified JavaScript toolchain that bundles tools like Vite, Vitest, Oxlint, and Rolldown under a single command-line interface called 'vp'. Developers can install it via a one-line script on any operating system and immediately scaffold new projects using built-in templates for monorepos, applications, or libraries. The 'vp dev' command launches Vite's development server with hot module replacement, while 'vp install' handles package management regardless of whether the project uses npm, pnpm, yarn, or bun. Rather than running separate commands for formatting, linting, and type checking, developers can consolidate all three into a single 'vp check' command. The platform is designed to reduce workflow fragmentation across large teams managing multiple repositories and CI pipelines, without forcing teams to abandon their existing underlying tools.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How proving a bug's root cause beat guessing in a CameraX Android rotation fix

A developer debugging an Android barcode scanner discovered that rotating a tablet 180° left the camera preview intact but caused the image analyzer to receive upside-down frames. Initial investigation by an AI coding agent suggested waiting for a display rotation value to update before rebinding the camera, but the developer paused and demanded evidence of the true cause instead. Structured logging revealed that while the display correctly updated to ROTATION_270, ImageAnalysis.targetRotation remained stuck at ROTATION_90, leaving ImageProxy.rotationDegrees at zero. With the root cause confirmed, the fix was straightforward: use an OrientationEventListener to update both Preview and ImageAnalysis target rotations together, eliminating any need for delays or camera rebinding. The case highlights how demanding measurable proof before applying a fix can lead to simpler, more reliable solutions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Alleged Leaked Document Claims NSA Allocated $3.9 Billion for AI Surveillance

A document identified as 'NSA-AI-2023-B' and dated 2023 has circulated online, purportedly revealing a $3.9 billion NSA budget allocated to artificial intelligence development, including $2.4 billion for large language model training and $1.5 billion for GPU infrastructure. The document was published on DocumentLeaks.org and is claimed to have been verified by The Intercept and CyberScoop, though independent confirmation of those verifications has not been established. According to the alleged report, the AI models described could combine call metadata, emails, and social media activity to build real-time behavioral profiles without judicial oversight. The claims have circulated rapidly across security forums and media outlets, raising concerns among privacy advocates about warrantless surveillance and the concentration of computing power within state agencies. The NSA has not publicly commented on the document's authenticity.

0 comments Read more at DEV Community