SShortSingh.
Back to feed

How Rust's Type System and a Single Vec<f64> Shrunk an ML Docker Image 400x

0
·3 views

A developer building 'datarust', a Rust-based machine learning library, has detailed how a single flat memory structure called Matrix underpins the entire library's efficiency. The library splits data into two strictly typed structures at the source — Matrix for numeric data and StrMatrix for strings — preventing type mismatches at compile time rather than at runtime. This design means errors like mismatched column shapes are caught during construction, not silently passed downstream to corrupt model training. The approach contrasts with Python tools like pandas and scikit-learn, where type mismatches and ragged arrays can go undetected until deep in the processing pipeline. The result is a binary of just 2.3 MB and a Docker image reduced from roughly 900 MB to 8 MB, a roughly 400-fold reduction.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Why Building Projects Beats Watching Coding Tutorials, According to a Developer

A software developer argues that passive tutorial-watching creates a false sense of learning, leaving coders unable to build independently when the video ends. The author draws on four years of personal experience to explain how long tutorials often compress little practical value while shielding viewers from real problem-solving. Following a creator's pre-planned path, the piece contends, limits critical thinking and stifles original ideas. Instead, the developer advocates starting with any idea, opening a code editor directly, and using AI tools for targeted research rather than step-by-step video guides. The core argument is that making mistakes while building original projects leads to deeper, more durable learning than passive consumption of tutorial content.

0
ProgrammingDEV Community ·

AI-Assisted Profiling Cuts Lean 4 CI Build Time from 41 Minutes to 12

A Lean 4 and mathlib formal verification project reduced its continuous integration build time from 41 minutes to roughly 12 minutes at worst case, with typical pull requests completing in just a few minutes. The improvements came in three rounds, each following a structured problem-hypothesis-verification-fix approach, and in every round the initially suspected cause turned out to be innocent. A kernel axiom audit — a mandatory check ensuring no proofs were bypassed using Lean's 'sorry' escape hatch — was also slashed from over seven minutes to 11 seconds. Notably, nearly all measurement and implementation work across the three rounds was carried out by AI agents, with the human author primarily approving targets and accepting results. The project, a formal verification of a software architecture theory built on over 4,000 declarations, runs its CI pipeline on GitHub Actions.

0
ProgrammingDEV Community ·

Kaggle Mars Pressure Competition's Public Leaderboard Uses Chronological, Not Random, Split

A competitor in a Kaggle challenge to reconstruct missing pressure readings from NASA's MEDA instrument on the Perseverance rover discovered that the public leaderboard uses the first half of test data in time order, not a random sample. The boundary falls after roughly 1.97 million rows, cutting inside sol 253 of the Mars mission timeline. This distinction matters significantly because the test window spans a period where atmospheric pressure drops by over 100 pascals, meaning public and private scores measure different seasonal regimes. The competitor had spent six weeks tuning constants, offsets, and blend weights against the public score, which effectively amounted to extrapolation rather than interpolation once the chronological split was confirmed. The finding reframes the entire model-tuning process, as optimizing against an early-season public score does not reliably predict performance on the late-season private score.

0
ProgrammingDEV Community ·

Key questions to ask before hiring a Shopify developer for your store

Hiring a Shopify developer involves multiple decisions, including whether custom development is even necessary or if apps and theme configuration can meet your needs. Businesses typically require a developer for complex checkout logic, custom integrations with ERP or WMS systems, multi-app conflicts, or performance issues tied to conversion rates. Headless Shopify builds using Hydrogen or Next.js cost significantly more — between £25,000 and £80,000-plus — and take far longer to launch than standard Liquid theme projects, making them unsuitable for most stores below £5 million GMV. Before signing a contract, store owners should ask developers about live portfolio examples, how they handle app conflicts, their handoff process, and their experience with the relevant Shopify plan. Development budgets under £3,000 typically cover basic theme customisation, while custom Liquid theme builds generally fall in the £5,000–£15,000 range.