SShortSingh.
Back to feed

Developer builds 15MB .NET 10 LLM engine that bypasses CUDA toolkits entirely

0
·16 views

A developer has built Glacier.Inference, a local large language model inference engine written in pure C# using .NET 10's Native AOT compilation, producing a self-contained executable of roughly 15MB. The engine bypasses conventional CUDA runtime libraries like cudart64.dll by communicating directly with the base Windows GPU driver via low-level P/Invoke calls, replacing a toolchain that typically exceeds 4.5GB. Glacier.Inference supports NVIDIA, AMD, and Intel GPUs and reads GGUF model files using zero-copy memory mapping with cold load times under 30 milliseconds. A key optimization fuses the token selection step directly on the GPU, transferring just 4 bytes per token instead of the roughly 608KB logit tensor typically sent back to the CPU over PCIe. The project challenges the assumption that high-performance local LLM inference requires C++ runtimes or large native binary dependencies.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

How Lua obfuscation works and what it can and cannot protect

Developers shipping Lua scripts, such as Roblox games or embedded logic, risk exposing source code, game formulas, remote event patterns, and accidentally embedded secrets. The article outlines common obfuscation techniques, including minification, string encryption, control-flow flattening, and bytecode-level transforms, each trading off protection against runtime performance. It stresses that no client-side obfuscation is a guarantee, since determined reverse engineers can eventually recover logic, but it raises the cost of theft from minutes to hours. The author recommends keeping real secrets and trust-critical validation on the server, testing obfuscated output on the executors users actually run, and keeping unobfuscated source under version control. The piece is framed as defense-in-depth against casual copy-paste theft rather than a complete safeguard.

0
ProgrammingDEV Community ·

Setting up Supabase email sign-in and a secure profiles table for iOS apps

A developer walks through a standard iOS backend setup using Supabase, covering email sign-in, per-user data storage, and access control. The recommended approach stores profile data in a public.profiles table linked to the auth user ID, with cascading deletion so no orphaned rows remain. Row Level Security policies restrict each signed-in user to reading and updating only their own row, since the public app key itself provides no protection. Column-level grants further limit client edits to specific fields, keeping sensitive attributes such as billing status writable only by the backend. A database trigger creates the profile atomically at sign-up, avoiding gaps caused by network failures or interrupted app sessions.

0
ProgrammingDEV Community ·

Fan build planner encodes entire game builds in URL fragments with patch-version stamps

A developer of a fan-made planner for the browser survivor game Azeroth Survivors describes storing builds with no accounts or database, placing everything in the URL fragment so that saving and sharing are the same action. Builds are packed into a fixed byte layout covering class, skill slots, banished skills, and talent ranks in 3-bit fields, producing a 32-character base64url string for a 40-talent class. Skills are referenced by index rather than name, so the format carries a version nibble that must be bumped whenever offsets or widths change, and decoding returns null on mismatch. A later format adds a name, a game version stamp, and per-stage plans, letting the planner check whether a shared link's talents, skills, and upgrades still exist after a game patch.

0
ProgrammingDEV Community ·

Comparing subscription offers with a transparent, auditable cost function

A low monthly price can conceal an upfront fee or a longer contract, so comparison tools should expose their assumptions before ranking offers. The article recommends keeping upfront cost, monthly charge, term length, and unconditional credits as separate inputs, and withholding any conditional rebate until its conditions are satisfied. It includes a sample JavaScript function that validates inputs and computes total cost as upfront plus monthly multiplied by months, minus credits. Using illustrative figures, one offer with no upfront fee and $85 monthly over 24 months totals $2,040, while another with $600 upfront and $40 monthly totals $1,560, a $480 difference before taxes or fees. The author stresses showing the formula and time horizon with results, storing currency as integer minor units, and treating estimates as non-binding rather than guaranteed final bills.