SShortSingh.
Back to feed

RunAnywhere open-sources TrnRT, an LLM inference engine for AWS Trainium

0
·1 views

Startup RunAnywhere, a Y Combinator W26 company, has released TrnRT (Trainium RunTime) as open source, an inference engine built for AWS Trainium chips. The engine currently runs the GLM-5.3 flash model in FP8 precision on a single trn2.48xlarge instance, serving two model instances with a cache-aware router. The team says it reached 3.57 million tokens per minute on SemiAnalysis's AgentX benchmarks and used custom NKI kernels to speed up decoding. The company began the project because memory on Trainium is cheaper than on comparable Nvidia H200 setups, and it found that AWS's own inference frameworks lacked support for the model. The project is open to contributions, and the team plans to add more models.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingHacker News ·

Report Examines Israeli Government Pressure on Wikipedia

A report published by Tech for Palestine addresses the relationship between Israel and Wikipedia. The source text provided is limited to a headline, a link to the report, and a Hacker News discussion thread with 69 points and one comment. The full details of the report's findings, claims, and any specific actions are not available in the material provided. Accordingly, this summary cannot verify the substance of the allegations or responses involved.

0
ProgrammingHacker News ·

Web Browser Engineering: A Resource on Building Browsers

A resource titled Web Browser Engineering has been shared on Hacker News, linking to browser.engineering. The item has received 15 points, and the discussion thread currently shows no comments. The original text was not provided, so the specific contents, authors, and scope of the material cannot be confirmed here. Please supply the article text or a fuller description so a complete summary can be written.

0
ProgrammingHacker News ·

jQuery 4.1.0 Released on GitHub

The jQuery project has published version 4.1.0 as a tagged release on its GitHub repository. The announcement was shared on Hacker News, where it received a small amount of attention. The source material provided does not describe the specific changes, fixes, or new features included in this version. Readers interested in the details would need to consult the official release notes linked from the GitHub page.

0
ProgrammingDEV Community ·

Reading car fault codes in a browser with a cheap ELM327 adapter and Web APIs

A developer explains how a low-cost ELM327 OBD-II adapter, Chrome, and browser APIs such as Web Serial and Web Bluetooth can read engine fault codes without installing software. The write-up focuses less on the protocol itself than on the quirks of real adapters and browsers, which take up most of the code. The author, who built the diagnostic site diagnost.net, shares snippets from its client-side code. Support is limited to Chromium-based browsers on desktop and Android, since Safari and Firefox lack these APIs and iOS blocks Bluetooth and USB access for all browsers. The post also covers browser user-gesture requirements for opening device pickers and the fact that serial ports accept any baud rate, so adapters must be identified by sending a test command.

RunAnywhere open-sources TrnRT, an LLM inference engine for AWS Trainium · ShortSingh