Affon: a TypeScript runtime for training and running machine learning models

The article explores how much machine learning work can stay within TypeScript, from calling remote model services to running local inference and training custom models. It introduces Affon, an experimental runtime for numerical computing and ML, whose decoder language model app covers attention, embeddings, batching, training, evaluation, checkpoints, and text generation. A saved WikiText-2 run of 12 epochs and 52,128 steps ended with 4.55 training loss and 5.09 validation loss, though these figures are not compared against other frameworks. Inference examples cover text, image, and audio, including Apertus 8B on CUDA and Whisper speech transcription on Metal, run on separate machines. The author also describes a Program Inspector for examining model structure and a CPU-based housing price example, while noting that CPU support is the broadest and Metal and CUDA depend on workload specifics.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in