SShortSingh.
Back to feed

Ollama on Jetson Nano: qwen2.5:3b-instruct tops small benchmark for quiz generation

0
·1 views

A developer ran informal benchmarks of multiple large language models via Ollama on a Jetson Nano to find the best fit for a local, free flashcard and quiz-generation app. Due to limited RAM on the device, a swap file was created as a safety net before testing could begin. Models including qwen2.5, llama3.2, and mistral variants were tested across different quantization levels using a 10-question OSI model quiz as the reference. qwen2.5:3b-instruct achieved 100% accuracy across all tested quantizations, while some larger models could not run at all due to GPU memory constraints. The developer plans to switch their app from llama3.2:3b-instruct to qwen2.5:3b-instruct based on these results, acknowledging the test covers only one narrow use case.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Aspiring Developer Joins DEV Community to Pursue Mobile App Development

A new member has introduced themselves to the DEV Community, expressing a strong interest in mobile application development. They are currently focused on learning tools, frameworks, and programming languages used to build modern mobile apps. Their areas of interest include designing smooth user interfaces and understanding how backend data integrates with mobile devices. Their stated long-term goal is to create impactful applications that address real-world problems. They have joined the platform to connect with fellow developers and seek advice from those experienced in the field.

0
ProgrammingDEV Community ·

Mesh LLM on Iroh: How Distributed AI Can Scale Beyond Centralized Servers

A technical deep-dive published on DEV Community explores deploying Large Language Models across decentralized networks using a 'Mesh LLM' architecture built on iroh, a Rust-based peer-to-peer data synchronization platform. Traditional centralized LLM deployments face issues including single points of failure, scalability bottlenecks, and inefficient data movement, which distributed approaches aim to solve. The Mesh LLM paradigm distributes inference, training, and retrieval tasks across interconnected nodes rather than relying on a single server. Iroh enables this by providing content-addressed data storage, peer-to-peer connectivity, and efficient collection syncing across unreliable networks. The article focuses specifically on distributed inference and Retrieval-Augmented Generation, where iroh manages document corpora and model state synchronization across nodes.

0
ProgrammingDEV Community ·

TypeScript 7.0 Launches with Go-Rewritten Compiler, Up to 17x Faster Builds

Microsoft has released TypeScript 7.0 as a stable version, featuring a compiler fully rewritten in Go to leverage native code execution, multithreading, and improved parallelization. The new compiler delivers build speed improvements of 8x to 12x on large projects, with VS Code's build time dropping from 125.7 seconds to 10.6 seconds in benchmarks. Memory usage also improved, falling by 6% to 26% across tested projects. TypeScript 7.0 includes a new LSP-based language server that significantly speeds up editor features such as autocomplete, error diagnostics, and code navigation. The release has been tested for over a year by major companies including Slack, Canva, Figma, and Vercel, with Slack reporting CI type-checking time cut from 7.5 minutes to 1.25 minutes.

0
ProgrammingDEV Community ·

Developer builds open-source SEO platform to replace $140/month tools for under $4

A developer frustrated by rising costs of SEO platforms like Semrush and Ahrefs built a self-hosted alternative called SEO Command Center, which he MIT-licensed and published on GitHub. The tool replicates core features including rank tracking, keyword research, competitor gap analysis, site audits, and local map-pack grids. Instead of a fixed subscription, users connect their own DataForSEO API key and pay only for the data they actually pull, with the developer's own usage costing $3.55 per month. Built entirely with standard-library Python and SQLite, the project requires no external dependencies, no Docker, and runs with three terminal commands. The developer acknowledged key limitations upfront, including no proprietary backlink index and single-property Google Search Console support, and has opened the project to community contributions.