Self-hosted AI for small businesses: hardware needs, upkeep, and suitable tasks

·10 views

Running AI models on a company's own hardware means the model operates on a machine the business owns or controls rather than on a provider's servers. The main limit is graphics card memory, since an 8 billion parameter model needs about 4.9GB and a 70 billion parameter model about 43GB. This setup suits recurring overnight jobs and sensitive records, but it requires ongoing updates, security, monitoring, and hardware ownership. For that reason, most small service firms are advised to begin with cloud-based tools.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Using GBase 8s onstat Diagnostics to Guide Evidence-Based Performance Tuning

A DEV Community article argues that database performance tuning should start from measurable evidence rather than observed symptoms. For GBase 8s environments, it describes using onstat commands such as onstat -g ses, onstat -g sql, and onstat -g act to inspect sessions, SQL activity, and workload behavior, with command availability to be confirmed per release. The piece recommends capturing baselines during normal and degraded periods, mapping symptoms like slow queries, blocking, or timeouts to relevant evidence, and avoiding high CPU or session counts as root causes on their own. It also suggests recording metrics such as response time, throughput, and storage latency, and following a cycle of hypothesis, single-factor change, and measurement, ideally in a representative test environment.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

NVIDIA Research Argues Small Language Models Suit Most AI Agent Tasks

NVIDIA Research published a position paper arguing that models under 10 billion parameters are sufficient, better suited, and far cheaper for most calls made by AI agents. The paper claims serving a 7-billion-parameter model can be 10 to 30 times cheaper than a 70-to-175-billion-parameter model, and that specializing a small model takes only a few GPU-hours. An independent 2026 DFKI study found the largest model tested used about 32 times the energy of a small one for roughly a 2-point accuracy gain on a math benchmark. The same study showed smaller models sometimes consumed more energy overall because they produced longer outputs in a substantial share of comparisons. Analyst data cited in the article indicate infrastructure limits are a major barrier to AI deployment for many organizations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Three reusable developer prompts for PR review, changelogs and test scaffolding

A reusable prompt can be pasted into a fresh chat with new material and still return useful output without edits. The article argues this depends on settling one key decision in advance and writing it into the prompt. It offers three examples covering common sprint tasks: pull request review, turning commits into changelogs, and scaffolding pytest tests. Each example is tied to a different decision, such as a defined severity scale for reviews or a named target reader for changelogs. Naming the reader, the article notes, lets one line change produce a separate maintainer-focused version of the same prompt.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Styx wires seven coding agent CLIs into one app via four integration methods

The open-source Styx app runs seven coding agent command-line tools plus a plain shell on the same projects. Its developers found there is no single way to control an agent from another program, so they use four approaches: streaming JSON, a dedicated app server, the Agent Client Protocol, and an embedded terminal. Each method has trade-offs, such as whether mid-turn messages can be sent or whether approvals can be answered from outside the agent's own interface. A small router selects the right method per agent, so the rest of the app sees a uniform session type. The team also tests with fake CLI binaries to avoid contacting real providers or accounts.

0 comments Read more at DEV Community