DevOps Engineer Builds Self-Hosted ChatGPT-Style Platform on a Laptop

·4 views

A DevOps engineer built OzAI, a self-hosted AI platform running open-weight models Qwen3 8B and Qwen2.5-Coder 3B on a MacBook Pro M1 Pro with 16 GB of RAM. The setup uses Ollama for serving models, LiteLLM as an OpenAI-compatible gateway with per-user API keys and rate limits, and Open WebUI as a chat interface. Ollama runs natively on macOS rather than in Docker, since containers cannot access the Apple GPU. Benchmarks showed the 3B model generating about 53 tokens per second and the 8B model about 22, which the author considers sufficient for single-user use. The project is the first of a three-part series, with later parts covering observability and a move to GPUs on Amazon EKS.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Verifying RDP certificates and retrieving Windows VM passwords over Google Cloud IAP

A developer tried to follow Google's guidance for checking a Windows VM's RDP certificate against its serial port 1 log on a Windows Server 2025 instance. The serial output contained no certificate details, and the guest attribute holding the thumbprint was blocked by an organization policy, leaving nothing to compare against. The article explains that IAP tunnels carry the RDP session, while password resets are handled separately through metadata and serial port 4, which requires a different IAM role than tunnel access. It describes how a client generates a temporary key pair, the guest agent returns an encrypted password, and resets overwrite any earlier password. The author also notes timing issues and a bug in their own tool that repeatedly reset the admin password.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Shares Movie Explorer React Project Built with API Fetching

A developer has posted on DEV Community about an upcoming project called Movie Explorer. The post centers on a React component that uses useEffect to fetch movie data automatically when it mounts. The fetched data is displayed as a list, showing each movie's original title, overview, release date, and rating. The source text is a fragmentary code snippet, so further details about the project's scope or features are not provided.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Anthropic Files for IPO Amid Questions About AI Bubble Risks

Anthropic, maker of the Claude AI assistant, filed to go public on October 26, reporting over $16 billion in revenue in six months alongside an $80 billion loss. The article compares the current AI boom with the 2000 dot-com crash and the 2008 debt-driven collapse, noting that some features look similar while the underlying revenue appears more real this time. It warns that if investors push AI firms toward profitability, users may first see tighter usage limits, reduced free-tier features, and shorter trials. The author recommends saving prompts and exporting important data from AI tools, and testing a backup AI for a regular task. The piece explicitly states it is not financial advice.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Connected WebSocket Does Not Guarantee Fresh Crypto Prices, Developers Warned

A crypto dashboard can show a green connection indicator and a plausible price while the displayed quote is outdated. The article argues that connection status only reflects transport availability, not whether recent, valid data is arriving for the instrument shown. It recommends tracking connection state, feed activity, and quote freshness as separate signals, and keeping both source event times and local receive times. It also cautions that fixed staleness timeouts need context about how each feed publishes, and that stale prices should be labeled rather than replaced with zero.

0 comments Read more at DEV Community