How to Set Up K3s with NVIDIA GPU Operator on Ubuntu 22.04

·1 views

K3s is a lightweight, single-binary Kubernetes distribution built for edge computing and resource-constrained environments, retaining full Kubernetes functionality with a minimal footprint. A technical guide outlines the step-by-step process of deploying K3s on a GPU-enabled Ubuntu 22.04 server, covering Docker disablement, K3s installation, and kubeconfig setup. Helm is then used to add the NVIDIA chart repository and deploy the GPU Operator into a dedicated namespace, with the bundled driver installation skipped since host drivers are pre-installed. Firewall rules are configured via UFW to allow external cluster access on the Kubernetes API port and NodePort range. The end result is a production-ready, lightweight Kubernetes cluster capable of scheduling and managing GPU workloads through the NVIDIA GPU Operator.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why navigator.clipboard.writeText() Silently Fails When Users Switch Tabs

A subtle browser behavior can cause clipboard copy functions to fail without any visible error when users switch tabs or windows mid-operation. The Async Clipboard API requires the document to have active focus at the exact moment writeText() is called, not merely when the user first clicked. If an awaited operation — such as a fetch request — creates a timing gap between the click and the clipboard call, focus may have shifted, causing the browser to silently reject the promise with a NotAllowedError. Common fixes like loading spinners, try/catch blocks, or retries do not resolve the issue because they fail to address the root cause of lost document focus. The recommended solution is to pass a promise directly into ClipboardItem so the clipboard slot is claimed during the original click gesture, while the data resolves asynchronously afterward.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GPT4All Lets You Run AI Language Models Locally Without Internet Access

GPT4All is a free, open-source desktop application that allows users to run large language models locally on Windows, macOS, and Linux without an internet connection. It supports GGUF-format models from sources like Hugging Face and can also connect to API-based providers such as OpenAI, Groq, and Mistral. The application can be installed on Ubuntu and Debian-based systems via an installation binary or Flatpak, while Windows and macOS users have dedicated installers available. GPT4All also supports retrieval-augmented generation (RAG), enabling users to chat with their own local documents. Additionally, it includes a built-in API server that can be secured and made remotely accessible via HTTPS behind a reverse proxy.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

NVIDIA Dynamo and vLLM Enable Scalable LLM Inference Across Multi-GPU Setups

NVIDIA Dynamo is an open-source inference framework designed to deploy large-scale generative AI models across multi-node, multi-GPU environments with high throughput and low latency. It supports both aggregated serving for single-GPU configurations and disaggregated serving, which separates prompt processing and response generation across different GPUs for independent optimization. The framework integrates with backends such as vLLM, SGLang, and NVIDIA TensorRT-LLM, using etcd for service discovery and NATS for inter-component message passing. A deployment guide covers infrastructure setup, container configuration via Docker Compose, and two serving patterns using vLLM as the inference backend. The setup requires a minimum of one GPU for aggregated serving and four GPUs for disaggregated serving, along with the NVIDIA Container Toolkit and Docker Engine.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Deploy FastAPI with Gunicorn and Nginx on Ubuntu 24.04

A technical guide outlines the steps to deploy a FastAPI application on Ubuntu 24.04 using Gunicorn as the application server and Nginx as a reverse proxy. FastAPI, a modern Python web framework, supports asynchronous programming and ships with built-in interactive API documentation via Swagger UI. The setup uses Gunicorn with a Uvicorn worker class to handle ASGI support, while a systemd unit file manages the Gunicorn process as a persistent background service. Nginx sits in front of the application to handle incoming web traffic and route requests through a Unix socket. The deployment is completed by securing the application with a free SSL certificate, making it accessible over HTTPS on a custom domain.

0 comments Read more at DEV Community