Fintech Engineer Builds In-House LLM Gateway to Avoid Handing Out AWS Credentials

·1 views

A fintech infrastructure engineer faced a common dilemma when non-technical teams requested access to AWS Bedrock: sharing IAM credentials or a single shared key both posed serious security and cost-tracking risks. Distributing individual AWS keys risks credentials spreading across .env files and CI variables, while a shared key makes it impossible to attribute usage or isolate a runaway process draining the monthly budget. Rather than routing sensitive prompts through third-party hosted gateways like LiteLLM or Portkey, the engineer chose to keep all traffic within the company's own network perimeter. The solution was a self-hosted LLM gateway that issues internal proxy tokens to teams, while the real AWS credentials remain exclusively within the gateway process itself. The design allows per-key model restrictions and usage tracking, giving finance teams an auditable record without exposing cloud credentials to any downstream service or user.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

CALEC Seeks Volunteer Full-Stack Developers for Education Platform Launch

CALEC, a mission-driven organization focused on language education and bilingual learning, is recruiting volunteer Full-Stack Developers to support an ongoing digital project. A current volunteer developer shared the opportunity on DEV Community, noting that Phase 1 of the project is targeted for release before 2027. Candidates are expected to have a background in Computer Science, some full-stack experience or a strong project portfolio, and familiarity with GitHub and Git workflows. The role requires a commitment of at least 10 hours per week and is fully remote. Those interested should be self-motivated, comfortable in a fast-paced environment, and passionate about expanding access to education globally.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why a Boring Deployment Is Actually the Mark of Good Engineering

A software engineer recently deployed a notification scheduler refactor that went entirely without incident, prompting reflection on what made it so uneventful. The engineer noted that pre-deployment anxiety typically signals unresolved questions carried forward from earlier development phases, not genuine deploy-time risk. In this case, testing had already uncovered and resolved a timing-related failure before the release, meaning the deployment itself had no open questions left to answer. A particular concern was an external notification provider that silently drops malformed payloads without throwing errors, making a clean dashboard indistinguishable from a hidden failure. The engineer concluded that the quiet deploy was not luck but the result of evidence gathered earlier, including documented rollback triggers designed specifically for silent failures.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why optimizing AI token count alone won't reduce your coding agent costs

A developer analysis on DEV Community argues that measuring AI cost by total token count is misleading, since token type, model choice, and number of retries all affect the true cost of completing a task. A 100,000-token request can cost less than a 20,000-token one depending on how much context is reused versus newly processed. Choosing a cheaper model that requires multiple attempts may ultimately cost more than a pricier model that solves the problem on the first try. The article outlines seven strategies, including protecting reusable context, controlling new inputs, and loading knowledge on demand, all aimed at reducing cost per completed task. The core argument is that the right metric is not tokens spent but the total expenditure needed to reach a correct result.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How a Digital Provenance System Helps Vineyards Defend Quality and Command Premium Prices

A concept called the Supply Chain Guardian proposes embedding real-time field data — including Brix, pH, fruit temperature, and GPS-linked thermal logs — into a digital chain of custody that travels with each harvest batch. The system generates a timestamped Provenance Certificate, functioning as an automated notary record from vine to buyer. In a hypothetical dispute scenario, a buyer claiming fruit arrived too warm was countered by thermal transport logs showing the cold chain remained unbroken until offload at the buyer's dock. Because the data is stored locally and privately, growers can selectively share verified quality information with buyers on their own terms. The article argues that this level of traceable provenance can shift a load from bulk commodity pricing to a premium contract, potentially adding $200 to $400 per ton in revenue.

0 comments Read more at DEV Community