LLM

OmniRoute is a self-hostable AI gateway that brings multiple model providers and coding agents together behind a single OpenAI-compatible endpoint, centralizing routing, quotas, costs, and context compression.

Page Agent is an open-source JavaScript GUI agent that runs inside a webpage. This article explains its DOM-driven design, LLM integration, minimal setup, practical use cases, and the security boundaries required before letting an agent operate a real UI.

The hardest part of maintaining an LLM application is often not connecting a model, but knowing why answer quality changed. Promptfoo brings prompts, models, test cases, assertions, and red-team scans into one CLI and library workflow, so model comparisons, regression tests, and security checks can run locally and in CI/CD. This article builds a practical evaluation process and explains where Promptfoo fits, how to adopt it, and what limitations still require engineering judgment.

AstrBot is an open-source, Python-based multi-platform Agent chatbot and development framework. It brings large language models, multimodality, MCP, knowledge bases, plugins, WebUI, and multiple instant-messaging platforms into one deployable environment. This article examines its architecture, deployment options, security boundaries, and fit for personal assistants, customer service, and team knowledge portals.

Jan integrates local LLMs, cloud models, custom assistants, an OpenAI-compatible API, and MCP into a desktop workbench. This article examines its architecture, build options, adoption pilot, and security boundaries.

CloudWeGo Eino is a Go framework for AI application development that combines models, prompts, RAG components, tools, Chains, Graphs, streaming, and callbacks into testable and observable LLM-agent workflows. Based on the source structure, this article explains its engineering use cases, adoption sequence, and security boundaries.

Connecting an open-source model to an API is only the first step. SGLang focuses on high-performance serving for LLMs and multimodal models, offering an OpenAI-compatible interface, native generation APIs, and multiple deployment paths. This article examines the problems it addresses, how to get started, the boundaries of performance engineering, and which teams should consider trying it first.

DeepEval is an open-source LLM evaluation framework that connects test cases, quality metrics, and thresholds to a familiar pytest and CI workflow. This article examines black-box evaluation, RAG faithfulness, agent trajectories, tool correctness, and rollout governance.

Many teams can call an LLM, yet still get stuck on document parsing, chunking, retrieval, citations, and agent workflows. This article explains how LlamaIndex connects private data to LLM applications, how to start with a verifiable document question-answering flow, and which production limits require careful evaluation.

Microsoft bitnet.cpp brings BitNet b1.58 and other low-bit models to CPU and GPU inference through optimized kernels, model formats, and benchmark tools, opening practical paths for local LLMs, RAG embeddings, and edge AI.

Haystack 3.0 uses components and Pipelines to make RAG, agents, retrieval, tool calls, and generation composable and testable. This article explains the design from the official source code, builds a minimal RAG example, and discusses where the framework fits in production.

LangExtract does more than produce structured LLM output: it maps each extraction to a character span in the source text and provides JSONL plus interactive HTML for review. This article explains its data model, long-document pipeline, provider architecture, and a practical local start.

AnythingLLM is an open-source, local-first AI workspace that brings document Q&A, RAG, AI Agents, multi-user permissions, and multiple model and vector-database options into one self-hosted application. This article examines its architecture, data flow, deployment choices, and privacy boundaries, with a focus on the teams it fits and the engineering details that are easiest to overlook during adoption.

RTK is a CLI proxy written in Rust that filters, groups, truncates and deduplicates the shell output according to the command type before entering the AI coding agent. This article breaks down the scope of its real savings, how installation integrates with agents, and how to import without sacrificing error visibility.

LlamaFactory unifies fine-tuning for 100+ LLMs and VLMs through a CLI, Web UI, and OpenAI-style API. This article walks through data preparation, LoRA/QLoRA training, inference, merging, and deployment.

Cherry Studio uses a unified desktop application to solve the pain point of fragmentation of AI tools - it also supports 300+ AI providers such as OpenAI, Anthropic, Gemini, Ollama, etc., has a built-in MCP server architecture, and is completely open source.

Andrej Karpathy posted a project with only three Python files, but it detonated the entire AI community. Autoresearch allows the AI agent to modify code, train, evaluate, and iterate independently - when you wake up, the model has become stronger.