kgateway

This project focuses on creating clear, approachable, and practical documentation that shows how to integrate kgateway’s AI Gateway (agentgateway) with popular AI tools, developer UIs, and other CNCF ecosystem projects. Its aim is to make kgateway’s documentation more practical, discoverable, and reflective of real ecosystem usage, so users can better understand what’s possible and how kgateway fits into the broader AI and cloud-native landscape. Examples of integrations and tutorials include: 1. Open WebUI / OpenAI Codex / Claude Code: Step-by-step guides showing how to connect agentgateway to interactive UIs for testing, demos, and common integration patterns 2. Demonstrate how agentgateway fits into the CNCF ecosystem by integrating with tools such as: a. Argo Rollouts: Update the Argo Rollout Gateway API guides with the latest agentgateway example config. Create docs using Argo Rollouts with AgentgatewayBackends for LLM providers and MCP servers. b. KServe: Using agentgateway as an ingress for model serving, enabling rate limiting, authentication, and observability c. Knative: Add a guide for setting up kgateway with agentgateway as a custom ingress gateway Expected Outcome: 1. A series of integration guides and tutorials demonstrating how to use kgateway with AI developer tools and CNCF ecosystem projects 2. New examples, improvements, and documentation pages....

This project aims to design and build a suite of automated scale tests for kgateway that would run as part of our builds and releases process. These tests will help us understand how kgateway behaves under a large number of Kubernetes gateway resources and kgateway extensions, and load tests to ensure it performs reliably as usage scales. You'll gain hands-on experience with performance testing, infrastructure automation, and Kubernetes-based systems. Expected Outcome: 1. Design and a scale testing suite 2. Build a scale test suite for kgateway 3. Analyze test results to identify bottlenecks or failure points 4. Write developer-facing documentation 5. Explore Oracle Developer cloud and determine if it is suitable for kgateway’s scale tests.

Kgateway can be used as an “AI Gateway” that allows applying traditional traffic management, security, and resiliency policies when reaching out to LLM providers. It also allows sending a request to another server for processing things like prompt guards or enrichment before we send it off to the LLM providers. An important aspect in any system that has lots of moving parts is observability: the collection of metrics and other information that allow you to identify issues and troubleshoot a live system. This project aims to enable OpenTelemetry support in kgateway’s AI extensions by allowing users to configure a gRPC tracing collector. This feature enables span propagation from the AI extensions extproc server and exporting traces to an OpenTelemetry-compatible backend (e.g., OpenTelemetry Collector, Jaeger, Zipkin, or Datadog). This feature will build on Envoy’s native tracing capabilities and aligns with industry-standard observability practices. This will make it easier to debug, monitor, and optimize traffic flowing to LLM Providers through kgateway. Expected Outcome: 1. Design and implement an API to enable tracing for the kgateway’s AI extproc extension. 2. Extend the AI extensions extproc server with OpenTelemetry tracing. 3. Create end-to-end (e2e) tests to validate configuration and trace propagation. 4. Write documentation for plugin developers and end users.....

This project focuses on enhancing observability in kgateway’s agentgateway integration by adding support for OpenTelemetry-based tracing, exposing Prometheus-compatible metrics, and configuring access logging. It builds on a previous LFX project that introduced tracing support for kgateway’s AI extproc server, extending those capabilities to support span propagation from agentgateway. Expected Outcome: 1. Translate the existing tracing and access logging APIs in kgateway to enable configuring tracing for the kgateway’s agentgateway integration 2. Create end-to-end (e2e) tests to validate configuration and trace propagation 3. Raise issues for any gaps or friction points discovered during testing 4. Write documentation for plugin developers and end users 5. Writing user-facing documentation and blogs on Otel tracing with agentgateway and kgateway integration with OpenTelemetry-compatible backends 6. Gain hands-on experience with AI providers, OpenTelemetry, tracing platforms, MCP, a2a, Kubernetes, and kgate 7. Fun!

This project focuses on adding fault injection support in kgateway, enabling platform operators and developers to test system resiliency under controlled failure scenarios. Fault injection allows teams to proactively identify weaknesses by introducing network latency, service errors, or resource constraints. This project will involve designing a configuration API for specifying fault injection rules, implementing support in the kgateway plugin framework, integrating with Envoy’s native fault injection capabilities, and creating documentation and examples to demonstrate practical use cases in Kubernetes environments. Expected Outcome: 1. Create a design doc outlining the proposed API for fault injection and present at a community meeting 2. Implement the fault injection plugin in kgateway, leveraging Envoy capabilities 3. Develop e2e tests to validate fault injection scenarios 4. Write developer-facing documentation with example configurations 5. Create blogs and tutorials demonstrating how to use Chaos Engineering in kgateway 6. Demo fault injection features during kgateway community meetings