An AI Consumer Group is the AI Gateway entity that represents a collection of AI Consumers grouped for the purpose of applying shared AI Policies and access controls.
By grouping AI Consumers together, you eliminate the need to manage AI Policies and access controls individually, providing a scalable, efficient approach to AI governance. With AI Consumer Groups, you can scope AI Policies to specifically defined groups, making configurations and customizations more flexible.
For example, you could define three groups (Bronze, Gold, and Enterprise) and attach an AI Rate Limiting Advanced policy to each with different token quotas and cost budgets. Without AI Consumer Groups, you would attach a separate AI Rate Limiting Advanced policy to each individual AI Consumer. In production, that could be thousands of individual policy attachments instead of three group-level ones.
flowchart LR
A((AI Consumers 1-5))
B("<b>AI Consumer Group Gold</b><br/>AI Consumer 1, AI Consumer 2, AI Consumer 5")
C("<b>AI Consumer Group Bronze</b><br/>AI Consumer 3, AI Consumer 4")
D["AI Rate Limiting Advanced<br/>1M tokens/hour<br/>AND<br/>$100/hour budget"]
E["AI Rate Limiting Advanced<br/>100K tokens/hour<br/>AND<br/>$10/hour budget"]
F("<b>AI Model</b><br/>GPT-4")
H["OpenAI<br/>Service"]
A--> B & C
subgraph id1 ["AI Gateway"]
direction LR
B --> D --> F
C --> E --> F
end
F --> H