LLM Ops

Last updated Nov 14, 2024 Edit Source

See ML Ops, LLMs

• LLMOps - DeepLearning.AI

• GitHub - AgentOps-AI/agentops
  • Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen
  • AgentOps

• GitHub - vllm-project/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs
  • https://docs.vllm.ai/en/stable/index.html