推理部署 on 黄文卓 | DevOps Engineerhttps://socake.github.io/categories/%E6%8E%A8%E7%90%86%E9%83%A8%E7%BD%B2/Recent content in 推理部署 on 黄文卓 | DevOps EngineerHugo -- gohugo.iozh-CN17691281867@163.com (Wenzhuo Huang)17691281867@163.com (Wenzhuo Huang)© 2026 Wenzhuo HuangSun, 29 Mar 2026 10:45:00 +0800Ray Serve 模型部署实战：Deployment、DAG 编排与弹性伸缩https://socake.github.io/posts/ray-serve-model-deployment/Sun, 29 Mar 2026 10:45:00 +080017691281867@163.com (Wenzhuo Huang)https://socake.github.io/posts/ray-serve-model-deployment/Ray Serve 是被很多团队忽视的模型服务框架。它在复杂 DAG、异构资源、弹性伸缩上的表现远超单纯的 FastAPI。本文讲清它的核心抽象和生产落地。SGLang 结构化生成实战：RadixAttention、约束解码与多轮对话优化https://socake.github.io/posts/sglang-structured-generation/Sat, 14 Mar 2026 16:45:00 +080017691281867@163.com (Wenzhuo Huang)https://socake.github.io/posts/sglang-structured-generation/SGLang 是被低估的 LLM 推理框架，RadixAttention 对多轮对话和 Agent 场景收益巨大。本文讲清 SGLang 的核心机制、前端 DSL、约束解码、部署方式和踩坑。Triton Inference Server 生产部署：模型编排、动态批处理与多框架混部https://socake.github.io/posts/triton-inference-server-production/Wed, 11 Mar 2026 10:00:00 +080017691281867@163.com (Wenzhuo Huang)https://socake.github.io/posts/triton-inference-server-production/把 Triton 从一个陌生的 NVIDIA 推理服务器讲清楚：model repository、backend、动态批处理、ensemble、BLS、Python backend、生产监控和踩坑实录。TensorRT-LLM 推理加速实战：从 engine 编译到 kernel 调优https://socake.github.io/posts/tensorrt-llm-inference/Sat, 07 Mar 2026 14:20:00 +080017691281867@163.com (Wenzhuo Huang)https://socake.github.io/posts/tensorrt-llm-inference/TensorRT-LLM 是 NVIDIA 端到端推理栈的关键一环，这篇把 engine 编译流程、plugin 机制、量化策略、inflight batching、kernel 调优和生产踩坑都梳理清楚。vLLM 多机多卡分布式推理：Tensor Parallel 调优与踩坑实录https://socake.github.io/posts/vllm-multi-node-distributed/Tue, 03 Mar 2026 09:30:00 +080017691281867@163.com (Wenzhuo Huang)https://socake.github.io/posts/vllm-multi-node-distributed/从单机 8 卡讲到多机多卡，把 vLLM 的 TP/PP 拆分、Ray 启动方式、NCCL 调优、PagedAttention 显存核算和常见翻车场景串成一条完整的落地路径。