TimelyLLM: 時間制約のあるロボットアプリケーション向け分割型LLMサービングシステム
TimelyLLM: Segmented LLM Serving System for Time-sensitive Robotic Applications
ロボット制御向けに、計画生成と実行の冗長性を活かした分割生成とスケジューリングで、時間制約のある複数ロボットのLLMリクエストを効率的に処理するサービングシステムを提案。
著者: Neiwen Ling, Guojun Chen, Lin Zhong
分類: cs.RO, cs.DC, cs.LG
原文アブストラクト
Large Language Models (LLMs) such as GPT-4 and Llama3 can already comprehend complex commands and process diverse tasks. This advancement facilitates their application in controlling drones and robots for various tasks. However, existing LLM serving systems typically employ a first-come, first-served (FCFS) batching mechanism, which fails to address the time-sensitive requirements of robotic applications. To address it, this paper proposes a new system named TimelyLLM serving multiple robotic agents with time-sensitive requests. TimelyLLM introduces novel mechanisms of segmented generation and scheduling that optimally leverage redundancy between robot plan generation and execution phases. We report an implementation of TimelyLLM on a widely-used LLM serving framework and evaluate it on a range of robotic applications. Our evaluation shows that TimelyLLM improves the time utility up to 1.97x, and reduces the overall waiting time by 84%.
