Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocation

arXiv (Cornell University)

Abstract

Although current Large Language Models (LLMs) exhibit impressive capabilities, performing complex real-world tasks still requires tool learning. Mainstream methods, such as CoT/ReAct, rely on step-by-step tool invocation to interact with external environments, but they are limited in perceptual scope and lack adequate task-planning capability. To address these limitations, other studies introduce the first Search-based Decision Tree (DFSDT), which still suffers from the high computational cost. In this paper, we introduce a novel parallel tool invocation paradigm, DTA-Llama (Divide-Then-Aggregate Llama). First, we transform traditional tree-based tool search paths into Directed Acyclic Graph (DAG) structure, generating a high-quality parallel tool invocation dataset. The DTA-Llama is then trained on the dataset to learn to iteratively divide the current task into several parallel tool invocation sub-tasks and aggregate the invocation results to decide the next actions. Furthermore, we introduce an efficient inference framework inspired by the Process/Threads mechanism when applying the DTA-Llama to practical tasks. Experimental results show that our approach substantially enhances task performance while reducing token consumption and inference time. Llama2-7B, using our method, is comparable to the official parallel function calling method of GPT-3.5. The relevant code, dataset, and model weights are available at https://corn0205.github.io/

Authors 4

1. Zhaochun Ren Aachen

   Leiden University

   Affiliation as printed

   Leiden University , Leiden , The Netherlands

2. Shuaiqiang Wang

   Baidu (China)

   Affiliation as printed

   Baidu Inc. , Beijing , China

3. Lisong Yan

   Baidu (China)

   Affiliation as printed

   Baidu Inc. , Beijing , China

4. Dawei Yin

   Baidu (China)

   Affiliation as printed

   Baidu Inc. , Beijing , China