Skip to content

function call 微调后部署问题 #5518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 task done
pugnazhaotianqi opened this issue Sep 23, 2024 · 1 comment · Fixed by #6367
Closed
1 task done

function call 微调后部署问题 #5518

pugnazhaotianqi opened this issue Sep 23, 2024 · 1 comment · Fixed by #6367
Labels
solved This problem has been already solved

Comments

@pugnazhaotianqi
Copy link

Reminder

  • I have read the README and searched the existing issues.

System Info

  • llamafactory version: 0.8.4.dev0
  • Platform: Linux-6.8.0-40-generic-x86_64-with-glibc2.35
  • Python version: 3.10.12
  • PyTorch version: 2.3.0a0+ebedce2 (GPU)
  • Transformers version: 4.43.4
  • Datasets version: 2.20.0
  • Accelerate version: 0.32.0
  • PEFT version: 0.12.0
  • TRL version: 0.9.6
  • GPU type: NVIDIA H20
  • DeepSpeed version: 0.14.5

Reproduction

使用glaive fuction call数据集微调llama3.1后使用vllm部署调用tools时生成的prompt和原始llama3.1 8b的prompt不一致，tools不会正确组装到system prompt中

Expected behavior

No response

Others

No response
@github-actions github-actions bot added the pending This problem is yet to be addressed label Sep 23, 2024
@hiyouga hiyouga mentioned this issue Dec 17, 2024
Merged
2 tasks
@hiyouga hiyouga added solved This problem has been already solved and removed pending This problem is yet to be addressed labels Dec 17, 2024
@hiyouga
Copy link
Owner

hiyouga commented Dec 17, 2024

fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
solved This problem has been already solved
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[model&template] add llama3.3 & support llama3 tool prompt hiyouga/LLaMA-Factory
2 participants
@hiyouga @pugnazhaotianqi