Skip to content

Qwen2Moe zero3卡住的问题，已找到原因 #6669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 task done
MrLittleB opened this issue Jan 16, 2025 · 2 comments · Fixed by #6684
Closed
1 task done

Qwen2Moe zero3卡住的问题，已找到原因 #6669

MrLittleB opened this issue Jan 16, 2025 · 2 comments · Fixed by #6684
Labels
solved This problem has been already solved

Comments

@MrLittleB
Copy link

Reminder

  • I have read the above rules and searched the existing issues.

System Info

Reproduction

src/llamafactory/model/model_utils/moe.py

Others

No response
@MrLittleB MrLittleB added bug Something isn't working pending This problem is yet to be addressed labels Jan 16, 2025
@MrLittleB
Copy link
Author

src/llamafactory/model/model_utils/moe.py文件第64行 if model_type == "qwen2moe":
from transformers.models.qwen2_moe.modeling_qwen2_moe import Qwen2MoeSparseMoeBlock

qwen2moe改为qwen2_moe

hiyouga added a commit that referenced this issue Jan 17, 2025
@hiyouga
fix #6669
61ff95a
@hiyouga hiyouga mentioned this issue Jan 17, 2025
Merged
2 tasks
@hiyouga
Copy link
Owner

hiyouga commented Jan 17, 2025

fixed, thanks for pointing it out

@hiyouga hiyouga added solved This problem has been already solved and removed bug Something isn't working pending This problem is yet to be addressed labels Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
solved This problem has been already solved
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[model] fix qwen2 moe hiyouga/LLaMA-Factory
2 participants
@hiyouga @MrLittleB