Skip to content

Issues: BlackSamorez/tensor_parallel

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

33 Open 33 Closed
33 Open 33 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Compatibility with transformers > 4.36: error: AttributeError: 'tuple' object has no attribute 'to_legacy_cache'
#137 opened Aug 21, 2024 by Dr-Left
2
Customized generate func support?
#136 opened May 1, 2024 by MonolithFoundation
tensor_parallel int4 LLM is not working since release v2.0.0
#133 opened Jan 2, 2024 by ReinForce-II
Now, does tensor_parallel no longer support the huggingface trainer?
#132 opened Dec 31, 2023 by HanGyeol-Yoo
Can I use tensor_parallel to inference for a GPTQ quantized model?
#131 opened Nov 15, 2023 by minlik
No implement of generate() when using models from hugging face.
#130 opened Nov 14, 2023 by 342215448
TensorParallel object has no attribute save_pretrained
#129 opened Nov 3, 2023 by toufunao
No output when using tensor_parallel
#128 opened Oct 27, 2023 by yyya9
1
AttributeError: object has no attribute 'devices'
#125 opened Oct 9, 2023 by QiueY514
ValueError: Model parameters were moved to incorrect devices, did call on model.cuda() or model.to(device)? If so, please avoid doing that
#124 opened Oct 4, 2023 by Khyat
Max Recursion Error when using with lora
#122 opened Oct 2, 2023 by Ar-Kareem
2
RuntimeError: NCCL Error 3: internal error
#121 opened Sep 12, 2023 by smallmocha
1
Segmentation fault (core dumped)
#120 opened Sep 11, 2023 by jameswu2014
Would it suitable for the multi-GPU parallel inference for llama2?
#118 opened Sep 1, 2023 by aclie
2x slowdown using TP
#117 opened Aug 26, 2023 by jph00
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
#116 opened Aug 11, 2023 by SparkJiao
distributed TP model forward output's requires_grad is False
#115 opened Aug 10, 2023 by lxuechen
5
tensor_parallel method distributed=True
#114 opened Aug 2, 2023 by Johnno1011
2
When I try to do the tensor_parallel on NLLB from meta, there is an error
#105 opened Jul 22, 2023 by 342215448
Question on custom models
#88 opened Jun 13, 2023 by vince62s
23
Does tensor_parallel support data parallel and tensor parallel hybrid training?
#85 opened Jun 7, 2023 by liguodongiot
Does tensor_parallel support multi-node tensor parallel training?
#84 opened Jun 7, 2023 by liguodongiot
6
Can I parallelize just one large layer?
#83 opened Jun 6, 2023 by chinmayjog13
1
Great work！ and can this work with deepspeedzero?
#75 opened May 22, 2023 by laoda513
Slow inference performance for large Llama models compared to naive MP
#66 opened Apr 6, 2023 by sgsdxzy
26
ProTip! Mix and match filters to narrow down what you’re looking for.