Pinned to Ideas

• 13 You must be logged in to vote

  💡

  GPU optimization across different cards

  performance Speed related topics hardware Hardware related

  JohannesGaessler started May 13, 2023 in Ideas

  16

Discussions

• 1 You must be logged in to vote

  💡

  GGML types IQ1_M and IQ2_M?

  Nexesenex started Mar 22, 2024 in Ideas

  

  

  3
• 2 You must be logged in to vote

  💡

  Possibly possible size optimization, with a better precision

  drazdra started Mar 16, 2024 in Ideas

  

  

  6
• 2 You must be logged in to vote

  💡

  BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences

  sorasoras started Mar 21, 2024 in Ideas

  

  

  0
• 4 You must be logged in to vote

  💡

  Make llama.cpp brew install-able on Mac!

  Vaibhavs10 started Mar 17, 2024 in Ideas

  

  

  1
• 2 You must be logged in to vote

  💡

  IQ3_S Improvements

  PeterReid started Mar 4, 2024 in Ideas

  

  

  5
• 3 You must be logged in to vote

  💡

  Support for Embedding Models

  SpaceCowboy850 started Nov 17, 2023 in Ideas

  

  

  

  

  12
• 4 You must be logged in to vote

  💡

  SGLang & LookaheadDecoding

  ostix360 started Feb 11, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  MOEish ideas

  Swight1423 started Feb 6, 2024 in Ideas

  

  

  0
• 18 You must be logged in to vote

  💡

  New SOTA 2-Bit Quant released: QuIP-Sharp

  Dampfinchen started Dec 4, 2023 in Ideas

  

  

  

  

  50
• 9 You must be logged in to vote

  💡

  Requesting support for OLMO models from AI2

  kerekovskik started Feb 2, 2024 in Ideas

  

  

  0
• 6 You must be logged in to vote

  💡

  Accelerating IQ2_XS

  PeterReid started Jan 26, 2024 in Ideas

  

  

  

  

  16
• 4 You must be logged in to vote

  💡

  New hierarchical+parallel speculative decoding method, badly named "Lookahead"

  bullno1 started Jan 23, 2024 in Ideas

  

  

  2
• 5 You must be logged in to vote

  💡

  LLaMA* : non-uniform hidden state

  ggerganov started Nov 20, 2023 in Ideas

  

  

  

  

  4
• 7 You must be logged in to vote

  💡

  Should we add a ngram cache API to llama.cpp?

  question Further information is requested

  ggerganov started Nov 27, 2023 in Ideas

  

  

  

  

  4
• 1 You must be logged in to vote

  💡

  Reuse Layers without consuming extra memory

  LiquidGunay started Jan 15, 2024 in Ideas

  

  

  1
• 0 You must be logged in to vote

  💡

  MoE gate logging

  LorenzoBoccaccia started Jan 13, 2024 in Ideas

  

  

  0
• 2 You must be logged in to vote

  💡

  Support for intfloat/e5-mistral-7b-instruct

  nathanpbell started Jan 10, 2024 in Ideas

  

  

  1
• 5 You must be logged in to vote

  💡

  Borrow ideas from LLM in a flash

  bachittle started Dec 22, 2023 in Ideas

  

  

  

  

  3
• 11 You must be logged in to vote

  💡

  Prompt Cache

  apaz-cli started Apr 3, 2023 in Ideas · Closed

  

  

  2
• 3 You must be logged in to vote

  💡

  HQQ (Half-Quadratic Quantization of Large Machine Learning Models)

  joseph777111 started Jan 7, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  Separate thread counts for prompt eval and generation

  stduhpf started Jan 3, 2024 in Ideas

  

  

  2
• 2 You must be logged in to vote

  💡

  PowerInfer 11x speedup

  iplayfast started Dec 29, 2023 in Ideas · Closed

  

  

  1
• 7 You must be logged in to vote

  💡

  Is YaLM supportable?

  patrakov started Apr 13, 2023 in Ideas

  

  

  2
• 1 You must be logged in to vote

  💡

  [Model Request] Cerebras' BTLM-3B-8K

  joseph777111 started Dec 28, 2023 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  Named cache in /completion

  aniljava started Dec 27, 2023 in Ideas

  

  

  0