umarigan/neural-chat-7b-v3-1-GGUF

Made this model using following script: https://github.com/mlabonne/llm-course/blob/main/Quantize_Llama_2_models_using_GGUF_and_llama_cpp.ipynb try it as follow:

from ctransformers import AutoModelForCausalLM, AutoConfig,AutoTokenizer
llm = AutoModelForCausalLM.from_pretrained("umarigan/neural-chat-7b-v3-1-GGUF", 
                                           model_file="neural-chat-7b-v3-1.gguf.q5_k_m.bin",
                                           lib='avx2', #for cpu use
                                           max_new_tokens = 2000,
                                           context_length = 6000)

llm(f"AI is going to")