Home » Education » How to run MPT-30B 4-bit on CPU - Full Tutorial

How to run MPT-30B 4-bit on CPU - Full Tutorial

Written By 1littlecoder on Tuesday, Jun 27, 2023 | 09:23 AM

 
Run inference on the latest MPT-30B model using your CPU. This inference code uses a ggml quantized model. MPT 30B inference code using CPU - https://github.com/abacaj/mpt-30B-inference 4-bit GGML Model - https://huggingface.co/TheBloke/mpt-30B-chat-GGML/blob/main/mpt-30b-chat.ggmlv0.q4_1.bin (19 GB) ❤️ If you want to support the channel ❤️ Support here: Patreon - https://www.patreon.com/1littlecoder/ Ko-Fi - https://ko-fi.com/1littlecoder

Disclaimer:
All images, musics and videos copyright are belong to their respective owners. We just put embed contents from Youtube.com and do not host any content in our server. These content(s) may be protected by copyright law or other laws regarding intellectual property of the United States or other countries. Downloading file(s) on this site is just for review purpose. If you love the song, please support the artists by buying the original CD/Cassette or buying the song from iTunes.