Our own quantization and the Argon engine take an open 27B model to 30 tokens a second, on CPU.
Building every layer
It was not the easy way. But building every layer ourselves means we know each one, and can change any of them, for you.
Our own quantization and the Argon engine take an open 27B model to 30 tokens a second, on CPU.
It was not the easy way. But building every layer ourselves means we know each one, and can change any of them, for you.
