Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962
In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.
See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix
| def is_valid_state(state): | |
| # check if it is a valid solution | |
| return True | |
| def get_candidates(state): | |
| return [] | |
| def search(state, solutions): | |
| if is_valid_state(state): | |
| solutions.append(state.copy()) |
| --- | |
| version: 2 | |
| services: | |
| pihole: | |
| container_name: pihole | |
| image: pihole/pihole:latest | |
| ports: | |
| - 53:53/tcp #DNS Port | |
| - 53:53/udp #DNS Port | |
| #- 67:67/udp #DHCP Port |
repo -> repository
clone -> bring a repo down from the internet (remote repository like Github) to your local machine
add -> track your files and changes with Git
commit -> save your changes into Git
push -> push your changes to your remote repo on Github (or another website)
pull -> pull changes down from the remote repo to your local machine
status -> check to see which files are being tracked or need to be commited
init -> use this command inside of your project to turn it into a Git repository and start using Git with that codebase
