awesome llms https://github.com/Hannibal046/Awesome-LLM
- https://github.com/Mooler0410/LLMsPracticalGuide
- počty parametrů https://informationisbeautiful.net/visualizations/the-rise-of-generative-ai-large-language-models-llms-like-chatgpt/
- pytorch tf https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2023/
openai embeddingy
- llama https://arxiv.org/pdf/2302.13971
- gpt2 https://huggingface.co/transformers/v2.2.0/pretrained_models.html
Docker desktop alternativy
- https://rancherdesktop.io/
- https://www.virtualizationhowto.com/2022/01/rancher-desktop-v1-0-features-installation-and-configuration/
- https://stackoverflow.com/questions/70885146/equivalent-to-docker-desktops-host-docker-internal-in-rancher-desktop
- https://podman.io/
- https://dalethestirling.github.io/Macos-volumes-with-Podman/
- https://minikube.sigs.k8s.io/docs/start/
- https://medium.com/rahasak/replace-docker-desktop-with-minikube-and-hyperkit-on-macos-783ce4fb39e3
- https://dhwaneetbhatt.com/blog/run-docker-without-docker-desktop-on-macos
- https://www.redhat.com/sysadmin/podman-windows-wsl2
anaconda license
- https://legal.anaconda.com/policies/en/?name=end-user-license-agreement#end-user-license-agreement
- https://docs.conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html
- https://www.anaconda.com/pricing
cat >$CONDA_ROOT/.condarc <<EOL
default_channels:
- conda-forge
- pytorch
- nvidia
- rapidsai
channels:
- defaults
allow_other_channels: False
EOLza proxy
cat >$CONDA_ROOT/.condarc <<EOL
default_channels:
- https://<proxy>/anaconda.org-conda-forge/
- https://<proxy>/anaconda.org-pytorch/
- https://<proxy>/anaconda.org-nvidia/
- https://<proxy>/anaconda.org-rapidsai/
channels:
- defaults
allow_other_channels: False
EOLnastavení pipu:
obsah pip.conf
[global]
index-url=https://<proxy for pypi>/simple
extra-index-url=https://<local pypi repository>/simplehttps://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2023/
- https://pytorch.org/tutorials/beginner/saving_loading_models.html
- https://huggingface.co/docs/diffusers/using-diffusers/using_safetensors
- https://huggingface.co/docs/safetensors/index
- https://medium.com/@sandyeep70/ggml-to-gguf-a-leap-in-language-model-file-formats-cd5d3a6058f9
- https://medium.com/@phillipgimmi/what-is-gguf-and-ggml-e364834d241c
- https://deci.ai/blog/ggml-vs-gguf-comparing-formats-amp-top-5-methods-for-running-gguf-files/
- https://github.com/ggerganov/ggml
- https://ggml.ai/
- https://www.tensorflow.org/tfx/tutorials/serving/rest_simple#serve_your_model_with_tensorflow_serving
- https://pytorch.org/
- https://github.com/ggerganov/llama.cpp
- https://github.com/ollama/ollama
- https://github.com/Mozilla-Ocho/llamafile
- https://github.com/huggingface/text-generation-inference
- https://github.com/vllm-project/vllm
- https://lmstudio.ai/
- https://github.com/imartinez/privateGPT
- https://github.com/PygmalionAI/aphrodite-engine
- https://github.com/oobabooga/text-generation-webui
- https://onnx.ai/supported-tools.html
- https://onnxruntime.ai/
- https://github.com/microsoft/onnxruntime-genai
- https://github.com/microsoft/onnxruntime-extensions
- https://huggingface.co/docs/hub/gguf
- https://medium.com/@phillipgimmi/what-is-gguf-and-ggml-e364834d241c
srovnání https://betterprogramming.pub/frameworks-for-serving-llms-60b7f7b23407 Speeding up inference https://betterprogramming.pub/speed-up-llm-inference-83653aa24c47 TogetherAI inference server: muc faster than TGI or vllm: https://medium.com/@datadrifters/the-worlds-fastest-llm-inference-engine-3x-faster-than-vllm-and-tgi-a2ed9e33c55f
- k-quants ggml-org/llama.cpp#1684 (comment)
- https://medium.com/@siddharth.vij10/llm-quantization-gptq-qat-awq-gguf-ggml-ptq-2e172cd1b3b5
- https://maartengrootendorst.substack.com/p/which-quantization-method-is-right
- https://medium.com/nixiesearch/how-to-compute-llm-embeddings-3x-faster-with-model-quantization-25523d9b4ce5