Moje první lokálně běžící LLM! 2024-12-06
Okamžitě funguje.
$ docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Stáhlo necelý 2 GB a pustilo se na pozadí. Je to jen platforma, nutno stáhnout jeden z dostupných modelů. Následující příkaz stáhne model o velikosti skoro 4 GB a hned spustí command line interface:
$ docker exec -it ollama ollama run llama2
>>> /?
Available Commands:
/set Set session variables
/show Show model information
/load <model> Load a session or model
/save <model> Save your current session
/clear Clear session context
/bye Exit
/?, /help Help for a command
/? shortcuts Help for keyboard shortcuts
Use """ to begin a multi-line message.
>>> /show info
Model
architecture llama
parameters 6.7B
context length 4096
embedding length 4096
quantization Q4_0
...
>>> /show system
No system message was specified for this model.
>>> Can you tell me who are you? What is your purpose?
Hello! I'm just an AI assistant developed by Meta AI, and my primary purpose is to assist users like you in a variety of tasks. My
capabilities include:
1. Answering questions: I can provide information on a wide range of topics, from science and history to entertainment and
culture.
2. Generating text: I can create text based on a given prompt or topic, and can even help with writing tasks such as
proofreading and editing.
3. Translation: I can translate text from one language to another, helping to break down language barriers and facilitate
communication between people who speak different languages.
4. Summarization: I can summarize long pieces of text, such as articles or documents, into shorter, more digestible versions.
5. Creative writing: I can generate creative content, such as poetry or short stories, based on prompts or topics provided by
users.
6. Conversation: I can engage in natural-sounding conversations, using context and understanding to respond to questions and
statements.
7. Data analysis: I can analyze and process large amounts of data, extracting insights and meaningful patterns.
8. Content creation: I can create content, such as articles, social media posts, or even entire websites, based on user input and
preferences.
9. Sentiment analysis: I can analyze text to determine the sentiment or emotional tone behind it, such as detecting whether a
piece of text is positive, negative, or neutral.
10. Language understanding: I can understand natural language and detect things like sarcasm, idioms, and figurative language.
These are just a few examples of what I can do. If you have any specific questions or tasks in mind, feel free to ask me anything!
Měl to napsané docela rychle! A to podle mě nepoužívá GPU, je moje CPU. Všech 12 jader ale maká!
Zároveň ten server na portu 11434 nabízí služby třeba Python klientovi:
$ python3 -m venv ollama
$ ./ollama/bin/pip install ollama
$ ./ollama/bin/python
Python 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ollama
>>> ollama.list()
ListResponse(models=[Model(model='llama2:latest', modified_at=datetime.datetime(2024, 12, 6, 9, 56, 31, 793383, tzinfo=TzInfo(UTC)), digest='78e26419b4469263f75331927a00a0284ef6544c1975b826b15abdaef17bb962', size=3826793677, details=ModelDetails(parent_model='', format='gguf', family='llama', families=['llama'], parameter_size='7B', quantization_level='Q4_0'))])
>>> response = ollama.generate(model='llama2:latest', prompt='what is a qubit?')
>>> response
GenerateResponse(model='llama2:latest', created_at='2024-12-06T09:59:12.875202056Z', done=True, done_reason='stop', total_duration=80779008014, load_duration=7705745, prompt_eval_count=26, prompt_eval_duration=701000000, eval_count=569, eval_duration=80069000000, response='\nA qubit (short for "quantum bit") is the fundamental unit of quantum information in quantum computing. It is the basic building block for representing and manipulating quantum states, which are the basis of quantum computations.\n\nIn classical computing, a bit is either 0 or 1, but in quantum computing, a qubit can exist in multiple states simultaneously, known as superposition. This means that a qubit can represent both 0 and 1 at the same time, which allows for much more complex and powerful computations than classical computers.\n\nA qubit can also be entangled with other qubits, meaning that the state of one qubit is linked to the state of another qubit, even when they are separated by large distances. This property of entanglement is a key feature of quantum computing and allows for the creation of highly complex and powerful quantum algorithms.\n\nThere are several types of qubits that have been proposed or implemented, including:\n\n1. Spin qubits: These are based on the spin of an electron or nucleus in a material and can be manipulated using magnetic fields.\n2. Photon qubits: These are based on the polarization of photons (light particles) and can be manipulated using optical components such as mirrors, lenses, and beam splitters.\n3. Superconducting qubits: These are made from superconducting materials and can be manipulated using magnetic fields.\n4. Quantum dot qubits: These are based on small particles called quantum dots, which can be manipulated using optical or electrical means.\n5. Topological qubits: These are based on the spin of electrons in a material and can be manipulated using topological defects such as Majorana fermions.\n\nQubits can be used to perform a wide range of quantum computations, including:\n\n1. Quantum simulation: Qubits can be used to simulate the behavior of complex quantum systems, such as chemical reactions or materials properties, that are difficult or impossible to model using classical computers.\n2. Quantum optimization: Qubits can be used to optimize complex systems, such as machine learning models or traffic flow, by searching through a vast number of possible solutions in parallel.\n3. Quantum cryptography: Qubits can be used to create highly secure encryption algorithms that are resistant to classical computer attacks.\n4. Quantum artificial intelligence: Qubits can be used to create highly advanced AI systems that can solve complex problems that are beyond the capabilities of classical computers.\n\nOverall, qubits are the fundamental building blocks of quantum computing and have the potential to revolutionize many fields of science and technology.', context=[518, 25580, 29962, 3532, 14816, 29903, 29958, 5299, 829, 14816, 29903, 6778, 13, 13, 5816, 338, 263, 439, 2966, 29973, 518, 29914, 25580, 29962, 13, 13, 29909, 439, 2966, 313, 12759, 363, 376, 12150, 398, 2586, 1159, 338, 278, 15281, 5190, 310, 12101, 2472, 297, 12101, 20602, 29889, 739, 338, 278, 6996, 5214, 2908, 363, 15783, 322, 11525, 18099, 12101, 5922, 29892, 607, 526, 278, 8405, 310, 12101, 2912, 800, 29889, 13, 13, 797, 14499, 20602, 29892, 263, 2586, 338, 2845, 29871, 29900, 470, 29871, 29896, 29892, 541, 297, 12101, 20602, 29892, 263, 439, 2966, 508, 1863, 297, 2999, 5922, 21699, 29892, 2998, 408, 2428, 3283, 29889, 910, 2794, 393, 263, 439, 2966, 508, 2755, 1716, 29871, 29900, 322, 29871, 29896, 472, 278, 1021, 931, 29892, 607, 6511, 363, 1568, 901, 4280, 322, 13988, 2912, 800, 1135, 14499, 23226, 29889, 13, 13, 29909, 439, 2966, 508, 884, 367, 875, 574, 839, 411, 916, 439, 14836, 29892, 6593, 393, 278, 2106, 310, 697, 439, 2966, 338, 9024, 304, 278, 2106, 310, 1790, 439, 2966, 29892, 1584, 746, 896, 526, 13055, 491, 2919, 24610, 29889, 910, 2875, 310, 875, 574, 944, 338, 263, 1820, 4682, 310, 12101, 20602, 322, 6511, 363, 278, 11265, 310, 10712, 4280, 322, 13988, 12101, 14009, 29889, 13, 13, 8439, 526, 3196, 4072, 310, 439, 14836, 393, 505, 1063, 7972, 470, 8762, 29892, 3704, 29901, 13, 13, 29896, 29889, 1706, 262, 439, 14836, 29901, 4525, 526, 2729, 373, 278, 10917, 310, 385, 11966, 470, 22699, 375, 297, 263, 5518, 322, 508, 367, 11525, 7964, 773, 15611, 4235, 29889, 13, 29906, 29889, 19040, 265, 439, 14836, 29901, 4525, 526, 2729, 373, 278, 16755, 2133, 310, 6731, 787, 313, 4366, 17105, 29897, 322, 508, 367, 11525, 7964, 773, 27070, 7117, 1316, 408, 19571, 29879, 29892, 301, 11259, 29892, 322, 22913, 6219, 2153, 29889, 13, 29941, 29889, 5670, 535, 2199, 292, 439, 14836, 29901, 4525, 526, 1754, 515, 2428, 535, 2199, 292, 17279, 322, 508, 367, 11525, 7964, 773, 15611, 4235, 29889, 13, 29946, 29889, 22746, 398, 8329, 439, 14836, 29901, 4525, 526, 2729, 373, 2319, 17105, 2000, 12101, 270, 1862, 29892, 607, 508, 367, 11525, 7964, 773, 27070, 470, 3546, 16888, 2794, 29889, 13, 29945, 29889, 7488, 5996, 439, 14836, 29901, 4525, 526, 2729, 373, 278, 10917, 310, 27149, 297, 263, 5518, 322, 508, 367, 11525, 7964, 773, 25002, 23503, 29879, 1316, 408, 11019, 1648, 16895, 1080, 29889, 13, 13, 29984, 431, 1169, 508, 367, 1304, 304, 2189, 263, 9377, 3464, 310, 12101, 2912, 800, 29892, 3704, 29901, 13, 13, 29896, 29889, 22746, 398, 17402, 29901, 660, 431, 1169, 508, 367, 1304, 304, 29611, 278, 6030, 310, 4280, 12101, 6757, 29892, 1316, 408, 22233, 337, 7387, 470, 17279, 4426, 29892, 393, 526, 5189, 470, 9301, 304, 1904, 773, 14499, 23226, 29889, 13, 29906, 29889, 22746, 398, 13883, 29901, 660, 431, 1169, 508, 367, 1304, 304, 24656, 4280, 6757, 29892, 1316, 408, 4933, 6509, 4733, 470, 12469, 4972, 29892, 491, 11975, 1549, 263, 13426, 1353, 310, 1950, 6851, 297, 8943, 29889, 13, 29941, 29889, 22746, 398, 24941, 5275, 29901, 660, 431, 1169, 508, 367, 1304, 304, 1653, 10712, 11592, 20956, 14009, 393, 526, 9241, 424, 304, 14499, 6601, 16661, 29889, 13, 29946, 29889, 22746, 398, 23116, 21082, 29901, 660, 431, 1169, 508, 367, 1304, 304, 1653, 10712, 12862, 319, 29902, 6757, 393, 508, 4505, 4280, 4828, 393, 526, 8724, 278, 27108, 310, 14499, 23226, 29889, 13, 13, 3563, 497, 29892, 439, 14836, 526, 278, 15281, 5214, 10930, 310, 12101, 20602, 322, 505, 278, 7037, 304, 19479, 675, 1784, 4235, 310, 10466, 322, 15483, 29889])
Just wow...