Skip to content

Instantly share code, notes, and snippets.

@RillonDodgers
Created April 14, 2025 16:04
Show Gist options
  • Save RillonDodgers/7878ba951fcb2097dc4a5fe8cf2624e2 to your computer and use it in GitHub Desktop.
Save RillonDodgers/7878ba951fcb2097dc4a5fe8cf2624e2 to your computer and use it in GitHub Desktop.
Ollama instead of OpenAI in RubyLLM gem
require "ruby_llm"
module RubyLLM::Providers::OpenAI
def self.api_base
"http://localhost:11434/v1"
end
end
RubyLLM.configure do |config|
config.openai_api_key = 'ollama'
end
phi4_model = RubyLLM::ModelInfo.new({
id: "phi4",
created_at: "2024-08-13T04:12:11+02:00",
display_name: "Phi4",
provider: "openai",
context_window: 128000,
max_tokens: 16384,
type: "chat",
family: "chatgpt4o",
supports_vision: true,
supports_functions: false,
supports_json_mode: true,
input_price_per_million: 5.0,
output_price_per_million: 15.0,
metadata: {
object: "model",
owned_by: "system"
}
})
RubyLLM::Models.all.append(phi4_model)
chat = RubyLLM.chat(model: "phi4")
chat.ask "What's the difference between attr_reader and attr_accessor?"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment