diff options
| author | Paul Buetow <paul@buetow.org> | 2025-08-18 09:11:20 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2025-08-18 09:11:20 +0300 |
| commit | 3217d2738af345629e7da14c52fa4ee5cb288fe9 (patch) | |
| tree | 29381af9217aabc8fb9029225bfd7650e8f20717 | |
| parent | 041d1f140436c6fdd223844b04c6592c84951878 (diff) | |
feat(config): per-provider temperature defaults and docs\n\n- Add , , to config with coding-friendly default 0.2.\n- Wire defaults through providers (OpenAI, Copilot, Ollama).\n- Update CLI and LSP runners to pass configured temperatures.\n- Document temperature behavior and examples in README.\n- Update config.json.example to show new keys.
| -rw-r--r-- | IDEAS.md | 3 | ||||
| -rw-r--r-- | README.md | 56 | ||||
| -rw-r--r-- | config.json.example | 7 | ||||
| -rw-r--r-- | internal/appconfig/config.go | 37 | ||||
| -rw-r--r-- | internal/hexaicli/run.go | 21 | ||||
| -rw-r--r-- | internal/hexailsp/run.go | 3 | ||||
| -rw-r--r-- | internal/llm/copilot.go | 13 | ||||
| -rw-r--r-- | internal/llm/ollama.go | 45 | ||||
| -rw-r--r-- | internal/llm/openai.go | 23 | ||||
| -rw-r--r-- | internal/llm/provider.go | 76 | ||||
| -rw-r--r-- | internal/llm/util.go | 6 |
11 files changed, 200 insertions, 90 deletions
@@ -15,7 +15,8 @@ ### New features -* [ ] Use hexai as a gh copilot... CLI replacemant for command line questions +* [ ] implement a code action for selected code block the way via a unix pipe as faster access in helix +* [x] Use hexai as a gh copilot... CLI replacemant for command line questions * [ ] Resolve diagnostics code action feature * [X] LSP server to be used with the Helix text editor * [X] Code completion using LLMs @@ -4,6 +4,8 @@ Hexai, the AI LSP for the Helix editor and also a simple command line tool to interact with LLMs in general. +It has been coded with AI and human review. + Hexai exposes a simple LLM provider interface. It supports OpenAI, GitHub Copilot, and a local Ollama server. Provider selection and models are configured via a JSON configuration file. ## Configuration @@ -25,18 +27,21 @@ Hexai exposes a simple LLM provider interface. It supports OpenAI, GitHub Copilo "provider": "ollama", "copilot_model": "gpt-4.1", "copilot_base_url": "https://api.githubcopilot.com", + "copilot_temperature": 0.2, "openai_model": "gpt-4.1", "openai_base_url": "https://api.openai.com/v1", - "ollama_model": "qwen2.5-coder:latest", - "ollama_base_url": "http://localhost:11434" + "openai_temperature": 0.2, + "ollama_model": "qwen3-coder:30b-a3b-q4_K_M", + "ollama_base_url": "http://localhost:11434", + "ollama_temperature": 0.2 } ``` * context_mode: minimal | window | file-on-new-func | always-full * provider: openai | copilot | ollama -* openai_model, openai_base_url: OpenAI-only options -* copilot_model, copilot_base_url: Copilot-only options -* ollama_model, ollama_base_url: Ollama-only options +* openai_model, openai_base_url, openai_temperature: OpenAI-only options +* copilot_model, copilot_base_url, copilot_temperature: Copilot-only options +* ollama_model, ollama_base_url, ollama_temperature: Ollama-only options Ensure `OPENAI_API_KEY` or `COPILOT_API_KEY` is set in your environment according to your chosen provider. @@ -51,6 +56,7 @@ Ensure `OPENAI_API_KEY` or `COPILOT_API_KEY` is set in your environment accordin - In config file: - `openai_model` — model name (default: `gpt-4.1`). - `openai_base_url` — API base (default: `https://api.openai.com/v1`). + - `openai_temperature` — default temperature (coding-friendly default `0.2`). ### GitHub Copilot configuration @@ -58,15 +64,46 @@ Ensure `OPENAI_API_KEY` or `COPILOT_API_KEY` is set in your environment accordin - In config file: - `copilot_model` — model name (default: `gpt-4.1`). - `copilot_base_url` — API base (default: `https://api.githubcopilot.com`). + - `copilot_temperature` — default temperature (coding-friendly default `0.2`). ### Ollama configuration (local) - In config file: - - `ollama_model` — model name/tag (default: `qwen2.5-coder:latest`). + - `ollama_model` — model name/tag (default: `qwen3-coder:30b-a3b-q4_K_M`). - `ollama_base_url` — base URL to Ollama (default: `http://localhost:11434`). + - `ollama_temperature` — default temperature (coding-friendly default `0.2`). + +### Temperature behavior + +* What it is: Temperature controls how random/creative the model's word choices are. + Lower values (≈0–0.3) are more deterministic and precise; higher values (≈0.7+) + produce more diverse, creative outputs. +* Default for coding: When not specified in the config, Hexai uses a + coding-friendly default temperature of `0.2` for all providers. +* Per-provider override: Set `openai_temperature`, `copilot_temperature`, or + `ollama_temperature` to override. Valid ranges depend on the provider, but + typically `0.0`–`2.0`. +* LSP vs CLI: The LSP sometimes overrides temperature for specific actions + (e.g., `0.1`–`0.2` for completions). The CLI uses the configured provider + default unless you change it. + +Recommended ranges and use cases: + +- 0.0–0.3: Deterministic, precise, minimal tangents. Best for code + refactoring, bug fixes, tests, and data extraction. +- 0.4–0.7: Balanced creativity and coherence. General Q&A and most writing. +- 0.8–1.2+: Highly creative/varied. Brainstorming, fiction, or ad copy; may + increase risk of off-target or verbose outputs. + +Guidance: + +- Lower temperature increases consistency and predictability, but can repeat + or be terse. +- Higher temperature increases diversity of phrasing and ideas, but can wander + or introduce mistakes. Notes: -- For Ollama, ensure the model is available locally (e.g., `ollama pull qwen2.5-coder:latest`). +- For Ollama, ensure the model is available locally (e.g., `ollama pull qwen3-coder:30b-a3b-q4_K_M`). - If you run Ollama in OpenAI‑compatible mode, you may alternatively use the OpenAI provider with `openai_base_url` in the config pointing to your local endpoint. @@ -104,11 +141,11 @@ Note, that we have also configured other LSPs here (for Go, `gopls` and `golangc Hexai LSP supports inline trigger tags you can type in your code to request an action from the LLM and then clean up the tag automatically. -- `;some prompt here;`: Do what is written in `some prompt text here`, then remove just the prompt. +- ``: Do what is written in `some prompt text here`, then remove just the prompt. - Strict form: no space after the first ``. - An optional single space immediately after the closing `;` is also removed. - Spaced variants such as `; text ; spaced ;` are ignored. -- `some text here ;;some prompt;` + ## Code actions @@ -156,4 +193,3 @@ hexai 'install ripgrep on macOS' # Verbose explanation hexai 'install ripgrep on macOS and explain' ``` - diff --git a/config.json.example b/config.json.example index 359e862..04ecfb7 100644 --- a/config.json.example +++ b/config.json.example @@ -11,10 +11,13 @@ "openai_model": "gpt-4.1", "openai_base_url": "https://api.openai.com/v1", + "openai_temperature": 0.2, - "ollama_model": "qwen2.5-coder:latest", + "ollama_model": "qwen3-coder:30b-a3b-q4_K_M", "ollama_base_url": "http://localhost:11434", + "ollama_temperature": 0.2, "copilot_model": "gpt-4.1", - "copilot_base_url": "https://api.githubcopilot.com" + "copilot_base_url": "https://api.githubcopilot.com", + "copilot_temperature": 0.2 } diff --git a/internal/appconfig/config.go b/internal/appconfig/config.go index 7027547..c377467 100644 --- a/internal/appconfig/config.go +++ b/internal/appconfig/config.go @@ -13,7 +13,7 @@ import ( // App holds user-configurable settings read from ~/.config/hexai/config.json. type App struct { - MaxTokens int `json:"max_tokens"` + MaxTokens int `json:"max_tokens"` ContextMode string `json:"context_mode"` ContextWindowLines int `json:"context_window_lines"` MaxContextTokens int `json:"max_context_tokens"` @@ -22,23 +22,35 @@ type App struct { TriggerCharacters []string `json:"trigger_characters"` Provider string `json:"provider"` - // Provider-specific options - OpenAIBaseURL string `json:"openai_base_url"` - OpenAIModel string `json:"openai_model"` - OllamaBaseURL string `json:"ollama_base_url"` - OllamaModel string `json:"ollama_model"` - CopilotBaseURL string `json:"copilot_base_url"` - CopilotModel string `json:"copilot_model"` + // Provider-specific options + OpenAIBaseURL string `json:"openai_base_url"` + OpenAIModel string `json:"openai_model"` + // Default temperature for OpenAI requests (nil means use provider default) + OpenAITemperature *float64 `json:"openai_temperature"` + OllamaBaseURL string `json:"ollama_base_url"` + OllamaModel string `json:"ollama_model"` + // Default temperature for Ollama requests (nil means use provider default) + OllamaTemperature *float64 `json:"ollama_temperature"` + CopilotBaseURL string `json:"copilot_base_url"` + CopilotModel string `json:"copilot_model"` + // Default temperature for Copilot requests (nil means use provider default) + CopilotTemperature *float64 `json:"copilot_temperature"` } // Constructor: defaults for App (kept first among functions) func newDefaultConfig() App { + // Coding-friendly default temperature across providers + // Users can override per provider in config.json (including 0.0). + t := 0.2 return App{ MaxTokens: 4000, ContextMode: "always-full", ContextWindowLines: 120, MaxContextTokens: 4000, LogPreviewLimit: 100, + OpenAITemperature: &t, + OllamaTemperature: &t, + CopilotTemperature: &t, } } @@ -115,18 +127,27 @@ func (a *App) mergeWith(other *App) { if strings.TrimSpace(other.OpenAIModel) != "" { a.OpenAIModel = other.OpenAIModel } + if other.OpenAITemperature != nil { // allow explicit 0.0 + a.OpenAITemperature = other.OpenAITemperature + } if strings.TrimSpace(other.OllamaBaseURL) != "" { a.OllamaBaseURL = other.OllamaBaseURL } if strings.TrimSpace(other.OllamaModel) != "" { a.OllamaModel = other.OllamaModel } + if other.OllamaTemperature != nil { // allow explicit 0.0 + a.OllamaTemperature = other.OllamaTemperature + } if strings.TrimSpace(other.CopilotBaseURL) != "" { a.CopilotBaseURL = other.CopilotBaseURL } if strings.TrimSpace(other.CopilotModel) != "" { a.CopilotModel = other.CopilotModel } + if other.CopilotTemperature != nil { // allow explicit 0.0 + a.CopilotTemperature = other.CopilotTemperature + } } func getConfigPath() (string, error) { diff --git a/internal/hexaicli/run.go b/internal/hexaicli/run.go index 11e64b3..839daef 100644 --- a/internal/hexaicli/run.go +++ b/internal/hexaicli/run.go @@ -68,15 +68,18 @@ func readInput(stdin io.Reader, args []string) (string, error) { // newClientFromConfig builds an LLM client from the app config and env keys. func newClientFromConfig(cfg appconfig.App) (llm.Client, error) { - llmCfg := llm.Config{ - Provider: cfg.Provider, - OpenAIBaseURL: cfg.OpenAIBaseURL, - OpenAIModel: cfg.OpenAIModel, - OllamaBaseURL: cfg.OllamaBaseURL, - OllamaModel: cfg.OllamaModel, - CopilotBaseURL: cfg.CopilotBaseURL, - CopilotModel: cfg.CopilotModel, - } + llmCfg := llm.Config{ + Provider: cfg.Provider, + OpenAIBaseURL: cfg.OpenAIBaseURL, + OpenAIModel: cfg.OpenAIModel, + OpenAITemperature: cfg.OpenAITemperature, + OllamaBaseURL: cfg.OllamaBaseURL, + OllamaModel: cfg.OllamaModel, + OllamaTemperature: cfg.OllamaTemperature, + CopilotBaseURL: cfg.CopilotBaseURL, + CopilotModel: cfg.CopilotModel, + CopilotTemperature: cfg.CopilotTemperature, + } oaKey := os.Getenv("OPENAI_API_KEY") cpKey := os.Getenv("COPILOT_API_KEY") return llm.NewFromConfig(llmCfg, oaKey, cpKey) diff --git a/internal/hexailsp/run.go b/internal/hexailsp/run.go index 8a79b12..2eee3aa 100644 --- a/internal/hexailsp/run.go +++ b/internal/hexailsp/run.go @@ -52,10 +52,13 @@ func RunWithFactory(logPath string, stdin io.Reader, stdout io.Writer, logger *l Provider: cfg.Provider, OpenAIBaseURL: cfg.OpenAIBaseURL, OpenAIModel: cfg.OpenAIModel, + OpenAITemperature: cfg.OpenAITemperature, OllamaBaseURL: cfg.OllamaBaseURL, OllamaModel: cfg.OllamaModel, + OllamaTemperature: cfg.OllamaTemperature, CopilotBaseURL: cfg.CopilotBaseURL, CopilotModel: cfg.CopilotModel, + CopilotTemperature: cfg.CopilotTemperature, } oaKey := os.Getenv("OPENAI_API_KEY") cpKey := os.Getenv("COPILOT_API_KEY") diff --git a/internal/llm/copilot.go b/internal/llm/copilot.go index 680e7ec..47ce11e 100644 --- a/internal/llm/copilot.go +++ b/internal/llm/copilot.go @@ -22,6 +22,7 @@ type copilotClient struct { baseURL string defaultModel string chatLogger logging.ChatLogger + defaultTemperature *float64 } type copilotChatRequest struct { @@ -55,7 +56,7 @@ type copilotChatResponse struct { } // Constructor (kept among the first functions by convention) -func newCopilot(baseURL, model, apiKey string) Client { +func newCopilot(baseURL, model, apiKey string, defaultTemp *float64) Client { if strings.TrimSpace(baseURL) == "" { baseURL = "https://api.githubcopilot.com" } @@ -68,6 +69,7 @@ func newCopilot(baseURL, model, apiKey string) Client { baseURL: strings.TrimRight(baseURL, "/"), defaultModel: model, chatLogger: logging.NewChatLogger("copilot"), + defaultTemperature: defaultTemp, } } @@ -101,9 +103,12 @@ func (c copilotClient) Chat(ctx context.Context, messages []Message, opts ...Req for i, m := range messages { req.Messages[i] = copilotMessage{Role: m.Role, Content: m.Content} } - if o.Temperature != 0 { - req.Temperature = &o.Temperature - } + if o.Temperature != 0 { + req.Temperature = &o.Temperature + } else if c.defaultTemperature != nil { + t := *c.defaultTemperature + req.Temperature = &t + } if o.MaxTokens > 0 { req.MaxTokens = &o.MaxTokens } diff --git a/internal/llm/ollama.go b/internal/llm/ollama.go index 14aa558..20dfe2a 100644 --- a/internal/llm/ollama.go +++ b/internal/llm/ollama.go @@ -22,13 +22,14 @@ type ollamaClient struct { baseURL string defaultModel string chatLogger logging.ChatLogger + defaultTemperature *float64 } type ollamaChatRequest struct { - Model string `json:"model"` - Messages []oaMessage `json:"messages"` - Stream bool `json:"stream"` - Options any `json:"options,omitempty"` + Model string `json:"model"` + Messages []oaMessage `json:"messages"` + Stream bool `json:"stream"` + Options any `json:"options,omitempty"` } type ollamaChatResponse struct { @@ -41,21 +42,23 @@ type ollamaChatResponse struct { } // Constructor (kept among the first functions by convention) -func newOllama(baseURL, model string) Client { - if strings.TrimSpace(baseURL) == "" { - baseURL = "http://localhost:11434" - } - if strings.TrimSpace(model) == "" { - model = "qwen2.5-coder:latest" - } +func newOllama(baseURL, model string, defaultTemp *float64) Client { + if strings.TrimSpace(baseURL) == "" { + baseURL = "http://localhost:11434" + } + if strings.TrimSpace(model) == "" { + model = "qwen3-coder:30b-a3b-q4_K_M`" + } return ollamaClient{ httpClient: &http.Client{Timeout: 30 * time.Second}, baseURL: strings.TrimRight(baseURL, "/"), defaultModel: model, chatLogger: logging.NewChatLogger("ollama"), + defaultTemperature: defaultTemp, } } +// TODO: This function is too long and should be refactored for readability and maintainability. func (c ollamaClient) Chat(ctx context.Context, messages []Message, opts ...RequestOption) (string, error) { o := Options{Model: c.defaultModel} for _, opt := range opts { @@ -86,9 +89,11 @@ func (c ollamaClient) Chat(ctx context.Context, messages []Message, opts ...Requ // Build options map only if any option is set optsMap := map[string]any{} - if o.Temperature != 0 { - optsMap["temperature"] = o.Temperature - } + if o.Temperature != 0 { + optsMap["temperature"] = o.Temperature + } else if c.defaultTemperature != nil { + optsMap["temperature"] = *c.defaultTemperature + } if o.MaxTokens > 0 { optsMap["num_predict"] = o.MaxTokens } @@ -177,9 +182,11 @@ func (c ollamaClient) ChatStream(ctx context.Context, messages []Message, onDelt } // Build options map optsMap := map[string]any{} - if o.Temperature != 0 { - optsMap["temperature"] = o.Temperature - } + if o.Temperature != 0 { + optsMap["temperature"] = o.Temperature + } else if c.defaultTemperature != nil { + optsMap["temperature"] = *c.defaultTemperature + } if o.MaxTokens > 0 { optsMap["num_predict"] = o.MaxTokens } @@ -241,6 +248,6 @@ func (c ollamaClient) ChatStream(ctx context.Context, messages []Message, onDelt break } } - logging.Logf("llm/ollama ", "stream end duration=%s", time.Since(start)) - return nil + logging.Logf("llm/ollama ", "stream end duration=%s", time.Since(start)) + return nil } diff --git a/internal/llm/openai.go b/internal/llm/openai.go index 8dc2907..5348def 100644 --- a/internal/llm/openai.go +++ b/internal/llm/openai.go @@ -23,6 +23,7 @@ type openAIClient struct { baseURL string defaultModel string chatLogger logging.ChatLogger + defaultTemperature *float64 } type oaChatRequest struct { @@ -75,7 +76,7 @@ type oaStreamChunk struct { // Constructor (kept among the first functions by convention) // newOpenAI constructs an OpenAI client using explicit configuration values. // The apiKey may be empty; calls will fail until a valid key is supplied. -func newOpenAI(baseURL, model, apiKey string) Client { +func newOpenAI(baseURL, model, apiKey string, defaultTemp *float64) Client { if strings.TrimSpace(baseURL) == "" { baseURL = "https://api.openai.com/v1" } @@ -88,6 +89,7 @@ func newOpenAI(baseURL, model, apiKey string) Client { baseURL: baseURL, defaultModel: model, chatLogger: logging.NewChatLogger("openai"), + defaultTemperature: defaultTemp, } } @@ -120,9 +122,13 @@ func (c openAIClient) Chat(ctx context.Context, messages []Message, opts ...Requ for i, m := range messages { req.Messages[i] = oaMessage{Role: m.Role, Content: m.Content} } - if o.Temperature != 0 { - req.Temperature = &o.Temperature - } + // Decide temperature: request option overrides config default. + if o.Temperature != 0 { + req.Temperature = &o.Temperature + } else if c.defaultTemperature != nil { + t := *c.defaultTemperature + req.Temperature = &t + } if o.MaxTokens > 0 { req.MaxTokens = &o.MaxTokens } @@ -212,9 +218,12 @@ func (c openAIClient) ChatStream(ctx context.Context, messages []Message, onDelt for i, m := range messages { req.Messages[i] = oaMessage{Role: m.Role, Content: m.Content} } - if o.Temperature != 0 { - req.Temperature = &o.Temperature - } + if o.Temperature != 0 { + req.Temperature = &o.Temperature + } else if c.defaultTemperature != nil { + t := *c.defaultTemperature + req.Temperature = &t + } if o.MaxTokens > 0 { req.MaxTokens = &o.MaxTokens } diff --git a/internal/llm/provider.go b/internal/llm/provider.go index c605081..ed9ca59 100644 --- a/internal/llm/provider.go +++ b/internal/llm/provider.go @@ -54,40 +54,56 @@ func WithStop(stop ...string) RequestOption { // Config defines provider configuration read from the Hexai config file. type Config struct { - Provider string - // OpenAI options - OpenAIBaseURL string - OpenAIModel string - // Ollama options - OllamaBaseURL string - OllamaModel string - // Copilot options - CopilotBaseURL string - CopilotModel string + Provider string + // OpenAI options + OpenAIBaseURL string + OpenAIModel string + OpenAITemperature *float64 + // Ollama options + OllamaBaseURL string + OllamaModel string + OllamaTemperature *float64 + // Copilot options + CopilotBaseURL string + CopilotModel string + CopilotTemperature *float64 } // NewFromConfig creates an LLM client using only the supplied configuration. // The OpenAI API key is supplied separately and may be read from the environment // by the caller; other environment-based configuration is not used. func NewFromConfig(cfg Config, openAIAPIKey, copilotAPIKey string) (Client, error) { - p := strings.ToLower(strings.TrimSpace(cfg.Provider)) - if p == "" { - p = "openai" - } - switch p { - case "openai": - if strings.TrimSpace(openAIAPIKey) == "" { - return nil, errors.New("missing OPENAI_API_KEY for provider openai") - } - return newOpenAI(cfg.OpenAIBaseURL, cfg.OpenAIModel, openAIAPIKey), nil - case "ollama": - return newOllama(cfg.OllamaBaseURL, cfg.OllamaModel), nil - case "copilot": - if strings.TrimSpace(copilotAPIKey) == "" { - return nil, errors.New("missing COPILOT_API_KEY for provider copilot") - } - return newCopilot(cfg.CopilotBaseURL, cfg.CopilotModel, copilotAPIKey), nil - default: - return nil, errors.New("unknown LLM provider: " + p) - } + p := strings.ToLower(strings.TrimSpace(cfg.Provider)) + if p == "" { + p = "openai" + } + switch p { + case "openai": + if strings.TrimSpace(openAIAPIKey) == "" { + return nil, errors.New("missing OPENAI_API_KEY for provider openai") + } + // Set coding-friendly default temperature if none provided + if cfg.OpenAITemperature == nil { + t := 0.2 + cfg.OpenAITemperature = &t + } + return newOpenAI(cfg.OpenAIBaseURL, cfg.OpenAIModel, openAIAPIKey, cfg.OpenAITemperature), nil + case "ollama": + if cfg.OllamaTemperature == nil { + t := 0.2 + cfg.OllamaTemperature = &t + } + return newOllama(cfg.OllamaBaseURL, cfg.OllamaModel, cfg.OllamaTemperature), nil + case "copilot": + if strings.TrimSpace(copilotAPIKey) == "" { + return nil, errors.New("missing COPILOT_API_KEY for provider copilot") + } + if cfg.CopilotTemperature == nil { + t := 0.2 + cfg.CopilotTemperature = &t + } + return newCopilot(cfg.CopilotBaseURL, cfg.CopilotModel, copilotAPIKey, cfg.CopilotTemperature), nil + default: + return nil, errors.New("unknown LLM provider: " + p) + } } diff --git a/internal/llm/util.go b/internal/llm/util.go new file mode 100644 index 0000000..b99d7c8 --- /dev/null +++ b/internal/llm/util.go @@ -0,0 +1,6 @@ +package llm + +import "errors" + +// small helper to keep return type consistent +func nilStringErr(msg string) (string, error) { return "", errors.New(msg) } |
