The code in this repository is under constant revision/refactoring. You could say that I am learning Swift as I develop LLMFarm. So don't expect too much from the code you'll find here. If you have any useful comments on the code, its style or architecture, I will be glad to hear them.
LLMFarm_core swift library to work with large language models (LLM). It allows you to load different LLMs with certain parameters.
Based on ggml and llama.cpp by Georgi Gerganov.
- MacOS (13+)
- iOS (16+)
- Various inferences
- Various sampling methods
- Metal (dont work on intel Mac)
- Model setting templates
- LoRA adapters support (read more)
- LoRA train support
- LoRA export as model support
- Restore context state (now only chat history)
See full list here.
- Temperature (temp, tok-k, top-p)
- Tail Free Sampling (TFS)
- Locally Typical Sampling
- Mirostat
- Greedy
- Grammar
- Classifier-Free Guidance
git clone https://github.com/guinmoon/llmfarm_core.swift
Add llmfarm_core to your project using Xcode (File > Add Packages...) or by adding it to your project's Package.swift file:
dependencies: [
.package(url: "https://github.com/guinmoon/llmfarm_core.swift")
]To Debug llmfarm_core package, do not forget to comment .unsafeFlags(["-Ofast"]) in Package.swift.
Don't forget that the debug version is slower than the release version.
- rwkv.cpp by saharNooby.
- Mia by byroneverson.
- swift-markdown-ui by gonzalezreal
App to run LLaMA and other large language models locally on iOS and MacOS.