Skip to content

Ramalama sits idle for several minutes #1443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
almusil opened this issue May 28, 2025 · 5 comments
Closed

Ramalama sits idle for several minutes #1443

almusil opened this issue May 28, 2025 · 5 comments
Labels
bug Something isn't working

Comments

@almusil
Copy link
Contributor

almusil commented May 28, 2025

Issue Description

Running ramalama --debug run deepseek-coder-v2:236b will sit idle (0% cpu being used by the process) without any output for up to 40 minutes. Then it starts the model download, which can get stuck at 99% for another several minutes.

It might be something machine specific.

Steps to reproduce the issue

ramalama --debug run deepseek-coder-v2:236b

Describe the results you received

Very long waiting time for the actual download to happen.

Describe the results you expected

The download to start right away.

ramalama info output

# ramalama info
{
    "Accelerator": "none",
    "Engine": {
        "Info": {
            "host": {
                "arch": "amd64",
                "buildahVersion": "1.39.2",
                "cgroupControllers": [
                    "cpuset",
                    "cpu",
                    "io",
                    "memory",
                    "hugetlb",
                    "pids",
                    "rdma",
                    "misc",
                    "dmem"
                ],
                "cgroupManager": "systemd",
                "cgroupVersion": "v2",
                "conmon": {
                    "package": "conmon-2.1.13-1.fc42.x86_64",
                    "path": "/usr/bin/conmon",
                    "version": "conmon version 2.1.13, commit: "
                },
                "cpuUtilization": {
                    "idlePercent": 99.99,
                    "systemPercent": 0,
                    "userPercent": 0.01
                },
                "cpus": 64,
                "databaseBackend": "sqlite",
                "distribution": {
                    "distribution": "fedora",
                    "variant": "server",
                    "version": "42"
                },
                "eventLogger": "journald",
                "freeLocks": 2047,
                "hostname": "*REDACTED*",
                "idMappings": {
                    "gidmap": null,
                    "uidmap": null
                },
                "kernel": "6.14.0-63.fc42.x86_64",
                "linkmode": "dynamic",
                "logDriver": "journald",
                "memFree": 84061331456,
                "memTotal": 201339006976,
                "networkBackend": "netavark",
                "networkBackendInfo": {
                    "backend": "netavark",
                    "dns": {
                        "package": "aardvark-dns-1.14.0-1.fc42.x86_64",
                        "path": "/usr/libexec/podman/aardvark-dns",
                        "version": "aardvark-dns 1.14.0"
                    },
                    "package": "netavark-1.14.1-1.fc42.x86_64",
                    "path": "/usr/libexec/podman/netavark",
                    "version": "netavark 1.14.1"
                },
                "ociRuntime": {
                    "name": "crun",
                    "package": "crun-1.20-2.fc42.x86_64",
                    "path": "/usr/bin/crun",
                    "version": "crun version 1.20\ncommit: 9c9a76ac11994701dd666c4f0b869ceffb599a66\nrundir: /run/user/0/crun\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL"
                },
                "os": "linux",
                "pasta": {
                    "executable": "/usr/bin/pasta",
                    "package": "passt-0^20250320.g32f6212-2.fc42.x86_64",
                    "version": ""
                },
                "remoteSocket": {
                    "exists": true,
                    "path": "/run/podman/podman.sock"
                },
                "rootlessNetworkCmd": "pasta",
                "security": {
                    "apparmorEnabled": false,
                    "capabilities": "CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT",
                    "rootless": false,
                    "seccompEnabled": true,
                    "seccompProfilePath": "/usr/share/containers/seccomp.json",
                    "selinuxEnabled": true
                },
                "serviceIsRemote": false,
                "slirp4netns": {
                    "executable": "",
                    "package": "",
                    "version": ""
                },
                "swapFree": 8589668352,
                "swapTotal": 8589930496,
                "uptime": "339h 39m 39.00s (Approximately 14.12 days)",
                "variant": ""
            },
            "plugins": {
                "authorization": null,
                "log": [
                    "k8s-file",
                    "none",
                    "passthrough",
                    "journald"
                ],
                "network": [
                    "bridge",
                    "macvlan",
                    "ipvlan"
                ],
                "volume": [
                    "local"
                ]
            },
            "registries": {
                "search": [
                    "registry.fedoraproject.org",
                    "registry.access.redhat.com",
                    "docker.io"
                ]
            },
            "store": {
                "configFile": "/usr/share/containers/storage.conf",
                "containerStore": {
                    "number": 1,
                    "paused": 0,
                    "running": 1,
                    "stopped": 0
                },
                "graphDriverName": "overlay",
                "graphOptions": {
                    "overlay.additionalImageStores": [
                        "/usr/lib/containers/storage"
                    ],
                    "overlay.imagestore": "/usr/lib/containers/storage",
                    "overlay.mountopt": "nodev,metacopy=on"
                },
                "graphRoot": "/var/lib/containers/storage",
                "graphRootAllocated": 238299381760,
                "graphRootUsed": 121273004032,
                "graphStatus": {
                    "Backing Filesystem": "xfs",
                    "Native Overlay Diff": "false",
                    "Supports d_type": "true",
                    "Supports shifting": "true",
                    "Supports volatile": "true",
                    "Using metacopy": "true"
                },
                "imageCopyTmpDir": "/var/tmp",
                "imageStore": {
                    "number": 16
                },
                "runRoot": "/run/containers/storage",
                "transientStore": false,
                "volumePath": "/var/lib/containers/storage/volumes"
            },
            "version": {
                "APIVersion": "5.4.1",
                "BuildOrigin": "Fedora Project",
                "Built": 1741651200,
                "BuiltTime": "Mon Mar 10 20:00:00 2025",
                "GitCommit": "b79bc8afe796cba51dd906270a7e1056ccdfcf9e",
                "GoVersion": "go1.24.0",
                "Os": "linux",
                "OsArch": "linux/amd64",
                "Version": "5.4.1"
            }
        },
        "Name": "podman"
    },
    "Image": "quay.io/ramalama/ramalama:0.8",
    "Runtime": "llama.cpp",
    "Shortnames": {
        "Files": [
            "/root/.local/share/uv/tools/ramalama/share/ramalama/shortnames.conf"
        ],
        "Names": {
            "cerebrum": "huggingface://froggeric/Cerebrum-1.0-7b-GGUF/Cerebrum-1.0-7b-Q4_KS.gguf",
            "deepseek": "ollama://deepseek-r1",
            "dragon": "huggingface://llmware/dragon-mistral-7b-v0/dragon-mistral-7b-q4_k_m.gguf",
            "gemma3": "hf://ggml-org/gemma-3-4b-it-GGUF",
            "gemma3:12b": "hf://ggml-org/gemma-3-12b-it-GGUF",
            "gemma3:1b": "hf://ggml-org/gemma-3-1b-it-GGUF",
            "gemma3:27b": "hf://ggml-org/gemma-3-27b-it-GGUF",
            "gemma3:4b": "hf://ggml-org/gemma-3-4b-it-GGUF",
            "granite": "ollama://granite3.1-dense",
            "granite-code": "hf://ibm-granite/granite-3b-code-base-2k-GGUF/granite-3b-code-base.Q4_K_M.gguf",
            "granite-code:20b": "hf://ibm-granite/granite-20b-code-base-8k-GGUF/granite-20b-code-base.Q4_K_M.gguf",
            "granite-code:34b": "hf://ibm-granite/granite-34b-code-base-8k-GGUF/granite-34b-code-base.Q4_K_M.gguf",
            "granite-code:3b": "hf://ibm-granite/granite-3b-code-base-2k-GGUF/granite-3b-code-base.Q4_K_M.gguf",
            "granite-code:8b": "hf://ibm-granite/granite-8b-code-base-4k-GGUF/granite-8b-code-base.Q4_K_M.gguf",
            "granite-lab-7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "granite-lab-8b": "huggingface://ibm-granite/granite-8b-code-base-GGUF/granite-8b-code-base.Q4_K_M.gguf",
            "granite-lab:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "granite:2b": "ollama://granite3.1-dense:2b",
            "granite:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "granite:8b": "ollama://granite3.1-dense:8b",
            "hermes": "huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q4_K_M.gguf",
            "ibm/granite": "ollama://granite3.1-dense:8b",
            "ibm/granite:2b": "ollama://granite3.1-dense:2b",
            "ibm/granite:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "ibm/granite:8b": "ollama://granite3.1-dense:8b",
            "merlinite": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "merlinite-lab-7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "merlinite-lab:7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "merlinite:7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "mistral": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
            "mistral-small3.1": "hf://bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ2_M.gguf",
            "mistral-small3.1:24b": "hf://bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ2_M.gguf",
            "mistral:7b": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
            "mistral:7b-v1": "huggingface://TheBloke/Mistral-7B-Instruct-v0.1-GGUF/mistral-7b-instruct-v0.1.Q5_K_M.gguf",
            "mistral:7b-v2": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
            "mistral:7b-v3": "huggingface://MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf",
            "mistral_code_16k": "huggingface://TheBloke/Mistral-7B-Code-16K-qlora-GGUF/mistral-7b-code-16k-qlora.Q4_K_M.gguf",
            "mistral_codealpaca": "huggingface://TheBloke/Mistral-7B-codealpaca-lora-GGUF/mistral-7b-codealpaca-lora.Q4_K_M.gguf",
            "mixtao": "huggingface://MaziyarPanahi/MixTAO-7Bx2-MoE-Instruct-v7.0-GGUF/MixTAO-7Bx2-MoE-Instruct-v7.0.Q4_K_M.gguf",
            "openchat": "huggingface://TheBloke/openchat-3.5-0106-GGUF/openchat-3.5-0106.Q4_K_M.gguf",
            "openorca": "huggingface://TheBloke/Mistral-7B-OpenOrca-GGUF/mistral-7b-openorca.Q4_K_M.gguf",
            "phi2": "huggingface://MaziyarPanahi/phi-2-GGUF/phi-2.Q4_K_M.gguf",
            "qwen2.5vl": "hf://ggml-org/Qwen2.5-VL-32B-Instruct-GGUF",
            "qwen2.5vl:2b": "hf://ggml-org/Qwen2.5-VL-2B-Instruct-GGUF",
            "qwen2.5vl:32b": "hf://ggml-org/Qwen2.5-VL-32B-Instruct-GGUF",
            "qwen2.5vl:3b": "hf://ggml-org/Qwen2.5-VL-3B-Instruct-GGUF",
            "qwen2.5vl:7b": "hf://ggml-org/Qwen2.5-VL-7B-Instruct-GGUF",
            "smollm:135m": "ollama://smollm:135m",
            "smolvlm": "hf://ggml-org/SmolVLM-500M-Instruct-GGUF",
            "smolvlm:256m": "hf://ggml-org/SmolVLM-256M-Instruct-GGUF",
            "smolvlm:2b": "hf://ggml-org/SmolVLM-Instruct-GGUF",
            "smolvlm:500m": "hf://ggml-org/SmolVLM-500M-Instruct-GGUF",
            "tiny": "ollama://tinyllama"
        }
    },
    "Store": "/var/lib/ramalama",
    "UseContainer": true,
    "Version": "0.8.5"
}

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

No response
@almusil almusil added the bug Something isn't working label May 28, 2025
@almusil
Copy link
Contributor Author

almusil commented May 28, 2025 •
edited
Loading

It happens when the registry.ollama.ai resolves to IPv6 on hosts that don't have IPv6 access, but have IPv6 default route. It would be nice to have a way of specifying the preferred address family e.g. ramalama -4 run/ramalam -6 run.

@rhatdan
Copy link
Member

rhatdan commented May 28, 2025

Interested in opening a PR?

@almusil
Copy link
Contributor Author

almusil commented May 29, 2025 •
edited
Loading

Sure I can try to put something together.

@almusil almusil mentioned this issue May 29, 2025
Closed
@rhatdan
Copy link
Member

rhatdan commented May 29, 2025

I am now thinking this is not something we can or need to fix in RamaLama, since your system is badly configured. We can not work around all of the issues that this bad network would cause, and users expectations of ramalama to only use IPV4 if configured will not work, since container engines and the actual code within the containers will not necessarily follow the rules.

@almusil
Copy link
Contributor Author

almusil commented May 29, 2025

That's fair point, there are ways how to avoid this issue on the host.

@almusil almusil closed this as completed May 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add options to force preferred IP family. almusil/ramalama
2 participants
@rhatdan @almusil