--local not working on compute nodes without internet access

[swag2198]

Fix to use local venv on Leonardo interactive compute nodes that do not have internet connectivity.

Error message:

(oellm) (base) [shaldar0@lrdn1717 oellm-cli]$ oellm schedule-eval     --models "EleutherAI/pythia-160m"     --tasks "gsm8k"     --n_shot 0     --venv_path .venv     --local true     --limit 1


22:48:53 INFO     Using Python virtual environment at .venv                                                                                                                                   


22:48:54 INFO     Model EleutherAI/pythia-160m not found locally, assuming it is a 🤗 hub model                                                                                               
Using the latest cached version of the dataset since openai/gsm8k couldn't be found on the Hugging Face Hub (offline mode is enabled).
         WARNING  Using the latest cached version of the dataset since openai/gsm8k couldn't be found on the Hugging Face Hub (offline mode is enabled).                                      
⠋ Downloading 'openai/gsm8k/main' (1/1)Found the latest cached dataset configuration 'main' at /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/hf_data/datasets/openai___gsm8k/main/0.0.0/cc7b047b6e5bb11b4f1af84efc572db110a51b3c (last modified on Tue Feb 10 14:42:32 2026).
         WARNING  Found the latest cached dataset configuration 'main' at                                                                                                                     
                  /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/hf_data/datasets/openai___gsm8k/main/0.0.0/cc7b047b6e5bb11b4f1af84efc572db110a51b3c (last modified on Tue Feb 10   
                  14:42:32 2026).                                                                                                                                                             
22:48:55 INFO     Shuffled evaluation jobs for even load distribution across array workers                                                                                                    
         INFO     📊 Evaluation planning:                                                                                                                                                     
         INFO        Total evaluations: 1                                                                                                                                                     
         INFO        Estimated time per eval: 10 minutes                                                                                                                                      
         INFO        Total estimated time: 10 minutes (0.2 hours)                                                                                                                             
         INFO        Desired array size: 1                                                                                                                                                    
         INFO        Actual array size: 1 (limited by queue capacity: 1)                                                                                                                      
         INFO        Evaluations per job: 1                                                                                                                                                   
         INFO        Time per job: 10 minutes (0.2 hours)                                                                                                                                     
         INFO        Time limit with safety margin: 03:59:00                                                                                                                                  
         INFO     📁 Evaluation directory: /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/outputs/2026-04-09-22-48-55                                                               
         INFO     📄 Script: /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/outputs/2026-04-09-22-48-55/submit_evals.sbatch                                                         
         INFO     📋 Job configuration: /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/outputs/2026-04-09-22-48-55/jobs.csv                                                         
         INFO     📊 Results will be stored in: /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/outputs/2026-04-09-22-48-55/results                                                  
         INFO     Running evaluations locally with bash...                                                                                                                                    
Running eval on /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-evals/outputs/2026-04-09-22-48-55/jobs.csv with 1 array jobs distributing 1 evaluations
SLURM_JOB_ID: 39523302
SLURM_ARRAY_JOB_ID: 0
SLURM_ARRAY_TASK_ID: 0
This job (array task 0) will process evaluations from index 1 to 1.
Each array job handles approximately 1 evaluations.
----------------------------------------------------
Starting evaluation for:
  Model: EleutherAI/pythia-160m
  Task: gsm8k
  N-shot: 0
  Suite: lm_eval
----------------------------------------------------
2026-04-09:22:48:55 WARNING  [config.evaluate_config:281] --limit SHOULD ONLY BE USED FOR TESTING. REAL METRICS SHOULD NOT BE COMPUTED USING LIMIT.
2026-04-09:22:49:10 INFO     [_cli.run:375] Including path: /leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/oellm/resources/custom_lm_eval_tasks
2026-04-09:22:49:10 INFO     [_cli.run:376] Selected Tasks: ['gsm8k']
2026-04-09:22:49:15 INFO     [evaluator:211] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-09:22:49:15 INFO     [evaluator:236] Initializing hf model, with arguments: {'pretrained': 'EleutherAI/pythia-160m', 'trust_remote_code': True}
[RANK 0] Detected kernel version 4.18.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
2026-04-09:22:49:22 INFO     [models.huggingface:161] Using device 'cuda:0'
'[Errno 101] Network is unreachable' thrown while requesting HEAD https://huggingface.co/EleutherAI/pythia-160m/resolve/main/vocab.json
Retrying in 1s [Retry 1/5].
Traceback (most recent call last):
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1690, in from_pretrained
    resolved_vocab_files[file_id] = cached_file(
                                    ^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/transformers/utils/hub.py", line 278, in cached_file
    file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/transformers/utils/hub.py", line 512, in cached_files
    raise e
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/transformers/utils/hub.py", line 422, in cached_files
    hf_hub_download(
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 982, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1115, in _hf_hub_download_to_cache_dir
    _get_metadata_or_catch_error(
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1649, in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1572, in get_hf_file_metadata
    response = _httpx_follow_relative_redirects_with_backoff(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_http.py", line 644, in _httpx_follow_relative_redirects_with_backoff
    response = http_backoff(
               ^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_http.py", line 518, in http_backoff
    return next(
           ^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_http.py", line 426, in _http_backoff_base
    response = client.request(method=method, url=url, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/httpx/_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/httpx/_client.py", line 901, in send
    raise RuntimeError("Cannot send a request, as the client has been closed.")
RuntimeError: Cannot send a request, as the client has been closed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/__main__.py", line 14, in <module>
    cli_evaluate()
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/__main__.py", line 10, in cli_evaluate
    parser.execute(args)
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/_cli/harness.py", line 60, in execute
    args.func(args)
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/_cli/run.py", line 379, in _execute
    results = simple_evaluate(
              ^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/utils.py", line 498, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/evaluator.py", line 239, in simple_evaluate
    lm = lm_eval.api.registry.get_model(model).create_from_arg_obj(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/api/model.py", line 180, in create_from_arg_obj
    return cls(**arg_dict, **additional_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/models/huggingface.py", line 204, in __init__
    self._create_tokenizer(
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/lm_eval/models/huggingface.py", line 793, in _create_tokenizer
    self.tokenizer = transformers.AutoTokenizer.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 793, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/leonardo_work/OELLM_prod2026/users/shaldar0/oellm-cli/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1709, in from_pretrained
    raise OSError(
OSError: Can't load tokenizer for 'EleutherAI/pythia-160m'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'EleutherAI/pythia-160m' is the correct path to a directory containing all relevant files for a GPTNeoXTokenizer tokenizer.
Evaluation finished for model: EleutherAI/pythia-160m
Job 0 finished.
22:50:04 INFO     Local evaluation completed. 

[kerkathy]

A convenient fix, tested and it works for me. Just need to remember to set HF_HOME to your own instead of copy-pasting from README.