Skip to content

claude-code-jupyter-setup-guide.md

Latest commit

 

History

History
241 lines (171 loc) · 8 KB

claude-code-jupyter-setup-guide.md

File metadata and controls

241 lines (171 loc) · 8 KB

Claude Code + Jupyter Notebook Setup Guide

Architecture Overview

LOCAL MACHINE                          COMPUTE CLUSTER
┌──────────────────────┐               ┌──────────────────────┐
│  VS Code             │               │  sbatch job running   │
│  ├── Notebook editor │               │  JupyterLab server   │
│  ├── Claude Code     │  ◄── kernel ──│  (executes cells)    │
│  │   (in terminal)   │   connection  │                      │
│  └── Edits .ipynb    │               │  /path/to/workspace/ │
│       files locally  │               │  ├── notebooks/      │
│           │          │               │  ├── data/           │
│           ▼          │               │  └── scripts/        │
│  /Volumes/<mount>/ ──┼── filesystem ─┤                      │
│  (mounted via Finder)│     mount     │                      │
└──────────────────────┘               └──────────────────────┘
  • File editing happens locally (fast — VS Code + Claude Code)
  • Code execution happens on the cluster (where your data and GPUs live)
  • The filesystem mount bridges the two — same files, seen from both sides

Step 1: Mount the Cluster Filesystem via Finder

Note: If you are off the Fred Hutch campus, connect to the VPN first before mounting the filesystem. The mount will fail without VPN access.

  1. Open Finder
  2. Press Cmd + K to open the "Connect to Server" window
  3. Enter the server address (e.g., smb://cluster-address/share or whatever your institution uses) and click Connect
  4. Authenticate if prompted, then select the volume/folder you want to mount
  5. The mounted volume will appear in Finder's sidebar and at /Volumes/<mount-name>/

Verify the mount path in terminal:

ls /Volumes/
# Find your mount — e.g., /Volumes/my-cluster-share

Note this path — you'll use it in Steps 5 and 6. For the rest of this guide, we'll refer to it as /Volumes/<your-mount>/.

Step 2: Install Claude Code Locally

Claude Code now has a native installer that doesn't require Node.js:

# macOS / Linux
curl -fsSL https://claude.ai/install.sh | bash

Verify the installation:

claude --version

Then authenticate:

claude
# Follow the browser prompts to log in
# Requires a Claude Pro, Max, Teams, or Enterprise subscription

Step 3: Install VS Code + Extensions

  1. Install VS Code from https://code.visualstudio.com

  2. Install these extensions (search in the Extensions panel):

    • Jupyter (ms-toolsai.jupyter) — notebook editor + remote kernel support
    • Python (ms-python.python) — Python language support

That's it — no Remote-SSH extension needed since you're using a filesystem mount.

Step 4: Launch JupyterLab on the Cluster (for the kernel)

You still need a JupyterLab server running on the cluster to execute notebook cells. Modify your existing sbatch script to include one extra flag:

#!/bin/bash
#SBATCH --job-name=jupyter
#SBATCH --partition=<your-partition>
#SBATCH --time=08:00:00
#SBATCH --mem=64G
#SBATCH --cpus-per-task=8
# Add GPU flags if needed: #SBATCH --gres=gpu:1

# Activate your conda environment
source activate <your-env>

# Launch JupyterLab — add the allow_origin flag to your existing command
jupyter lab \
  --no-browser \
  --ServerApp.allow_origin='*'
  # keep any other flags you already use (port, etc.)

The --ServerApp.allow_origin='*' flag is what allows VS Code to connect to this kernel. After the job starts, grab the URL from the job's output log — something like:

http://gizmok47:56252/lab

You'll need this URL in the next step. If your server uses a password, VS Code will prompt you for it when you connect.

Step 5: Connect VS Code to the Remote Kernel

  1. Open VS Code
  2. File → Open Folder → select /Volumes/<your-mount>/ (your mounted workspace)
  3. Open any .ipynb file
  4. In the top-right corner of the notebook, click Select Kernel
  5. Choose Existing Jupyter Server
  6. Paste the JupyterLab URL from Step 4
  7. Select your Python kernel from the list

Test it: Run a cell with !hostname — it should print the cluster compute node's name, confirming execution is happening remotely.

Step 6: Launch Claude Code

Open VS Code's integrated terminal (Ctrl+`` or Cmd+`` on macOS) and start Claude Code in your mounted workspace:

cd /Volumes/<your-mount>
claude

Claude Code now has direct access to all your notebooks, scripts, and data file paths on the cluster filesystem.

Daily Workflow

Once everything is set up, your daily routine is:

  1. Mount the filesystem via Finder (Cmd + K), or confirm it's still mounted

  2. Start your sbatch Jupyter job on the cluster (if not already running)

  3. Open VS Code → Open /Volumes/<your-mount>/ → Open your notebook

  4. Select the remote kernel (VS Code remembers your last server, so after the first time this is usually one click)

  5. Open the terminal in VS Code → Run claude → Start working

  6. Keep an iTerm window open SSH'd into the same compute node running your sbatch job (e.g., ssh gizmok47). Since Claude Code occupies the VS Code terminal, use this iTerm session for things like htop, checking disk usage, inspecting files, or running quick one-off commands on the cluster.

Example interaction with Claude Code

You:  Add a new cell after cell 5 that computes LISA clusters using squidpy
      with Delaunay triangulation on the APM_score column, and store the
      results in adata.obs['APM_lisa_cluster']

Claude Code: [edits the .ipynb file directly]

→ VS Code notebook view updates immediately with the new cell
→ You click "Run Cell" to execute on the cluster
→ Output appears inline in VS Code

Tips: Let Claude Edit, You Execute

It's tempting to have Claude Code both write and run code, but in practice it's easier to let Claude Code handle the editing and run cells yourself in VS Code. A few reasons:

  • Claude Code doesn't have access to your kernel. It's running in a local terminal, not inside the Jupyter session on the cluster. It can't execute cells against your loaded AnnData objects, GPU, or conda environment.
  • You stay in control of execution order. Notebooks are stateful — running cells out of order or re-running expensive cells can cause issues. Clicking "Run Cell" yourself lets you decide when and what to execute.
  • You see outputs immediately. VS Code renders plots and cell outputs inline. If something looks wrong, you can ask Claude Code to fix the cell and re-run it — a much tighter feedback loop than having Claude try to interpret errors from the terminal.

The recommended pattern is: tell Claude Code what you want → it edits the notebook → you run the cell → you share any errors or outputs back with Claude Code if it needs to iterate.

Troubleshooting

Mount disconnects or edits don't appear

If the Finder mount drops, reconnect via Cmd + K. If Claude Code edits a file but VS Code doesn't reflect it, press Cmd+Shift+P → File: Revert File to force-reload the notebook from disk.

VS Code doesn't detect file changes from Claude Code

Press Cmd+Shift+P → File: Revert File to force-reload the notebook from disk. This should rarely be needed.

Kernel connection drops

Re-select the kernel in VS Code (click the kernel name in the top-right of the notebook). Your notebook content is safe on disk — only the execution connection is interrupted. If the sbatch job expired, submit a new one and update the server URL.

Claude Code can't find your files

Make sure you launched claude from within the mounted directory:

cd /Volumes/<your-mount>/path/to/project
claude