VITAL: king-man+woman=queen with CLIP Vision Transformers

+ Class Neuron Feature Visualization

+ Class Neuron Arithmetic Steering

+ Inner Neuron Feature Visualization

+ Concept Neuron Feature Visualization

+ Faceted Neuron Feature Visualization

This is a heavily modified and expanded implementation (quasi-fork) of VITAL: More Understandable Feature Visualization through Distribution Alignment and Relevant Information Flow. All credits for the original idea to the authors - check out their demo, paper & code!

---

VITAL is a feature visualization framework for deep neural networks. It generates images that reveal what a network perceives at any level of abstraction — from 'high level' class neurons (output), through intermediate hidden neurons, down to the concepts (feature directions) and compositional sub-concepts (facets) that emerge inside the visual encoder.

🎯 Features:

• GUI (quick start: simplified options, no determinism)
• CLI config: json dump / load
• Custom CLIP code adapted from OpenAI/CLIP
• For CLIP ViT and ResNet (CNN), supports loading from HF 🤗
• Sort-Matching alignment to real feature distributions (Wasserstein-1 approximation)
• Live CLIP 'opinion' (text encoder cos sim) about optimization image, compares against embeddings over a large vocab ~50k (precomputed once per model) + separate dynamic user prompts in input_prompts/prompts.txt.
• Laplacian pyramid + tanh
• Optional large visualizations (via embeddings interpolation for ViT)
• Attention gradient steering (default: on) backpropagates through the attention softmax directly into the pyramid parameters, providing real gradient signal for spatial composition; can be controlled via entropy and spatial priors in config.
• Spatial neural fields uses a small Coordinate-MLP to map patch positions to features (reduces noise).
  Adapted from paper: Denoising Vision Transformers

---

✅ Usage notes:

• Use --help for all argparse options; edit class CFG in code or use --json_cfg for all config options
• Use --save_steps to save intermediate steps (default: 200) instead of only final image
• --deterministic uses fixed seeds, torch.backends.cudnn.deterministic=True, torch.backends.cudnn.benchmark=False
• --hard_determinism uses CUBLAS_WORKSPACE_CONFIG=:4096:8, torch.use_deterministic_algorithms(True)
• All examples below made with presets and datasets included (just run & reproduce)
• See code for further info / documentation
• 🚨 Warning - may unexpectedly produce NSFW output, harmful stereotypes, etc. - even if benign 'PG' input images are used, and even if images do not contain people. Especially unpredictable with Class Neuron Arithmetic Steering. Refer to the CLIP MODEL CARD.

GUI

👉 An intuitive way to get started, with only basic options exposed.

Ensure requirements (pip install -r requirements.txt); then simply run:

python vital_gui.py

PS: Your generated visualizations will be saved to out_* folders!

---

Class Neuron Feature Visualization

👉 Class neuron visualization aims to reveal what a neural network "sees" when it thinks about a specific class (e.g., "dog"); it targets the output, i.e. provides a holistic / high-level view.
./class_fvis

python class_neurons_fvis.py --deterministic --hard_determinism

Class FVis is where number of steps and determinism matters most.
See code: in class CFG: steps_multires: Tuple[int, int] = (400, 1200).

ImageNet 'Goldfinch', ViT:

CLIP ResNet / CNN:

Class Neuron Arithmetic Steering

👉 Unlike Class Neuron Visualization, this takes a single input image. We embed additional add/subtract images for vector arithmetic, and backpropagate the resulting embedding vector to steer the latent concept.
🤴-👨+👩=👸🏻
./class_fvis

Exception: Here, deterministic is on by default. Use --no_determinism to disable.

python class_neurons_fvis_arithmetic.py --hard_determinism --sub_alpha 0.0 --add_beta 0.0

python class_neurons_fvis_arithmetic.py --hard_determinism --sub_alpha 0.9 --add_beta 1.0

Note: CLIP's typographic attack vulnerability is a feature here; when CLIP 'reads' the word MAN, it has a similar effect (direction) to 'seeing' a photo of a man; except it's much cleaner.
What's a King's robe minus a Suit & Tie, anyway?! Swap the word-images for photos and find out! :)

Baseline with factors=0 vs. producing a Queen:

Strong add vs. even stronger add... And typographic self-attack emerges. :))

---

Inner Neuron Feature Visualization

👉 Intermediate neuron visualization focuses on understanding how information is represented deep inside the network. Internal neurons often respond to abstract concepts like "furry", even if they are not directly tied to a class.
./inner_fvis

Use python query_model_layers.py if you need a layer mapping, especially for CNN.
Note: CNN works, but is currently not fully implemented, e.g. no auto-channel detection.

python inner_neurons_fvis.py --deterministic --hard_determinism

Neurons in ViT responding to bird neck (left); wings / flying (right).

---

Concept Neuron Feature Visualization

👉 Concept visualization applies CRAFT-style NMF decomposition to the patch activations at a chosen ViT resblock, decomposing the reference image set into n concept directions ranked by their contribution to the model's final representation.
./concept_fvis

python concept_neurons_fvis.py --deterministic --hard_determinism

Use --extract_only to only dump concept bank for Faceted Neuron FVis (no concept visualization):

python concept_neurons_fvis.py --deterministic --hard_determinism --extract_only

Example Concept Neuron Visualizations show unusual detail for block 12 (ViT-L/14); possible register (global) direction?

ViT-L/14 Block 12, closer look: Directions appear coherent (common features).
Curiously, everything seems to be represented with human entanglement.
Top: Presumably a 'contrast' direction.
Bottom: Common feature is a bird with food held in beak.

Kind of bird (magpie, red macaw)� and emotion direction? Top: 🤔, bottom: 😖
Although (top) may be a different kind (no 'register-suspicious' saliency).

Faceted Neuron Feature Visualization

👉 Facet visualization decomposes a single high-level concept direction (from concept visualization) into its constituent sub-concepts by applying a second NMF. Where concept visualization reveals what the network encodes, facet visualization reveals how that concept is composed from visual primitives.
./concept_fvis

You need to generate a concept bank before using this.
Ensure same image dataset and --target_layer as you want to use for faceted_fvis!
Tip: Dump multiple blocks' concept bank by passing concept_neuron_fvis.py --layer_block 12 --extract_only arguments.\

For faceted_fvis.py, pass --target_layer and --decomp_layer (or set in class CFG).

python faceted_fvis.py --target_layer 22 --decomp_layer 16 --deterministic --hard_determinism

Another curiosity with excluding registers (by threshold L2 norm in residual stream) vs. not: Block decomp 13 on target 22:

More Facets: ViT-L/14, Block 0:

Block 6 and 7:

Block 9 and 10:

Block 12 and 16:

---