SWEEP.md - MASQ Node Development Guide

This file contains essential information for AI assistants and developers working on the MASQ Node project, including coding standards, common commands, project structure, and development workflows.

Project Overview
Common Terminal Commands
Project Structure
Development Workflow
Code Style and Standards
Testing Guidelines
Build and CI/CD
Important Notes

Project Overview

MASQ Node is the foundation of the MASQ Network, an open-source decentralized mesh-network (dMN) written primarily in Rust. It combines the benefits of VPN and Tor technology to create next-generation privacy software.

Key Technologies

Language: Rust (2021 edition)
Build System: Cargo
Testing: Unit tests, integration tests, multinode integration tests (Docker-based)
CI/CD: GitHub Actions
Platforms: Linux, macOS, Windows (64-bit)

Repository Structure

The project is organized as a Cargo workspace with multiple crates:

node/ - Main MASQ Node implementation
masq/ - Command-line user interface
masq_lib/ - Shared library code
dns_utility/ - DNS configuration utility - deprecated
automap/ - Automatic public-IP detection and configuration
port_exposer/ - Port exposure utilities for testing only
ip_country/ - IP geolocation functionality
multinode_integration_tests/ - Docker-based integration tests employing multiple simultaneous Nodes

Common Terminal Commands

Building and Testing

Run All Tests and Checks (Full CI Pipeline)

ci/all.sh

This runs formatting, linting, unit tests, and integration tests for all components. You will be prompted for your password during zero-hop integration tests (requires sudo).

Run Multinode Integration Tests (Linux only)

ci/multinode_integration_test.sh

Note: These tests only run on Linux and require Docker.

Format Code

ci/format.sh

Formats all Rust code using rustfmt. This is required before submitting PRs.

Run Linting (Clippy)

cargo clippy -- -D warnings -Anon-snake-case

Or for a specific component:

cd node && ci/lint.sh

Run Unit Tests

# For the node component
cd node && ci/unit_tests.sh

# Or manually
cargo test --release --lib --no-fail-fast --features masq_lib/log_recipient_test -- --nocapture --skip _integration

Run Integration Tests

cd node && ci/integration_tests.sh

Note: Integration tests require sudo privileges on Linux and macOS.

Build Release Version

cargo build --release

Build Debug Version

cargo build

Version Management

Bump Version

cd ci && ./bump_version.sh <version>

Example:

cd ci && ./bump_version.sh 6.9.1

This updates version numbers in all Cargo.toml files and corresponding Cargo.lock files.

Running MASQ Node

Start MASQ Daemon (Linux/macOS)

sudo nohup ./MASQNode --initialization &

Start MASQ Daemon (Windows)

start /b MASQNode --initialization

Start MASQ CLI

./masq

Run MASQ Command (Non-interactive)

./masq setup --log-level debug --clandestine-port 1234

Shutdown MASQ Node

./masq shutdown

Revert DNS Configuration

sudo ./dns_utility revert

Git Workflow

Update Master Branch

git checkout master
git pull

Create Feature Branch

git checkout -b GH-<issue-number>
git push -u origin HEAD

Merge Master into Feature Branch

git checkout master
git pull
git checkout -  # Returns to previous branch
git merge master

Docker Commands (for Multinode Tests)

View Docker Logs

multinode_integration_tests/docker_logs.sh

Dump Docker State

multinode_integration_tests/docker_dump.sh

Build Docker Images

multinode_integration_tests/docker/build.sh

Project Structure

Main Components

Node (`node/`)

The core MASQ Node implementation containing:

src/accountant/ - Payment and accounting logic
src/blockchain/ - Blockchain interaction
src/entry_dns/ - Tiny DNS server that returns 'localhost' for any hostname - deprecated
src/hopper/ - Message routing
src/neighborhood/ - Network topology management
src/proxy_client/ - Exit-Node proxy
src/proxy_server/ - Originating-Node proxy
src/ui_gateway/ - User interface communication
src/sub_lib/ - Code shared among accountant, blockchain, hopper, etc.
src/test_utils/ - Testing utilities

Each component has its own README.md with detailed documentation.

MASQ CLI (`masq/`)

Command-line interface for controlling the MASQ Daemon and Node.

DNS Utility (`dns_utility/`) - deprecated

The standard way to add intercept processing to your network data flow is to configure your system network stack to use an HTTP and/or HTTPS proxy. However, early in the history of Substratum, Justin Tabb made a promise that Node would be "zero-configuration." This utility, and the code in Node that it supports, were intended to keep that promise. Essentially, Node contains a tiny DNS server that always returns 'localhost' for any hostname; therefore, when your browser or other application performs DNS resolution, it will be fooled into routing its traffic through Node, which is running on localhost. This utility wrangles your system DNS settings to point at that tiny DNS server, if you're going to run Node, or back at your real DNS server if you're not. Since Node is now part of MASQ and no longer part of Substratum and Justin Tabb is a part of history, the Node now operates as an HTTPS proxy and its entry_dns server is no longer used. However, the code is still available and active and usable, so this utility remains available as well, although its use is deprecated.

Multinode Integration Tests (`multinode_integration_tests/`)

Docker-based integration tests that simulate multi-node networks.

Configuration Files

Cargo.toml

Each component has its own Cargo.toml defining dependencies and metadata.

config.toml

Runtime configuration file (located in data directory by default).

Can be specified via --config-file parameter or MASQ_CONFIG_FILE environment variable
Uses TOML format with scalar settings
Example: clandestine-port = 1234

Development Workflow

Setting Up Development Environment

Install Rust toolchain:

rustup component add rustfmt
rustup component add clippy

Install Docker (for multinode tests on Linux)

Clone repository and test:

git clone <repository-url>
cd Node
ci/all.sh

Working on an Issue

Select issue from the MASQ Node Card Wall
Update master branch: git checkout master && git pull
Create feature branch: git checkout -b GH-<issue-number>
Complete the work (test-driven development required)
Merge in master regularly: git merge master
Run full test suite: ci/all.sh
Run multinode tests (Linux only): ci/multinode_integration_test.sh
Push changes: git push
Open pull request on GitHub
Watch GitHub Actions build
Address reviewer comments
Wait for QA approval

Commit Guidelines

Commit frequently (commits will be squashed before merging)
Commit when tests go green
Commit before trying risky changes
Write descriptive commit messages for your own reference

Code Style and Standards

Rust Standards

Formatting

Use rustfmt for all code formatting
Run ci/format.sh before committing (non-auto-formatted code will fail CI)
To skip formatting on specific code: #[cfg_attr(rustfmt, rustfmt_skip)]

Linting

All code must pass clippy with -D warnings
Non-snake-case warnings are allowed: -Anon-snake-case
Run cargo clippy -- -D warnings -Anon-snake-case

Compiler Flags

export RUSTFLAGS="-D warnings -Anon-snake-case"

Error Handling

Use Result types for fallible operations
Provide descriptive error messages
Handle errors appropriately (no unwrap() in production code; use expect() only, and that sparingly)
Important: Nothing that comes into the system from the network must ever be allowed to cause a panic. All the standard anti-injection rules apply, but whenever something from the outside generates an error, the error must be logged and execution must continue. (Sometimes additional action is appropriate, such as banning an evil Node; but logging the error is the bare minimum.) eprintln!() is not logging.

Testing

Test-Driven Development (TDD) is required
All new code must have corresponding tests
Tests must pass before code review
Use descriptive test names
Test both success and failure cases

Naming Conventions

Use snake_case for functions and variables
Use PascalCase for types and traits
Use SCREAMING_SNAKE_CASE for constants
All zero-hop integration tests must have names suffixed with _integration; otherwise they'll run as unit tests rather than integration tests.

Documentation

Document public APIs with doc comments (///)
Include examples in doc comments where helpful
Keep README.md files updated for each component

Testing Guidelines

Test Types

Unit Tests

Located in the same file as the code being tested (in #[cfg(test)] modules)
Test individual functions and methods in isolation
Run with: cargo test --lib
Should not require sudo or network access

Integration Tests

Test interactions between components
Starts up a real Node, always in --neighborhood-mode zero-hop
Will require sudo privileges, because a Node has to start with root privileges to open low ports
Run with: ci/integration_tests.sh

Multinode Integration Tests

Docker-based tests simulating multi-node networks
Linux only (do not run on macOS or Windows)
Require Docker installation
Run with: ci/multinode_integration_test.sh

Test Execution

Run All Tests

ci/all.sh

Run Specific Test

cargo test test_name -- --nocapture

Run Tests with Backtrace

RUST_BACKTRACE=full cargo test

Run Tests in Release Mode

cargo test --release

Test Features

Use --no-fail-fast to run all tests even if some fail
Use --nocapture to see println! output

Build and CI/CD

Local Build Process

The ci/all.sh script performs the following:

Format check and auto-format (ci/format.sh)
Install git hooks (install-hooks.sh)
Start sccache server (for faster compilation)
Build and test each component:
- masq_lib
- node
- dns_utility
- masq (CLI)
- automap
- ip_country

GitHub Actions

Builds run automatically on pull requests
Unit and single-Node integration tests run on Linux, macOS, and Windows
Multinode integration tests run on Linux only
All builds must pass before merging

Build Artifacts

After successful builds, artifacts are available in:

/target/release/ and /target/debug/ - Executable binaries
- MASQNode (or MASQNode.exe) - Node and Daemon
- masq - Command-line interface
- dns_utility - DNS configuration tool
- automap - Firewall penetration test utility: requires special remote setup

Caching

The project uses sccache for faster compilation:

export SCCACHE_DIR="$HOME/.cargo/sccache"
SCCACHE_IDLE_TIMEOUT=0 sccache --start-server

Important Notes

Platform-Specific Considerations

Linux

Full support for all features
Multinode integration tests available
Requires sudo for integration tests
May need to free port 53: sudo ci/free-port-53.sh

macOS

Full support except multinode integration tests
Requires sudo for integration tests
May need to adjust file descriptor limits in GitHub Actions

Windows

Use Git Bash for running scripts
Multinode integration tests not supported
Some services may need to be stopped (ICS, W3SVC)
32-bit Windows not reliably supported (use 64-bit)
Run as Administrator

Security Considerations

MASQ Node is currently in beta - not clandestine yet
Do not use for sensitive traffic
Traffic cannot be decrypted by attackers but MASQ traffic is identifiable
Database encryption uses a user-provided password
Password is never stored on disk
Forgetting the password means losing all the encrypted content in the database and starting over

Configuration Priority

Configuration sources in order of priority (highest to lowest):

masq UI commands
Environment variables (prefixed with MASQ_)
Configuration file (config.toml)
Defaults