Skip to content

Improve compression handling in NodeJS#1

Open
voltrevo wants to merge 1 commit intohazae41:mainfrom
voltrevo:main
Open

Improve compression handling in NodeJS#1
voltrevo wants to merge 1 commit intohazae41:mainfrom
voltrevo:main

Conversation

@voltrevo
Copy link
Copy Markdown

When running echalote in nodejs, multiple issues occur due to responses not getting decompressed correctly. The microdescs response seems to be truncated, and other short messages required for checking digests come through empty.

I'm not sure why this happens in NodeJS but not the browser.

This change seems to mostly fix the issue, although I'm still usually seeing pipe errors even when doing a successful tor fetch. It's also maybe still less reliable than browser. I'm not confident this is the right fix, but it's a big improvement in my testing.

Specifically:

git clone git@github.com:voltrevo/tor-hazae41
git checkout curlTor
cd tor-hazae41
npm install
npx tsx curlTor.ts -v --snowflake-url wss://snowflake.pse.dev/ https://httpbin.org/get

Sample bad output:

> GET https://httpbin.org/get
> Starting fetch request to https://httpbin.org/get
> Target: httpbin.org:443 (HTTPS: true)
> Initializing WASM modules
βœ“ WASM modules initialized successfully
> Connecting to Snowflake bridge at wss://snowflake.pse.dev/
> Creating Snowflake stream
> Connecting streams
> Waiting for Tor to be ready (timeout: 90000ms)
βœ“ Tor client ready!
> Creating circuit
βœ“ Circuit created successfully
> Fetching consensus
! Request failed: Missing flags // <<<<<<---------------- because the response was incomplete due to bug
Error details: Error: Error: Missing flags
Stack: Error: Missing flags
    at Object.parseOrThrow (/home/andrew/workspaces/tor-hazae41/node_modules/@hazae41/echalote/src/mods/tor/consensus/consensus.ts:385:17)
    at Object.fetchOrThrow (/home/andrew/workspaces/tor-hazae41/node_modules/@hazae41/echalote/src/mods/tor/consensus/consensus.ts:80:33)
    at async TorClient.createCircuit (/home/andrew/workspaces/tor-hazae41/src/TorClient.ts:161:23)
    at async <anonymous> (/home/andrew/workspaces/tor-hazae41/src/TorClient.ts:242:23)
    at async TorClient.getOrCreateCircuit (/home/andrew/workspaces/tor-hazae41/src/TorClient.ts:261:12)
    at async TorClient.fetch (/home/andrew/workspaces/tor-hazae41/src/TorClient.ts:335:23)
    at async Function.fetch (/home/andrew/workspaces/tor-hazae41/src/TorClient.ts:302:14)
    at async main (/home/andrew/workspaces/tor-hazae41/curlTor.ts:376:11)

To fix (use this version of fleche in tor-hazae41):

cd ..
git clone git@github.com:voltrevo/fleche
git checkout b3e96bd
npm install
npm pack
cd ../tor-hazae41
npm install ../fleche/hazae41-fleche-1.4.5.tgz
npx tsx curlTor.ts -v --snowflake-url wss://snowflake.pse.dev/ https://httpbin.org/get

Sample good output:

> GET https://httpbin.org/get
> Starting fetch request to https://httpbin.org/get
> Target: httpbin.org:443 (HTTPS: true)
> Initializing WASM modules
βœ“ WASM modules initialized successfully
> Connecting to Snowflake bridge at wss://snowflake.pse.dev/
> Creating Snowflake stream
> Connecting streams
> Waiting for Tor to be ready (timeout: 90000ms)
βœ“ Tor client ready!
> Creating circuit
βœ“ Circuit created successfully
> Fetching consensus
βœ“ Consensus fetched with 8938 microdescs
> Filtering relays
> Found 6903 middle relays and 2661 exit relays
> Extending circuit through middle relay
βœ“ Extended through middle relay
> Extending circuit through exit relay
βœ“ Extended through exit relay
> Scheduled next circuit update in 600000ms with 0ms advance
> Circuit used for first time, scheduling automatic updates
> Opening connection to httpbin.org:443
> Setting up TLS connection
> Making HTTPS request through Tor
βœ“ Request completed successfully
> Circuit disposed
HTTP 200 OK
access-control-allow-credentials: true
access-control-allow-origin: *
connection: close
content-length: 278
content-type: application/json
date: Mon, 27 Oct 2025 04:22:00 GMT
server: gunicorn/19.9.0
> Tor client closed: Connection closed normally
{
  "args": {}, 
  "headers": {
    "Accept-Encoding": "gzip, deflate", 
    "Host": "httpbin.org", 
    "Transfer-Encoding": "chunked", 
    "X-Amzn-Trace-Id": "Root=1-68fef368-144b31d8053029e7630a173c"
  }, 
  "origin": "192.42.116.192", 
  "url": "https://httpbin.org/get"
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant