Skip to content

FEAT: Generalize Colloquial Wordswap Attack Converter #1348

Open
taherakolawala wants to merge 2 commits intoAzure:mainfrom
taherakolawala:generalize-colloquial-wordswapper
Open

FEAT: Generalize Colloquial Wordswap Attack Converter #1348
taherakolawala wants to merge 2 commits intoAzure:mainfrom
taherakolawala:generalize-colloquial-wordswapper

Conversation

@taherakolawala
Copy link

@taherakolawala taherakolawala commented Feb 4, 2026

Description

In accordance with the Issue #418 the converter in colloquial_wordswap_converter.py has been generalized to use different versions of colloquial word swaps.

A new directory has been created in pyrit/datasets/prompt_converters called colloquial_wordswaps. This directory contains the original default Singaporean word substitutions as wells as a few different regional colloquial word swap YAML examples. The ColloquialWordSwapConverter class now accepts a new parameter called wordswap_path during initialization. It defaults to singaporean.yaml however the argument can be filled with any YAML file located in the colloquial_wordswaps directory mentioned before.

In the same vein, if you want to add a new set of word substitutions, all you need to do is create a new YAML file in the same format as any of the others and add it to the pyrit/datasets/prompt_converters/colloquial_wordswaps directory. Here is an example of how a YAML file must be formatted:

word: ['alternative1', 'alternative2', 'alternative3']
word2: ['alt1', 'alt2', 'alt3']

The following is an example initialization of the converter with a non-default wordswapper YAML:

converter = ColloquialWordswapConverter(deterministic=True, wordswap_path=filipino.yaml)
# converting the prompt
result = await converter.convert_async('This is my father')
print(result)
# output: This is my papa

People of whom this PR may be of interest to: @eugeniavkim @romanlutz
Closes #418

Tests and Documentation

  • Edited 2 preexisting tests in test_colloquial_wordswap_converter.py for the generalizable wordswap converter.
  • Added 1 new test in test_colloquial_wordswap_converter.py for checking multiple word custom word swapper conversions.
  • All tests pass : uv run pytest tests/unit/converter/test_colloquial_wordswap_converter.py

@taherakolawala
Copy link
Author

@microsoft-github-policy-service agree

Copy link
Contributor

@romanlutz romanlutz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! I am personally not convinced Singaporean should be the default BUT that's what we put in the issue description and it means current users won't be experiencing a breaking change. We may update that in the future, of course. Thanks a ton!

)
else:
file_path = (
Path(__file__).parent.parent
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a paths file under pyrit/common that includes DATASET_PATH which will shorten this

def __init__(
self, deterministic: bool = False, custom_substitutions: Optional[Dict[str, List[str]]] = None
) -> None:
def __init__(self, deterministic: bool = False, wordswap_path: Optional[str] = None) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this is a breaking change. My gut feeling is that it is better to keep the existing signature but having the files is also not a bad idea (even though the substitutions aren't very many and probably not that meaningful.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT: Generalize Colloquial Wordswap Attack Converter

2 participants