18th Place (Silver medal) solution for PII Detect challenge by The Learning Agency Lab

Summary

• Model used: microsoft/deberta-v3-large (6 layers frozen, 5 fold ensemble)
• Performance: F5 = 0.965
• Classes: ['I-URL_PERSONAL', 'I-ID_NUM', 'B-STREET_ADDRESS', 'B-PHONE_NUM', 'B-USERNAME', 'I-PHONE_NUM', 'I-STREET_ADDRESS', 'B-EMAIL', 'B-ID_NUM', 'B-URL_PERSONAL', 'I-NAME_STUDENT', 'B-NAME_STUDENT', 'O']
• Training time: ~ 15hrs on a single NVidia 4090
• Datasets used:
  • Competition Dataset: https://www.kaggle.com/competitions/pii-detection-removal-from-educational-data
  • Custom Datasets:
    • PII Detection Dataset (GPT):
    • PII | External Dataset
    • pii-dd-mistral-generated
    • PII - Mixtral8x7B generated essays
    • Mixtral Original Prompt

Technical Higlights

• Used stratified k-fold
• Used a custom loss function and a trainer to drop 30% of 'O' labels during the loss calculation while allocating 0.01 as the weight for 'O' class