Interesting hallucinations from LLM

[TheRealSteele]

I got this response when running test_good_fit_for_project.py in the examples:

PASSED [100%]
Here is the list of available developers and their skills:

    Developer A:
    - Skills: iOS Development, Swift, Objective-C
    - Availability: Available from May 1st, 2025
    - Skill Level: 5/5

    Developer B:
    - Skills: iOS Development, Swift
    - Availability: Available from June 1st, 2025
    - Skill Level: 4/5

    Developer C:
    - Skills: iOS Development, Swift, SwiftUI
    - Availability: Available from May 15th, 2025
    - Skill Level: 4/5

    Developer D:
    - Skills: iOS Development, Objective-C
    - Availability: Available from June 10th, 2025
    - Skill Level: 3/5

    Developer E:
    - Skills: iOS Development, Swift, UIKit
    - Availability: Available from May 20th, 2025
    - Skill Level: 3/5

    Based on the project requirements for a mobile iOS project using iOS Native tech stack and starting on June 3rd, 
    the best developer for this project would be:
    
    1. Developer C: 
       - Skills: iOS Development, Swift, SwiftUI
       - Availability: Available from May 15th, 2025
       - Skill Level: 4/5

    2. Developer B:
       - Skills: iOS Development, Swift
       - Availability: Available from June 1st, 2025
       - Skill Level: 4/5

[TheRealSteele]

Interesting to see made up developer names A-E when the given no input. I'll try running again to see what other bogus names appear

[TheRealSteele]

More believable hallucinations on a second run:

1. Sarah Adams

   • Skills:
     • iOS Development: Advanced
     • Experience with iPhone development
   • Availability: Available starting June 10th, 2025

2. Michael Chen

   • Skills:
     • iOS Development: Advanced
     • Focus on mobile app development
   • Availability: Available starting May 25th, 2025

3. Luisa Martinez

   • Skills:
     • iOS Development: Intermediate
     • Experience with iPhone development
   • Availability: Available starting June 5th, 2025

4. Khaled Khan

   • Skills:
     • iOS Development: Intermediate
     • Experience with iPhone development
   • Availability: Available starting June 7th, 2025

[paulz]

nice find @TheRealSteele we have CAT Validator detecting hallucinations very reliably.
At the same time how LLM hallucinates or what is the content of hallucinations is hard to predict.