Challenge-03.md

< Previous Challenge - Home - Next Challenge >

Challenge 03 - Semantic Kernel Plugins

Pre-requisites

Completed Challenge 02 and have a functional version of the solution running in Visual Studio or Visual Studio Code.

Introduction

Semantic Kernel truly shines in LLM development when you incorporate plugins. These plugins extend the AI's capabilities and provide access to additional knowledge that cannot be built directly into the model through training. Things such as time sensitive data, user specific information, and esoteric knowledge are all areas where the Plugin model can greatly improve the capabilities of your AI. In this challenge, you will implement a time plugin, and a plugin that retrieves the weather for a location to extend the capabilities of your chat bot.

Description

This challenge will introduce you to building Semantic Kernel Plugins in JAVA, and how to chain plugins using the Auto Function Calling capabilities of Semantic Kernel.

Continuing from the previous challenge, navigate to the application and open the provided application in Visual Studio or Visual Studio Code. If you have forgotten how to do this, refer to the Challenge 02 Reference App for a refresher.

Challenges:

• Use the previous URL (http://localhost:3000) and submit the prompt

    What is current time in Austin

  Since the AI does not have the capability to provide real-time information, you will get a response similar to the following:

    I can't provide real-time information, including the current time. You can check the time on your device or through various online sources        .
  

  Let's fix this by creating a plugin that can provide the current time and other related information.

• Verify a new class in ./Plugins directory for your Time Plugin. You can reference the documentation for more information on how to create a plugin using a class.

  • Write a time plugin with the following functions:

    1. Return the current Date

    2. Return the current Time

    3. Make sure you

       • Update the application to add your new Plugin to Semantic Kernel

         1. Search for Adding method for Plugin | Challenge 3 and uncomment next line.
         2. Go to AddPlugins method in AIService.java file and search Enable Plugin for Date and Time | Challenge 3and add the following code to register your plugin:

         KernelPlugin timePlugin = KernelPluginFactory
                   .createFromObject(new DateTimePlugin(azureAIConfig), "DateTimePlugin");

         3. Add the plugin to kernel by adding the following code to the AddPlugins(...) method in chapter3.java file:

           KernelPlugin timePlugin = KernelPluginFactory
                   .createFromObject(new DateTimePlugin(azureAIConfig), "DateTimePlugin");

         4. Search the Adding invocationContext | Challenge 03 and add the following code to the AddPlugins(...) method in chapter3.java file:

         invocationContext = InvocationContext.builder()
         .withToolCallBehavior(ToolCallBehavior.allowAllKernelFunctions(true)) // Allow unrestricted kernel function calls
         .withReturnMode(InvocationReturnMode.NEW_MESSAGES_ONLY) // Return the full conversation history
         .build();

  • This will enable automatic function calling.

  • Test the AI by http://localhost:3000 launching the application and asking the bot

    What is current Date and time in Austin"
    

    Now, the AI should be able to provide the current time by having Semantic Kernel call the GetTime function in your plugin. The response should be similar to the following:

    The current date in Austin, Texas is March 22, 2025, and the current time is 22:51 (10:51 PM).
    

    See the Success Criteria section for additional questions to test your Time Plugin.

• Review the GeocodingPlugin.java file located in the Plugins directory

  • Register for a free API key from Geocoding API to use in the plugin

  • Update application.properties with your GEOCODING_API_KEY

  • Register the Plugin by adding the following code to the AddPlugins(...) method in chapter3.java

    KernelPlugin geoPlugin = KernelPluginFactory
              .createFromObject(new GeocodingPlugin(azureAIConfig), "GeocodingPlugin");

  • Run the application and test the Geocoding plugin by submitting the following prompt:

    what are the geo-coordinates for Tampa, FL
    

    The AI should respond with the coordinates for Tampa, similar to the following:

    The geo-coordinates for Tampa, FL are approximately:
    
    Latitude: 27.9477595
    Longitude: -82.458444
    If you need more specific coordinates or details about a particular location within Tampa, let me know!
    

• Create a Plugin that calls a Weather API, and add it to the Semantic Kernel. You can utilize the Open Meteo API Here.

  • Add Methods to your plugin to:

    1. Get the forecast weather at lat/long location for up to 16 days in the future

      $"https://api.open-meteo.com/v1/forecast?latitude={latitude}&longitude={longitude}&current=temperature_2m,relative_humidity_2m,apparent_temperature,precipitation,rain,showers,snowfall,weather_code,wind_speed_10m,wind_direction_10m,wind_gusts_10m&hourly=temperature_2m,relative_humidity_2m,apparent_temperature,precipitation_probability,precipitation,rain,showers,snowfall,weather_code,cloud_cover,wind_speed_10m,uv_index&temperature_unit=fahrenheit&wind_speed_unit=mph&precipitation_unit=inch&forecast_days={days}");

    1. Get the current weather at lat/long location for a number of days in the past

        $"https://api.open-meteo.com/v1/forecast?latitude={latitude}&longitude={longitude}&daily=weather_code,temperature_2m_max,temperature_2m_min,apparent_temperature_max,apparent_temperature_min,sunrise,sunset,daylight_duration,uv_index_max,precipitation_sum,rain_sum,showers_sum,snowfall_sum,precipitation_hours,wind_speed_10m_max,wind_gusts_10m_max&temperature_unit=fahrenheit&wind_speed_unit=mph&precipitation_unit=inch&past_days={daysInPast}");

  • Test your Plugins by asking the following question:

    What is the weather in San Francisco next Tuesday?
    

    ❗ The AI should perform the following plan to answer the question but may do so in a different order or different set of functions:

    1️⃣ The AI should ask Semantic Kernel to call the GetDate function on the Time Plugin to get today's date in order to calculate the number of days until next Thursday

    2️⃣ Because the Weather Forecast requires a Latitude and Longitude, the AI should instruct Semantic Kernel to call the GetLocation function on the Geocoding Plugin to get the coordinates for San Francisco

    3️⃣ Finally, the AI should ask Semantic Kernel to call the GetWeatherForecast function on the Weather Plugin passing in the current date/time and Lat/Long to get the weather forecast for Next Thursday (expressed as the number of days in the future) at the coordinates for San Francisco

    A simplified sequence diagram between Semantic Kernel and AI is shown below:

    sequenceDiagram
        participant C as Client
        participant S as Semantic Kernel
        participant A as AI
        C->>S: What is the weather in San Francisco next Tuesday?
        activate C
        S->>+A: What is the weather in San Francisco next Tuesday?
        A-->>-S: Call get_date function
        S->>+A: Results of get_date
        A-->>-S: Call day_of_week function
        S->>+A: Results of day_of_week
        A-->>-S: Call geocode_address function
        S->>+A: Results of geocode_address
        A-->>-S: Call get_weather with lat/long and days in future
        S->>+A: Results of get_weather
        A-->>-S: The weather in San Francisco next Tuesday is...
        S->>C: Here is the weather for San Francisco next Tuesday
        deactivate C
    

    Loading

    💡 Set breakpoints in your plugins to verify that the functions are being called correctly and that the data is being passed between the plugins correctly.

Success Criteria

1. To complete this challenge successfully, the AI should be able to answer the following questions correctly:
   • What time is it?
   • What was the date 4 days ago?
   • What is the day of the week for the last day of next month?
   • What day of the week does today's date next year fall on?
2. In addition, you should have a second plugin that can can answer questions around the weather. You should now be able to get the Chatbot to answer the following questions:
   • What is today's weather in San Francisco?
   • Is it going to rain next week?
   • What's the high temp going to be today?
   • Do I need a raincoat for my movie date Friday?

Learning Resources

Semantic Kernel Resource Links:

• Plugins for Semantic Kernel

• Adding Native Plugins to Semantic Kernel

• Planning in Semantic Kernel

• Planning with Automatic Function Calling

API Client Resources:

• Microsoft Recommendations for HTTP Client

• The Right Way to use HttpClient in .NET

API Docs:

• Geocoding API

• Open-Meteo API

Tips

• Use a API test client such as Bruno or Postman to test the APIs first, before writing code to connect it to your plugin.

Advanced Challenges (Optional)

Too comfortable? Eager to do more? Try these additional challenges!

• Add debug logging messages to your Time plugin.

• Create a Plugin to pull data from a database instead of an API. You can find publicly available data sources here: Database Star - Free Datasets

• Use the resources below to find a public API to build another plugin with.

  • Public API Search Engine
  • Public API List on GitHub
  • Open API List

• Look for APIs that are either current event related - such as events and calendars - or something with esoteric knowledge that the AI does poorly with, such as the UPC database, or something like the PokeAPI - Pokemon Pokedex API.

< Previous Challenge - Home - Next Challenge >