Skip to content

Add LiveSession testing#8009

Open
emilypgoogle wants to merge 2 commits intomainfrom
ep/live-api-integ-testing
Open

Add LiveSession testing#8009
emilypgoogle wants to merge 2 commits intomainfrom
ep/live-api-integ-testing

Conversation

@emilypgoogle
Copy link
Copy Markdown
Contributor

Adds LiveSession integration testing using recorded clips to our daily AI integration tests.
This was based on the iOS testing.
Currently, there is a disabled test due to the inability of the Android SDK to specify turn complete manually.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 1, 2026

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

@google-oss-bot
Copy link
Copy Markdown
Collaborator

1 Warning
⚠️ Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

@emilypgoogle
Copy link
Copy Markdown
Contributor Author

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a suite of integration tests for LiveSession, covering real-time audio and video processing as well as function calling capabilities. The feedback identifies an improvement opportunity regarding resource management, specifically recommending that MediaMetadataRetriever be released within a finally block to prevent leaks. Additionally, it is noted that a commented-out assertion in the video test should be re-enabled or replaced with a more robust verification strategy to ensure the test remains effective.

Comment on lines +159 to +177
val retriever = MediaMetadataRetriever()
val fd = context.resources.openRawResourceFd(R.raw.videoplayback)
retriever.setDataSource(fd.fileDescriptor, fd.startOffset, fd.length)
fd.close()

val durationStr = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION)
val durationMs = durationStr?.toLong() ?: 0L

// Extract frames every 1 second
for (timeMs in 0 until durationMs step 1000) {
val bitmap =
retriever.getFrameAtTime(timeMs * 1000, MediaMetadataRetriever.OPTION_CLOSEST_SYNC)
if (bitmap != null) {
val stream = ByteArrayOutputStream()
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream)
session.sendVideoRealtime(InlineData(stream.toByteArray(), "image/png"))
}
}
retriever.release()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The MediaMetadataRetriever should be released in a finally block to ensure resources are freed even if an exception occurs during frame extraction. This prevents potential resource leaks in the test environment.

      val retriever = MediaMetadataRetriever()
      try {
        val fd = context.resources.openRawResourceFd(R.raw.videoplayback)
        retriever.setDataSource(fd.fileDescriptor, fd.startOffset, fd.length)
        fd.close()

        val durationStr = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION)
        val durationMs = durationStr?.toLong() ?: 0L

        // Extract frames every 1 second
        for (timeMs in 0 until durationMs step 1000) {
          val bitmap =
            retriever.getFrameAtTime(timeMs * 1000, MediaMetadataRetriever.OPTION_CLOSEST_SYNC)
          if (bitmap != null) {
            val stream = ByteArrayOutputStream()
            bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream)
            session.sendVideoRealtime(InlineData(stream.toByteArray(), "image/png"))
          }
        }
      } finally {
        retriever.release()
      }

Comment on lines +188 to +189
val matches = listOf("cat", "kitten", "kitty").any { response.contains(it) }
// matches shouldBe true // Real model calls might be flakey
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The assertion for the model's response is commented out, leaving the matches variable unused. While integration tests can be flaky, it's better to have an active assertion to verify the functionality. If flakiness is a concern, consider using a retry mechanism or a more flexible matching strategy rather than disabling the check entirely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants