Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/06-api-reference/classes/TextEmbeddingsModule.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ Always call this method when you're done with a model to prevent memory leaks.

> **forward**(`input`): `Promise`\<`Float32Array`\<`ArrayBufferLike`\>\>

Defined in: [modules/natural_language_processing/TextEmbeddingsModule.ts:60](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextEmbeddingsModule.ts#L60)
Defined in: [modules/natural_language_processing/TextEmbeddingsModule.ts:63](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextEmbeddingsModule.ts#L63)

Executes the model's forward pass, where `input` is a text that will be embedded.

Expand Down
71 changes: 64 additions & 7 deletions docs/docs/06-api-reference/classes/TextToSpeechModule.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Class: TextToSpeechModule

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L17)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:18](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L18)

Module for Text to Speech (TTS) functionalities.

Expand All @@ -20,7 +20,7 @@ Module for Text to Speech (TTS) functionalities.

> **nativeModule**: `any` = `null`

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:21](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L21)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:22](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L22)

Native module instance

Expand All @@ -30,7 +30,7 @@ Native module instance

> **delete**(): `void`

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:182](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L182)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:229](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L229)

Unloads the model from memory.

Expand All @@ -44,7 +44,7 @@ Unloads the model from memory.

> **forward**(`text`, `speed?`): `Promise`\<`Float32Array`\<`ArrayBufferLike`\>\>

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:109](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L109)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:118](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L118)

Synthesizes the provided text into speech.
Returns a promise that resolves to the full audio waveform as a `Float32Array`.
Expand All @@ -71,11 +71,43 @@ A promise resolving to the synthesized audio waveform.

---

### forwardFromPhonemes()

> **forwardFromPhonemes**(`phonemes`, `speed?`): `Promise`\<`Float32Array`\<`ArrayBufferLike`\>\>

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:135](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L135)

Synthesizes pre-computed phonemes into speech, bypassing the built-in phonemizer.
This allows using an external G2P system (e.g. the Python `phonemizer` library,
espeak-ng, or any custom phonemizer).

#### Parameters

##### phonemes

`string`

The pre-computed IPA phoneme string.

##### speed?

`number` = `1.0`

Optional speed multiplier for the speech synthesis (default is 1.0).

#### Returns

`Promise`\<`Float32Array`\<`ArrayBufferLike`\>\>

A promise resolving to the synthesized audio waveform.

---

### load()

> **load**(`config`, `onDownloadProgressCallback?`): `Promise`\<`void`\>

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:30](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L30)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L31)

Loads the model and voice assets specified by the config object.
`onDownloadProgressCallback` allows you to monitor the current progress.
Expand Down Expand Up @@ -104,7 +136,7 @@ Optional callback to monitor download progress.

> **stream**(`input`): `AsyncGenerator`\<`Float32Array`\<`ArrayBufferLike`\>\>

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:127](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L127)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:196](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L196)

Starts a streaming synthesis session. Yields audio chunks as they are generated.

Expand All @@ -124,11 +156,36 @@ An async generator yielding Float32Array audio chunks.

---

### streamFromPhonemes()

> **streamFromPhonemes**(`input`): `AsyncGenerator`\<`Float32Array`\<`ArrayBufferLike`\>\>

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:210](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L210)

Starts a streaming synthesis session from pre-computed phonemes.
Bypasses the built-in phonemizer, allowing use of external G2P systems.

#### Parameters

##### input

[`TextToSpeechStreamingPhonemeInput`](../interfaces/TextToSpeechStreamingPhonemeInput.md)

Input object containing phonemes and optional speed.

#### Returns

`AsyncGenerator`\<`Float32Array`\<`ArrayBufferLike`\>\>

An async generator yielding Float32Array audio chunks.

---

### streamStop()

> **streamStop**(): `void`

Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:175](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L175)
Defined in: [modules/natural_language_processing/TextToSpeechModule.ts:222](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/natural_language_processing/TextToSpeechModule.ts#L222)

Stops the streaming process if there is any ongoing.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/06-api-reference/functions/useTextToSpeech.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> **useTextToSpeech**(`TextToSpeechProps`): [`TextToSpeechType`](../interfaces/TextToSpeechType.md)
Defined in: [hooks/natural_language_processing/useTextToSpeech.ts:19](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/natural_language_processing/useTextToSpeech.ts#L19)
Defined in: [hooks/natural_language_processing/useTextToSpeech.ts:22](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/natural_language_processing/useTextToSpeech.ts#L22)

React hook for managing Text to Speech instance.

Expand Down
3 changes: 3 additions & 0 deletions docs/docs/06-api-reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,8 +262,11 @@
- [TextToImageType](interfaces/TextToImageType.md)
- [TextToSpeechConfig](interfaces/TextToSpeechConfig.md)
- [TextToSpeechInput](interfaces/TextToSpeechInput.md)
- [TextToSpeechPhonemeInput](interfaces/TextToSpeechPhonemeInput.md)
- [TextToSpeechProps](interfaces/TextToSpeechProps.md)
- [TextToSpeechStreamingCallbacks](interfaces/TextToSpeechStreamingCallbacks.md)
- [TextToSpeechStreamingInput](interfaces/TextToSpeechStreamingInput.md)
- [TextToSpeechStreamingPhonemeInput](interfaces/TextToSpeechStreamingPhonemeInput.md)
- [TextToSpeechType](interfaces/TextToSpeechType.md)
- [TokenizerProps](interfaces/TokenizerProps.md)
- [TokenizerType](interfaces/TokenizerType.md)
Expand Down
32 changes: 32 additions & 0 deletions docs/docs/06-api-reference/interfaces/TextToSpeechPhonemeInput.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Interface: TextToSpeechPhonemeInput

Defined in: [types/tts.ts:103](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L103)

Text to Speech module input for pre-computed phonemes.
Use this when you have your own phonemizer (e.g. the Python `phonemizer`
library, espeak-ng, or any custom G2P system) and want to bypass the
built-in phonemizer pipeline.

## Extended by

- [`TextToSpeechStreamingPhonemeInput`](TextToSpeechStreamingPhonemeInput.md)

## Properties

### phonemes

> **phonemes**: `string`

Defined in: [types/tts.ts:104](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L104)

pre-computed IPA phoneme string

---

### speed?

> `optional` **speed**: `number`

Defined in: [types/tts.ts:105](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L105)

optional speed argument - the higher it is, the faster the speech becomes
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Interface: TextToSpeechStreamingCallbacks

Defined in: [types/tts.ts:189](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L189)

Shared streaming lifecycle callbacks for TTS streaming modes.

## Extended by

- [`TextToSpeechStreamingInput`](TextToSpeechStreamingInput.md)
- [`TextToSpeechStreamingPhonemeInput`](TextToSpeechStreamingPhonemeInput.md)

## Properties

### onBegin()?

> `optional` **onBegin**: () => `void` \| `Promise`\<`void`\>

Defined in: [types/tts.ts:190](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L190)

Called when streaming begins

#### Returns

`void` \| `Promise`\<`void`\>

---

### onEnd()?

> `optional` **onEnd**: () => `void` \| `Promise`\<`void`\>

Defined in: [types/tts.ts:192](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L192)

Called when streaming ends

#### Returns

`void` \| `Promise`\<`void`\>

---

### onNext()?

> `optional` **onNext**: (`audio`) => `void` \| `Promise`\<`void`\>

Defined in: [types/tts.ts:191](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L191)

Called after each audio chunk gets calculated.

#### Parameters

##### audio

`Float32Array`

#### Returns

`void` \| `Promise`\<`void`\>
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Interface: TextToSpeechStreamingInput

Defined in: [types/tts.ts:156](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L156)
Defined in: [types/tts.ts:205](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L205)

Text to Speech streaming input definition

Expand All @@ -11,43 +11,51 @@ Callbacks can be both synchronous or asynchronous.

## Extends

- [`TextToSpeechInput`](TextToSpeechInput.md)
- [`TextToSpeechInput`](TextToSpeechInput.md).[`TextToSpeechStreamingCallbacks`](TextToSpeechStreamingCallbacks.md)

## Properties

### onBegin()?

> `optional` **onBegin**: () => `void` \| `Promise`\<`void`\>

Defined in: [types/tts.ts:157](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L157)
Defined in: [types/tts.ts:190](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L190)

Called when streaming begins

#### Returns

`void` \| `Promise`\<`void`\>

#### Inherited from

[`TextToSpeechStreamingCallbacks`](TextToSpeechStreamingCallbacks.md).[`onBegin`](TextToSpeechStreamingCallbacks.md#onbegin)

---

### onEnd()?

> `optional` **onEnd**: () => `void` \| `Promise`\<`void`\>

Defined in: [types/tts.ts:159](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L159)
Defined in: [types/tts.ts:192](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L192)

Called when streaming ends

#### Returns

`void` \| `Promise`\<`void`\>

#### Inherited from

[`TextToSpeechStreamingCallbacks`](TextToSpeechStreamingCallbacks.md).[`onEnd`](TextToSpeechStreamingCallbacks.md#onend)

---

### onNext()?

> `optional` **onNext**: (`audio`) => `void` \| `Promise`\<`void`\>

Defined in: [types/tts.ts:158](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L158)
Defined in: [types/tts.ts:191](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/tts.ts#L191)

Called after each audio chunk gets calculated.

Expand All @@ -61,6 +69,10 @@ Called after each audio chunk gets calculated.

`void` \| `Promise`\<`void`\>

#### Inherited from

[`TextToSpeechStreamingCallbacks`](TextToSpeechStreamingCallbacks.md).[`onNext`](TextToSpeechStreamingCallbacks.md#onnext)

---

### speed?
Expand Down
Loading
Loading