Skip to content

Allow tool calls through together-py and remove alternating roles check#244

Merged
connermanuel merged 12 commits intonextfrom
conner-shaping/mosh-1410-allow-tool-calling-datasets-through-together-py
Feb 10, 2026
Merged

Allow tool calls through together-py and remove alternating roles check#244
connermanuel merged 12 commits intonextfrom
conner-shaping/mosh-1410-allow-tool-calling-datasets-through-together-py

Conversation

@connermanuel
Copy link
Copy Markdown
Contributor

@connermanuel connermanuel commented Feb 4, 2026

This PR makes changes to the files validation logic to allow tool calling datasets through. To do this:

  • we allow tools to exist alongside messages for conversational data
  • we allow messages to have "tool_calls" instead of "content"
  • we allow the role of "tool"
  • we remove the requirement for alternating roles (since tools can be called sequentially).

Tested by uploading a relevant file.


Note

Medium Risk
Validation rules are loosened for conversational datasets (role alternation and content requirements), which could allow previously-rejected malformed training data to pass if the new constraints are insufficient.

Overview
Updates JSONL dataset validation to support tool-calling conversations by allowing a top-level tools field, permitting message roles to include tool, and accepting messages that contain tool_calls instead of content.

Removes the strict user/assistant alternation rule and adjusts message required fields accordingly, with unit tests updated to drop the alternation assertions and to reflect the new required-field behavior.

Written by Cursor Bugbot for commit 984e182. This will update automatically on new commits. Configure here.

@connermanuel connermanuel force-pushed the conner-shaping/mosh-1410-allow-tool-calling-datasets-through-together-py branch from cc93336 to e4981b4 Compare February 4, 2026 20:17
@connermanuel connermanuel marked this pull request as ready for review February 5, 2026 19:10
@connermanuel connermanuel force-pushed the conner-shaping/mosh-1410-allow-tool-calling-datasets-through-together-py branch from 308981e to 40bb052 Compare February 5, 2026 22:09
@connermanuel connermanuel changed the base branch from main to next February 5, 2026 22:09
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Comment thread src/together/lib/utils/files.py
@connermanuel connermanuel changed the title Allow tool calls through together-py and remove outdated checks Allow tool calls through together-py and remove alternating roles check Feb 9, 2026
@connermanuel connermanuel force-pushed the conner-shaping/mosh-1410-allow-tool-calling-datasets-through-together-py branch from 8681c79 to 36d9073 Compare February 9, 2026 21:40
@connermanuel connermanuel merged commit 0c75911 into next Feb 10, 2026
9 checks passed
stainless-app Bot added a commit that referenced this pull request Feb 10, 2026
…ck (#244)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* Allow tool calls through together-py and remove outdated checks

* Remove outdated tests

* lint

* Enforce that either content or tool calls exists

* Skip for empty content

* Condition field checks on role

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
stainless-app Bot added a commit that referenced this pull request Feb 10, 2026
* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* jig papercuts (#238)

* feat(jig): show a unique name tip when deployment create fails

* feat(jig): factor out config_path option and add short flag

* feat: jig support for multi deployment

* codegen metadata

* chore: Update descriptions for jig queue methods and properties

* chore(internal): bump dependencies

* Allow tool calls through together-py and remove alternating roles check (#244)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* Allow tool calls through together-py and remove outdated checks

* Remove outdated tests

* lint

* Enforce that either content or tool calls exists

* Skip for empty content

* Condition field checks on role

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>

* make submit --watch signal status through exit code

* fix(cli): fine-tuning retrieve now renders data instead of schema (#250)

* fix(jig): lint errors

* fix(jig): migrate old state files properly and be even more defensive about parsing deploy errors

* fix(jig): pyright does not handle isinstance type narrowing in ternary expressions. also fix migration logic

* format code

* release: 2.1.0

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Co-authored-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mohamad Rostami <mbrostami@proton.me>
Co-authored-by: Conner Manuel <57027354+connermanuel@users.noreply.github.com>
blainekasten added a commit that referenced this pull request Feb 17, 2026
* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* jig papercuts (#238)

* feat(jig): show a unique name tip when deployment create fails

* feat(jig): factor out config_path option and add short flag

* feat: jig support for multi deployment

* codegen metadata

* chore: Update descriptions for jig queue methods and properties

* chore(internal): bump dependencies

* Allow tool calls through together-py and remove alternating roles check (#244)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* Allow tool calls through together-py and remove outdated checks

* Remove outdated tests

* lint

* Enforce that either content or tool calls exists

* Skip for empty content

* Condition field checks on role

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>

* make submit --watch signal status through exit code

* fix(cli): fine-tuning retrieve now renders data instead of schema (#250)

* fix(jig): lint errors

* fix(jig): migrate old state files properly and be even more defensive about parsing deploy errors

* fix(jig): pyright does not handle isinstance type narrowing in ternary expressions. also fix migration logic

* format code

* Feat: Allow uploading local file during fine tuning create call in the CLI

* cleanup

* Add support for validation_file

* fix lint

---------

Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Co-authored-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mohamad Rostami <mbrostami@proton.me>
Co-authored-by: Conner Manuel <57027354+connermanuel@users.noreply.github.com>
stainless-app Bot added a commit that referenced this pull request Feb 17, 2026
* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* jig papercuts (#238)

* feat(jig): show a unique name tip when deployment create fails

* feat(jig): factor out config_path option and add short flag

* feat: jig support for multi deployment

* codegen metadata

* chore: Update descriptions for jig queue methods and properties

* chore(internal): bump dependencies

* Allow tool calls through together-py and remove alternating roles check (#244)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* Allow tool calls through together-py and remove outdated checks

* Remove outdated tests

* lint

* Enforce that either content or tool calls exists

* Skip for empty content

* Condition field checks on role

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>

* make submit --watch signal status through exit code

* fix(cli): fine-tuning retrieve now renders data instead of schema (#250)

* fix(jig): lint errors

* fix(jig): migrate old state files properly and be even more defensive about parsing deploy errors

* fix(jig): pyright does not handle isinstance type narrowing in ternary expressions. also fix migration logic

* format code

* Feat: Allow uploading local file during fine tuning create call in the CLI

* cleanup

* Add support for validation_file

* fix lint

---------

Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Co-authored-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mohamad Rostami <mbrostami@proton.me>
Co-authored-by: Conner Manuel <57027354+connermanuel@users.noreply.github.com>
stainless-app Bot added a commit that referenced this pull request Feb 19, 2026
* chore: format all `api.md` files

* codegen metadata

* feat: Update descriptions for endpoints

* chore: Better jig deployment progress (#242)

* chore: Fix various docstrings

* feat: Improve file uploads and FT create flows with checksums (#253)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* jig papercuts (#238)

* feat(jig): show a unique name tip when deployment create fails

* feat(jig): factor out config_path option and add short flag

* feat: jig support for multi deployment

* codegen metadata

* chore: Update descriptions for jig queue methods and properties

* chore(internal): bump dependencies

* Allow tool calls through together-py and remove alternating roles check (#244)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* Allow tool calls through together-py and remove outdated checks

* Remove outdated tests

* lint

* Enforce that either content or tool calls exists

* Skip for empty content

* Condition field checks on role

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>

* make submit --watch signal status through exit code

* fix(cli): fine-tuning retrieve now renders data instead of schema (#250)

* fix(jig): lint errors

* fix(jig): migrate old state files properly and be even more defensive about parsing deploy errors

* fix(jig): pyright does not handle isinstance type narrowing in ternary expressions. also fix migration logic

* format code

* Feat: Allow uploading local file during fine tuning create call in the CLI

* cleanup

* Add support for validation_file

* fix lint

---------

Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Co-authored-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mohamad Rostami <mbrostami@proton.me>
Co-authored-by: Conner Manuel <57027354+connermanuel@users.noreply.github.com>

* jig formatted status (#249)

* feat(jig): Format jig status

- chore(jig): typing fix
- chore(jig): minor improvements to status
- fix(jig): status formatter use Typed models
- fix(jig): reformat app status
- fix(jig): wip: reformat config status
- fix(jig): add env to config status
- fix(jig): replica event status
- fix(jig): remove unused age func
- fix(jig): simplify and condense status message
- fix(jig): only show 'ready since' for running replicas
- fix: fixed replica event grouping and image tag split
- fix: Fix minor status errors
- fix(jig): show replica_id instead of revision_id for replica events in jig status

* minor mypy type error

* chore: format_deployment_status(deployment) -> format_deployment_status(d)

* fix(jig): profile -> metric, targetValue -> target

---------

Co-authored-by: technillogue <technillogue@gmail.com>

* chore: Add documentation and changelog to project.urls (#264)

* chore: Remove broken field LineCount from FileResponse

* chore: Remove line_count field from files sdks/clis (#265)

* codegen metadata

* fix(cli): fine-tuning create regression (#270)

* feat(cli): Add json mode to `fine-tuning list --json` (#269)

* chore: Refactor argument options with CLI file downloads (#267)

* chore(cli): Improve messaging when attempting to cancel finetune that is not cancellable (#268)

* chore: format files (#266)

* chore: format files

* check format in CI

* run lint

* chore: configure new SDK language

* chore: Revert adding mcp code. Code additions were unexpected.

* chore: update mock server docs

* release: 2.2.0

---------

Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Co-authored-by: Rasul Nabiyev <dev.rasul.nabiyev@gmail.com>
Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mohamad Rostami <mbrostami@proton.me>
Co-authored-by: Conner Manuel <57027354+connermanuel@users.noreply.github.com>
Co-authored-by: Dulaj Disanayaka <8828757+dulaj-me@users.noreply.github.com>
blainekasten added a commit that referenced this pull request Feb 19, 2026
* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* jig papercuts (#238)

* feat(jig): show a unique name tip when deployment create fails

* feat(jig): factor out config_path option and add short flag

* feat: jig support for multi deployment

* codegen metadata

* chore: Update descriptions for jig queue methods and properties

* chore(internal): bump dependencies

* Allow tool calls through together-py and remove alternating roles check (#244)

* Update README.md (#245)

* Add from_checkpoint parameter to price estimation for FT Job creation (#247)

* codegen metadata

* codegen metadata

* codegen metadata

* Allow tool calls through together-py and remove outdated checks

* Remove outdated tests

* lint

* Enforce that either content or tool calls exists

* Skip for empty content

* Condition field checks on role

---------

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>

* make submit --watch signal status through exit code

* fix(cli): fine-tuning retrieve now renders data instead of schema (#250)

* fix(jig): lint errors

* fix(jig): migrate old state files properly and be even more defensive about parsing deploy errors

* fix(jig): pyright does not handle isinstance type narrowing in ternary expressions. also fix migration logic

* format code

* Feat: Allow uploading local file during fine tuning create call in the CLI

* cleanup

* Add support for validation_file

* fix lint

---------

Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>
Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Co-authored-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mohamad Rostami <mbrostami@proton.me>
Co-authored-by: Conner Manuel <57027354+connermanuel@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants