Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
0590f43
feat(hf): support writing and reading from both http and xet
kszucs Feb 9, 2026
1217692
style(hf): cargo and taplo fmt
kszucs Feb 9, 2026
78a8da2
build(hf): switch to use github branch as xet-core optional dependency
kszucs Feb 9, 2026
d4187ff
fix(hf): support config from options
kszucs Feb 9, 2026
e6842a5
fix(hf): only enable write capability if token is provided
kszucs Feb 9, 2026
ce30f7a
feat(hf): add support for deleting files
kszucs Feb 10, 2026
9407fbb
feat(hf): implement BatchDelete instead of OneShotDelete
kszucs Feb 10, 2026
bb982d3
feat(hf): retry requests by default
kszucs Feb 10, 2026
306e0a1
feat(hf): support multiple upload modes and more thorough testing
kszucs Feb 10, 2026
820bc25
style(hf): run taplo format
kszucs Feb 10, 2026
d065ac8
chore(deps): remove tokio dependency and use backon for exponential b…
kszucs Feb 11, 2026
c12b62e
chore(deps): align ctor version in xet
kszucs Feb 11, 2026
bcfad78
chore(deps): remove not unused tempfile dependency
kszucs Feb 11, 2026
84b05be
refactor(hf): use a more recent streaming xet client
kszucs Feb 11, 2026
57df7d5
chore(hf): remove duplicate deleter tests
kszucs Feb 11, 2026
6a253b3
chore(hf): remove essentially unused size field from HfWriter
kszucs Feb 11, 2026
2d037bf
chore(hf): update XetClient.write() call after removing progress upda…
kszucs Feb 11, 2026
72144cd
feat(hf): add support for buckets repository type
kszucs Feb 13, 2026
4020ff1
build(hf): switch to depend on subxet - a tree-shaken single crate ve…
kszucs Feb 13, 2026
e0e80ba
chore(hf): format cargo toml
kszucs Feb 13, 2026
444b80b
chore(hf): adjustments after rebase
kszucs Feb 18, 2026
c593a05
refactor(hf): make xet a mandatory dependency
kszucs Feb 21, 2026
2b39800
refactor(hf): remove the option to disable xet in runtime
kszucs Feb 21, 2026
7a727b6
chore(hf): more explicit error handling
kszucs Feb 21, 2026
8e6ba6c
chore(hf): remove redundant retry logic
kszucs Feb 21, 2026
466f6ef
refactor(services/hf): migrate from subxet to xet-core data/cas_clien…
kszucs Mar 16, 2026
8bc9a96
chore(hf): upgrade xet dependency to consolidated xet-data/xet-client…
kszucs Mar 16, 2026
91012ad
chore(hf): switch xet deps to git ref with range-stream support
kszucs Mar 17, 2026
e91739a
refactor(hf): cache TranslatorConfig instead of sessions, serialize i…
kszucs Mar 25, 2026
d9c778b
chore(hf): migrate to new XetSession API from xet-core PR #747
kszucs Mar 26, 2026
26aa471
chore(hf): upgrade xet-core to 126f30b9 with per-operation token API
kszucs Mar 31, 2026
0cff7d3
chore(hf): upgrade xet-core to 014ff2d7, remove XetSessionBuilder end…
kszucs Apr 3, 2026
d5a3565
chore(hf): sort Cargo.toml dependencies with taplo
kszucs Apr 3, 2026
4c70bd8
chore(hf): migrate hf-xet dependency from git to crates.io v1.5.0
kszucs Apr 4, 2026
82fda05
chore(hf): use crates.io hf-xet 1.5.0, clean up per-service tests, fi…
kszucs Apr 4, 2026
510de3d
feat(hf): always use XET for writes, fix bucket support, per-session …
kszucs Apr 5, 2026
46a3769
fix(hf): prefix-based listing, CAS propagation retry
kszucs Apr 5, 2026
23a0c20
fix(hf): write metadata version mismatch, increase CAS retry delays
kszucs Apr 5, 2026
00acb0f
fix(hf): enable SimulateLayer create_dir via write_can_empty
kszucs Apr 5, 2026
5c57e67
revert(hf): remove write_can_empty and trailing slash strip
kszucs Apr 5, 2026
3e1930c
fix(tests): gate create_dir-dependent list tests on create_dir capabi…
kszucs Apr 5, 2026
e22488a
ci(hf): rename service dir to hf, add ephemeral bucket and dataset co…
kszucs Apr 6, 2026
9359a7a
refactor(hf): move XET session management into HfCore
kszucs Apr 6, 2026
bec3e29
refactor(hf): rename commit_files/bucket_batch to commit_git/commit_b…
kszucs Apr 6, 2026
cae845a
refactor(hf): clean up writer, deleter, config
kszucs Apr 6, 2026
7479d3f
refactor(hf): remove custom retry logic from writer
kszucs Apr 6, 2026
2b58745
refactor(hf): add HfCore::is_bucket() helper
kszucs Apr 6, 2026
7d72d9e
refactor(hf): rename RepoType to HfRepoType, add docstrings
kszucs Apr 6, 2026
bf0faf5
refactor(hf): remove HfListerWrapper, use bucket tree API natively
kszucs Apr 6, 2026
26a4bc5
refactor(hf): tighten visibility, simplify writer, fix error format
kszucs Apr 6, 2026
2c76b11
fix(hf): support deletedFolders in git commits, move commit_url to Hf…
kszucs Apr 6, 2026
97d74a8
fix(ci): rename CI actions, use pre-created repos, fix test probe
kszucs Apr 6, 2026
a95a059
chore(hf): remove unused async-trait dep, move futures to dev-deps
kszucs Apr 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/services/hf/hf_bucket/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: hf_bucket
description: "Behavior test for Huggingface Bucket (read/write)"

runs:
using: "composite"
steps:
- name: Setup
uses: 1password/load-secrets-action@13f58eec611f8e5db52ec16247f58c508398f3e6 # v3
with:
export-env: true
env:
OPENDAL_HF_TOKEN: op://services/hf/token
OPENDAL_HF_REPO_ID: op://services/hf/bucket_repo_id
- name: Setup env
shell: bash
run: |
cat << EOF >> $GITHUB_ENV
OPENDAL_HF_REPO_TYPE=bucket
EOF
37 changes: 37 additions & 0 deletions .github/services/hf/hf_dataset/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: hf_dataset
description: "Behavior test for Huggingface Dataset (read/write)"

runs:
using: "composite"
steps:
- name: Setup
uses: 1password/load-secrets-action@13f58eec611f8e5db52ec16247f58c508398f3e6 # v3
with:
export-env: true
env:
OPENDAL_HF_TOKEN: op://services/hf/token
OPENDAL_HF_REPO_ID: op://services/hf/dataset_repo_id
- name: Setup env
shell: bash
run: |
cat << EOF >> $GITHUB_ENV
OPENDAL_HF_REPO_TYPE=dataset
RUST_TEST_THREADS=1
EOF
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ runs:
shell: bash
run: |
cat << EOF >> $GITHUB_ENV
OPENDAL_HUGGINGFACE_REPO_TYPE=dataset
OPENDAL_HUGGINGFACE_REPO_ID=opendal/huggingface-testdata
OPENDAL_HUGGINGFACE_REVISION=main
OPENDAL_HUGGINGFACE_ROOT=/testdata/
OPENDAL_HF_REPO_TYPE=dataset
OPENDAL_HF_REPO_ID=opendal/huggingface-testdata
OPENDAL_HF_REVISION=main
OPENDAL_HF_ROOT=/testdata/
OPENDAL_DISABLE_RANDOM_ROOT=true
EOF
Loading
Loading