feat(ci): add caching for CMake FetchContent in CI workflow#3448
feat(ci): add caching for CMake FetchContent in CI workflow#3448jihuayu merged 8 commits intoapache:unstablefrom
Conversation
PragmaTwice
left a comment
PragmaTwice
left a comment
There was a problem hiding this comment.
You can use DEPS_FETCH_DIR instead of caching these _deps directories (which may bring some chaos).
…or improved build process
|
@PragmaTwice Thanks for the reminder. I’ve made the changes. |
I don't think we need this. They are useless and introduce complexity.
I think we'd better to choose the simplest way. |
This variable is just for readability; otherwise, we’d be stuck facing a very long cache key. As I understand it, the Lua and jemalloc options must be cached separately because they pull different source codes. Is that correct? |
Nope. The dep dir can be shared. The filename is different for different deps. |
| prepare-deps-cache: | ||
| name: Prepare dependency cache | ||
| needs: [precondition] | ||
| if: ${{ needs.precondition.outputs.docs_only != 'true' }} | ||
| runs-on: ubuntu-22.04 | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Cache Kvrocks dependencies | ||
| id: cache-kvrocks-deps | ||
| uses: actions/cache@v5 | ||
| with: | ||
| path: ${{ env.KVROCKS_DEPS_CACHE_DIR }} | ||
| key: kvrocks-full-deps-${{ hashFiles('x.py', 'CMakeLists.txt', 'cmake/**/*.cmake') }} | ||
| enableCrossOsArchive: true | ||
| - name: Warm Kvrocks dependency cache | ||
| if: ${{ steps.cache-kvrocks-deps.outputs.cache-hit != 'true' }} | ||
| run: | | ||
| ./x.py build .github/ci-cache-default --skip-build --dep-dir ${{ env.KVROCKS_DEPS_CACHE_DIR }} | ||
| ./x.py build .github/ci-cache-lua --skip-build --dep-dir ${{ env.KVROCKS_DEPS_CACHE_DIR }} -D ENABLE_LUAJIT=OFF | ||
| rm -rf .github/ci-cache-default .github/ci-cache-lua |
There was a problem hiding this comment.
we don't need a job to "warm" the cache.. you can try --dep-dir on your local to know how it works.
There was a problem hiding this comment.
GitHub Actions caches are immutable. Once a cache is created for a specific key, it cannot be updated or overwritten.
So, we need to create all files intended for caching in a single step.
There was a problem hiding this comment.
ahh got it.
please put it in check-and-lint. also you can use ./x.py fetch-deps.
There was a problem hiding this comment.
All is right. Thank you!
|
2 participants
Due to the current instability of GitHub services, dependency download failures occur frequently during CI runs. I add FetchContent caching to prevent these network issues from failing the CI.