Skip to content

Support FOSS tests running locally #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
nettle wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Support FOSS tests running locally #149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions .github/workflows/foss.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,17 @@ jobs:
uses: ./.github/platform_environment_setup/ubuntu

- name: Set Bazel version
run: echo ${{ matrix.bazel_version }} >> test/foss/templates/.bazelversion
run: echo ${{ matrix.bazel_version }} > .bazelversion

- name: Print versions
run: |
bazel version
which bazel
CodeChecker version
echo "[NOTE]: CodeChecker may find different analyzer binaries" \
"when invoking directly, or in bazel's sandbox environment!" \
"Be sure to double check during debugging."
CodeChecker analyzers

- name: Run Test On Opensource Projects
working-directory: test
Expand All @@ -83,7 +93,17 @@ jobs:
uses: ./.github/platform_environment_setup/rhel9

- name: Set Bazel version
run: echo ${{ matrix.bazel_version }} >> test/foss/templates/.bazelversion
run: echo ${{ matrix.bazel_version }} > .bazelversion

- name: Print versions
run: |
bazel version
which bazel
CodeChecker version
echo "[NOTE]: CodeChecker may find different analyzer binaries" \
"when invoking directly, or in bazel's sandbox environment!" \
"Be sure to double check during debugging."
CodeChecker analyzers

- name: Run Test On Opensource Projects
working-directory: test
Expand Down
8 changes: 6 additions & 2 deletions test/common/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,10 +135,14 @@ def run_command(
cwd=working_dir,
) as process:
stdout, stderr = process.communicate()
stdout = stdout.decode("utf-8")
stderr = stderr.decode("utf-8")
logging.debug("stdout: %s", stdout)
logging.debug("stderr: %s", stderr)
return (
process.returncode,
f"stdout: {stdout.decode('utf-8')}",
f"stderr: {stderr.decode('utf-8')}",
f"stdout: {stdout}",
f"stderr: {stderr}",
)

@classmethod
Expand Down
1 change: 0 additions & 1 deletion test/foss/templates/.bazelversion
View file
Open in desktop

This file was deleted.

10 changes: 9 additions & 1 deletion test/foss/yaml-cpp/init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,16 @@
# See the License for the specific language governing permissions and
# limitations under the License.

set -x

git clone --recurse https://github.com/jbeder/yaml-cpp.git test-proj
cd test-proj
git checkout yaml-cpp-0.7.0

# This file must be in the root of the project to be analyzed for bazelisk to work
cp ../../templates/.bazelversion ./.bazelversion
bazelversion="../../../../.bazelversion"
[ -f $bazelversion ] && cp $bazelversion .
cat .bazelversion
Comment on lines +24 to +26
Copy link
Contributor

@Szelethus Szelethus Jan 21, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer if this...

Copy link
Collaborator Author

@nettle nettle Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll address


# Add codechecker to the project
cat <<EOF >> BUILD.bazel
Expand Down Expand Up @@ -52,3 +56,7 @@ EOF

# Add codechecker_bazel repo to WORKSPACE
cat ../../templates/WORKSPACE.template >> WORKSPACE

# Check
bazel version
which bazel
Comment on lines +59 to +62
Copy link
Contributor

@Szelethus Szelethus Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...and this change were reused from test/foss/templates, and not copy-pasted to each test.

Comment on lines +61 to +62
Copy link
Contributor

@furtib furtib Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not make sense to me using the current test setup:

  • python3 -m unittest discover foss -vvv

This script only runs the first time, and its output is completely ignored.

Copy link
Collaborator Author

@nettle nettle Jan 21, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's another huge problem!
Once we run FOSS tests they create test-proj directories which are invisible, cannot be updated, and cannot be removed! I will create a bug about this :)

11 changes: 10 additions & 1 deletion test/foss/zlib-module/init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,17 @@
# See the License for the specific language governing permissions and
# limitations under the License.

set -x

git clone --recurse https://github.com/madler/zlib.git test-proj
cd test-proj
git checkout 5a82f71ed1dfc0bec044d9702463dbdf84ea3b71

# This file must be in the root of the project to be analyzed for bazelisk to work
cp ../../templates/.bazelversion ./.bazelversion
bazelversion="../../../../.bazelversion"
[ -f $bazelversion ] && cp $bazelversion .
cat .bazelversion

# Add codechecker to the project
cat <<EOF >> BUILD.bazel
#-------------------------------------------------------
Expand Down Expand Up @@ -51,3 +56,7 @@ EOF
echo "common --enable_bzlmod" > .bazelrc
# Add codechecker_bazel repo MODULE.bazel
cat ../../templates/MODULE.template >> MODULE.bazel

# Check
bazel version
which bazel
10 changes: 9 additions & 1 deletion test/foss/zlib/init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,16 @@
# See the License for the specific language governing permissions and
# limitations under the License.

set -x

git clone --recurse https://github.com/madler/zlib.git test-proj
cd test-proj
git checkout 5a82f71ed1dfc0bec044d9702463dbdf84ea3b71

# This file must be in the root of the project to be analyzed for bazelisk to work
cp ../../templates/.bazelversion ./.bazelversion
bazelversion="../../../../.bazelversion"
[ -f $bazelversion ] && cp $bazelversion .
cat .bazelversion

# Add codechecker to the project
cat <<EOF >> BUILD.bazel
Expand Down Expand Up @@ -52,3 +56,7 @@ EOF

# Add codechecker_bazel repo to WORKSPACE
cat ../../templates/WORKSPACE.template >> WORKSPACE

# Check
bazel version
which bazel
2 changes: 1 addition & 1 deletion test/unit/external_repository/test_external_repo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def setUpClass(cls):
logging.debug("No bazel version set, using system default")
_, stdout, _ = cls.run_command("bazel --version")
cls.BAZEL_VERSION = stdout.split(' ')[2].strip()
logging.debug("Using Bazel", cls.BAZEL_VERSION)
logging.debug("Using Bazel %s", cls.BAZEL_VERSION)
Copy link
Contributor

@Szelethus Szelethus Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch.


@final
@classmethod
Expand Down