K8SPG-834 add test for huge pages#1424
Conversation
There was a problem hiding this comment.
Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit
shfmt
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1581 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1583 to 1584 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1586 to 1589 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1591 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1593 to 1594 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1598 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1600 to 1605 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1607 to 1610 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1612 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1614 to 1615 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1617 to 1623 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1627 to 1629 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1631 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1633 to 1635 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1637 to 1638 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1640 to 1641 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1643 to 1649 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1653 to 1654 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1656 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1658 to 1660 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1662 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1664 to 1670 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1674 to 1676 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1678 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1680 to 1681 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1683 to 1684 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1686 to 1687 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1689 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1691 to 1694 in f7f78ad
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1696 to 1703 in f7f78ad
egegunes
left a comment
There was a problem hiding this comment.
@nmarukovich could you please apply shfmt suggestions? it's too hard to review right now.
yes, sure |
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
| } | ||
|
|
||
| enable_hugepages_eks() { | ||
| local hugepage_count=$1 |
|
|
||
| echo "Updating node pool with hugepages configuration..." | ||
|
|
||
| if gcloud container node-pools update ${node_pool} \ |
There was a problem hiding this comment.
I think we need to move it in Jenkins pipeline level
There was a problem hiding this comment.
Ye, I thought the same, just did it.
There was a problem hiding this comment.
Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit
shfmt
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1609 to 1610 in a579e0e
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Line 1612 in a579e0e
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1614 to 1617 in a579e0e
[shfmt] reported by reviewdog 🐶
percona-postgresql-operator/e2e-tests/functions
Lines 1619 to 1626 in a579e0e
| } | ||
|
|
||
| enable_hugepages_gke() { | ||
| local hugepage_count=$1 |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| local hugepage_count=$1 | |
| local hugepage_count=$1 |
| echo "Hugepages configuration is now handled during cluster creation" | ||
| echo "Skipping runtime configuration for GKE" |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| echo "Hugepages configuration is now handled during cluster creation" | |
| echo "Skipping runtime configuration for GKE" | |
| echo "Hugepages configuration is now handled during cluster creation" | |
| echo "Skipping runtime configuration for GKE" |
| echo "Verifying hugepages configuration..." | ||
| local node_name=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| echo "Verifying hugepages configuration..." | |
| local node_name=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') | |
| echo "Verifying hugepages configuration..." | |
| local node_name=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') |
| if kubectl get node ${node_name} -o jsonpath='{.status.allocatable.hugepages-2Mi}' | grep -q '[0-9]'; then | ||
| echo "✓ Hugepages are already configured on nodes" | ||
| return 0 | ||
| else | ||
| echo "✗ Warning: Hugepages not found in node allocatable resources" | ||
| return 1 | ||
| fi |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| if kubectl get node ${node_name} -o jsonpath='{.status.allocatable.hugepages-2Mi}' | grep -q '[0-9]'; then | |
| echo "✓ Hugepages are already configured on nodes" | |
| return 0 | |
| else | |
| echo "✗ Warning: Hugepages not found in node allocatable resources" | |
| return 1 | |
| fi | |
| if kubectl get node ${node_name} -o jsonpath='{.status.allocatable.hugepages-2Mi}' | grep -q '[0-9]'; then | |
| echo "✓ Hugepages are already configured on nodes" | |
| return 0 | |
| else | |
| echo "✗ Warning: Hugepages not found in node allocatable resources" | |
| return 1 | |
| fi |
| path: /etc/sysctl.d/99-hugepages.conf | ||
| EOF | ||
|
|
||
| [ $? -eq 0 ] || return 1 |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| [ $? -eq 0 ] || return 1 | |
| [ $? -eq 0 ] || return 1 |
| if [[ "${huge_pages}" == *"${expected_value}"* ]]; then | ||
| echo "PostgreSQL huge_pages is set to '${expected_value}'" | ||
| return 0 | ||
| else | ||
| echo "PostgreSQL huge_pages not set to '${expected_value}' (value: ${huge_pages})" | ||
| return 1 | ||
| fi |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| if [[ "${huge_pages}" == *"${expected_value}"* ]]; then | |
| echo "PostgreSQL huge_pages is set to '${expected_value}'" | |
| return 0 | |
| else | |
| echo "PostgreSQL huge_pages not set to '${expected_value}' (value: ${huge_pages})" | |
| return 1 | |
| fi | |
| if [[ ${huge_pages} == *"${expected_value}"* ]]; then | |
| echo "PostgreSQL huge_pages is set to '${expected_value}'" | |
| return 0 | |
| else | |
| echo "PostgreSQL huge_pages not set to '${expected_value}' (value: ${huge_pages})" | |
| return 1 | |
| fi |
| local pod_name=$1 | ||
| local namespace=$2 | ||
| local container=${3:-database} |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| local pod_name=$1 | |
| local namespace=$2 | |
| local container=${3:-database} | |
| local pod_name=$1 | |
| local namespace=$2 | |
| local container=${3:-database} |
| local namespace=$2 | ||
| local container=${3:-database} | ||
|
|
||
| echo "Checking hugepages usage..." |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| echo "Checking hugepages usage..." | |
| echo "Checking hugepages usage..." |
| kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | ||
| grep HugePages /proc/meminfo |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | |
| grep HugePages /proc/meminfo | |
| kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | |
| grep HugePages /proc/meminfo |
| local hugepages_total=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | ||
| grep HugePages_Total /proc/meminfo | awk '{print $2}') |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| local hugepages_total=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | |
| grep HugePages_Total /proc/meminfo | awk '{print $2}') | |
| local hugepages_total=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | |
| grep HugePages_Total /proc/meminfo | awk '{print $2}') |
| local hugepages_free=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | ||
| grep HugePages_Free /proc/meminfo | awk '{print $2}') |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| local hugepages_free=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | |
| grep HugePages_Free /proc/meminfo | awk '{print $2}') | |
| local hugepages_free=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | |
| grep HugePages_Free /proc/meminfo | awk '{print $2}') |
| local hugepages_free=$(kubectl -n ${namespace} exec ${pod_name} -c ${container} -- \ | ||
| grep HugePages_Free /proc/meminfo | awk '{print $2}') | ||
|
|
||
| local hugepages_used=$((hugepages_total - hugepages_free)) |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| local hugepages_used=$((hugepages_total - hugepages_free)) | |
| local hugepages_used=$((hugepages_total - hugepages_free)) |
| echo "" | ||
| echo "HugePages usage:" | ||
| echo " Total: ${hugepages_total}" | ||
| echo " Used: ${hugepages_used}" |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| echo "" | |
| echo "HugePages usage:" | |
| echo " Total: ${hugepages_total}" | |
| echo " Used: ${hugepages_used}" | |
| echo "" | |
| echo "HugePages usage:" | |
| echo " Total: ${hugepages_total}" | |
| echo " Used: ${hugepages_used}" |
| if [ "${hugepages_used}" -gt 0 ]; then | ||
| echo "PostgreSQL is using hugepages" | ||
| return 0 | ||
| else | ||
| echo "Hugepages available but NOT being used by PostgreSQL" | ||
| return 1 | ||
| fi | ||
| } No newline at end of file |
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
| if [ "${hugepages_used}" -gt 0 ]; then | |
| echo "PostgreSQL is using hugepages" | |
| return 0 | |
| else | |
| echo "Hugepages available but NOT being used by PostgreSQL" | |
| return 1 | |
| fi | |
| } | |
| if [ "${hugepages_used}" -gt 0 ]; then | |
| echo "PostgreSQL is using hugepages" | |
| return 0 | |
| else | |
| echo "Hugepages available but NOT being used by PostgreSQL" | |
| return 1 | |
| fi | |
| } | |
| --monitoring=NONE \ | ||
| --logging=NONE \ | ||
| --no-enable-managed-prometheus \ | ||
| --system-config-from-file=${WORKSPACE}/hugepages-config-${CLUSTER_SUFFIX}.yaml \ |
There was a problem hiding this comment.
@nmarukovich please make sure that this will be added for all needed jobs
There was a problem hiding this comment.
you mean in jenkins? Percona-Lab/jenkins-pipelines#3785 we already have PR
commit: 72f165f |
mayankshah1607
left a comment
There was a problem hiding this comment.
LGTM, but please check the formatting
Due to the high volume of requests, we're unable to provide free service for this account. To continue using the service, please upgarde to a paid plan.
CHANGEDESCRIPTION
Problem:
Short explanation of the problem.
E2e test for huge pages was added.
Cause:
Short explanation of the root cause of the issue if applicable.
Solution:
Short explanation of the solution we are providing with this PR.
CHECKLIST
Jira
Needs Doc) and QA (Needs QA)?Tests
Config/Logging/Testability