feat: add support for hugepages_config #2223

DrFaust92 · 2024-12-21T18:18:58Z

Closes #2320

apeabody · 2024-12-23T17:44:52Z

/gcbrun

apeabody

Thanks for the contribution @DrFaust92!

Can you please add to an example for test coverage, otherwise other than some extra spaces, this LGTM.

NOTE: hugepages_config added in v6.5.0

autogen/main/cluster.tf.tmpl

DrFaust92 · 2025-01-02T20:01:51Z

@apeabody Tried my best with tests here as well 😊 . LMK if something needs fixing

apeabody · 2025-01-02T23:14:53Z

/gcbrun

apeabody · 2025-01-02T23:49:53Z

FYI @DrFaust92 - A few of the pools do use n1, but the rest are currently e2.

TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(e2-medium) does not support this configuration.
TestNodePool 2025-01-02T23:41:57Z command.go:185: Details:
TestNodePool 2025-01-02T23:41:57Z command.go:185: [
TestNodePool 2025-01-02T23:41:57Z command.go:185:   {
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "@type": "type.googleapis.com/google.rpc.RequestInfo",
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "requestId": "0x7b26367c30d761e2"
TestNodePool 2025-01-02T23:41:57Z command.go:185:   }
TestNodePool 2025-01-02T23:41:57Z command.go:185: ]
TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185:   with module.example.module.gke.google_container_node_pool.pools["pool-04"],
TestNodePool 2025-01-02T23:41:57Z command.go:185:   on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools":
TestNodePool 2025-01-02T23:41:57Z command.go:185:  597: resource "google_container_node_pool" "pools" {
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(e2-medium) does not support this configuration.
TestNodePool 2025-01-02T23:41:57Z command.go:185: Details:
TestNodePool 2025-01-02T23:41:57Z command.go:185: [
TestNodePool 2025-01-02T23:41:57Z command.go:185:   {
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "@type": "type.googleapis.com/google.rpc.RequestInfo",
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "requestId": "0x48e2d757411a4297"
TestNodePool 2025-01-02T23:41:57Z command.go:185:   }
TestNodePool 2025-01-02T23:41:57Z command.go:185: ]
TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185:   with module.example.module.gke.google_container_node_pool.pools["pool-01"],
TestNodePool 2025-01-02T23:41:57Z command.go:185:   on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools":
TestNodePool 2025-01-02T23:41:57Z command.go:185:  597: resource "google_container_node_pool" "pools" {
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(n1-standard-2) does not support this configuration.
TestNodePool 2025-01-02T23:41:57Z command.go:185: Details:
TestNodePool 2025-01-02T23:41:57Z command.go:185: [
TestNodePool 2025-01-02T23:41:57Z command.go:185:   {
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "@type": "type.googleapis.com/google.rpc.RequestInfo",
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "requestId": "0x16e561c8b65549ca"
TestNodePool 2025-01-02T23:41:57Z command.go:185:   }
TestNodePool 2025-01-02T23:41:57Z command.go:185: ]
TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185:   with module.example.module.gke.google_container_node_pool.pools["pool-03"],
TestNodePool 2025-01-02T23:41:57Z command.go:185:   on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools":
TestNodePool 2025-01-02T23:41:57Z command.go:185:  597: resource "google_container_node_pool" "pools" {
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(n1-standard-2) does not support this configuration.
TestNodePool 2025-01-02T23:41:57Z command.go:185: Details:
TestNodePool 2025-01-02T23:41:57Z command.go:185: [
TestNodePool 2025-01-02T23:41:57Z command.go:185:   {
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "@type": "type.googleapis.com/google.rpc.RequestInfo",
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "requestId": "0x40383fdeb6fa524a"
TestNodePool 2025-01-02T23:41:57Z command.go:185:   }
TestNodePool 2025-01-02T23:41:57Z command.go:185: ]
TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185:   with module.example.module.gke.google_container_node_pool.pools["pool-05"],
TestNodePool 2025-01-02T23:41:57Z command.go:185:   on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools":
TestNodePool 2025-01-02T23:41:57Z command.go:185:  597: resource "google_container_node_pool" "pools" {
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(n1-standard-2) does not support this configuration.
TestNodePool 2025-01-02T23:41:57Z command.go:185: Details:
TestNodePool 2025-01-02T23:41:57Z command.go:185: [
TestNodePool 2025-01-02T23:41:57Z command.go:185:   {
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "@type": "type.googleapis.com/google.rpc.RequestInfo",
TestNodePool 2025-01-02T23:41:57Z command.go:185:     "requestId": "0x37ecaeacf733a8ea"
TestNodePool 2025-01-02T23:41:57Z command.go:185:   }
TestNodePool 2025-01-02T23:41:57Z command.go:185: ]
TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z command.go:185:   with module.example.module.gke.google_container_node_pool.pools["pool-02"],
TestNodePool 2025-01-02T23:41:57Z command.go:185:   on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools":
TestNodePool 2025-01-02T23:41:57Z command.go:185:  597: resource "google_container_node_pool" "pools" {
TestNodePool 2025-01-02T23:41:57Z command.go:185: 
TestNodePool 2025-01-02T23:41:57Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1;

DrFaust92 · 2025-01-03T00:23:22Z

@apeabody should i change the node pools types to support the test case? or how should I approach this?

apeabody · 2025-01-03T00:33:13Z

@apeabody should i change the node pools types to support the test case? or how should I approach this?

The 2-m huge pages should be fine as-is, however for the 1-g perhaps just test a single node pool which has a supported machine type. Maybe change "pool-05" to c3-standard-4?

1-gigabtye-sized huge pages are only available on the A3, C2D, C3, C3A, C3D, C4, CT5E, CT5L, CT5LP, CT6E, H3, M2, M3, or Z3 [machine types](https://cloud.google.com/compute/docs/machine-resource).

DrFaust92 · 2025-01-04T23:04:01Z

apeabody lets try now

apeabody · 2025-01-06T16:24:44Z

/gcbrun

autogen/main/cluster.tf.tmpl

apeabody · 2025-04-03T15:25:03Z

/gcbrun

DrFaust92 · 2025-04-07T03:18:27Z

apeabody added try as requested

rawanbadawi · 2025-04-07T17:10:10Z

@apeabody My customer needs this feature, is it possible to approve the latest?

Thanks

apeabody · 2025-04-07T17:17:18Z

/gcbrun

apeabody · 2025-04-07T19:04:23Z

INT test results:

        	            	Error: Error in function call
        	            	
        	            	  on ../../../modules/beta-public-cluster/cluster.tf line 938, in resource "google_container_node_pool" "pools":
        	            	 938:             hugepage_size_1g = try(coalesce(local.node_pools_hugepage_size_1g[each.value["name"]], local.node_pools_hugepage_size_1g["all"], null))
        	            	    ├────────────────
        	            	    │ each.value["name"] is "pool-03"
        	            	    │ local.node_pools_hugepage_size_1g is object with 8 attributes
        	            	    │ local.node_pools_hugepage_size_1g["all"] is ""
        	            	
        	            	Call to function "try" failed: no expression succeeded:
        	            	- Error in function call (at
        	            	../../../modules/beta-public-cluster/cluster.tf:938,36-45)
        	            	  Call to function "coalesce" failed: no non-null, non-empty-string arguments.
        	            	
        	            	At least one expression must produce a successful result

Looks like try() can't be used in this situation, probably need to use a less elegant ternary, etc.

DrFaust92 · 2025-04-25T23:29:17Z

local plans looks ok 🙏

apeabody · 2025-04-25T23:33:13Z

/gcbrun

DrFaust92 · 2025-04-26T00:08:04Z

This seemed to have failed fast /: . my ability to test this is limited as i can at most check this as a side branch on a running cluster

apeabody · 2025-04-26T00:11:44Z

This seemed to have failed fast /: . my ability to test this is limited as i can at most check this as a side branch on a running cluster

Flaky run, an API failed to active in time. Re-running

DrFaust92 · 2025-04-29T01:34:28Z

apeabody any details?

apeabody · 2025-04-29T15:59:15Z

apeabody any details?

Looks like need to adjust the test:

Step #54 - "apply node-pool-local":         	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/[email protected]/modules/terraform/apply.go:34
Step #54 - "apply node-pool-local":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:571
Step #54 - "apply node-pool-local":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:630
Step #54 - "apply node-pool-local":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:669
Step #54 - "apply node-pool-local":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/utils/stages.go:31
Step #54 - "apply node-pool-local":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:669
Step #54 - "apply node-pool-local":         	Error:      	Received unexpected error:
Step #54 - "apply node-pool-local":         	            	FatalError{Underlying: error while running command: exit status 1; 
Step #54 - "apply node-pool-local":         	            	Error: error creating NodePool: googleapi: Error 400: 'pd-standard' with confidential storage mode: false is not compatible with the machine type c3-standard-4, please review the GCP online documentation for available persistent disk options.
Step #54 - "apply node-pool-local":         	            	Details:
Step #54 - "apply node-pool-local":         	            	[
Step #54 - "apply node-pool-local":         	            	  {
Step #54 - "apply node-pool-local":         	            	    "@type": "type.googleapis.com/google.rpc.RequestInfo",
Step #54 - "apply node-pool-local":         	            	    "requestId": "0x16c806feae5789fb"
Step #54 - "apply node-pool-local":         	            	  }
Step #54 - "apply node-pool-local":         	            	]
Step #54 - "apply node-pool-local":         	            	, badRequest
Step #54 - "apply node-pool-local":         	            	
Step #54 - "apply node-pool-local":         	            	  with module.example.module.gke.google_container_node_pool.pools["pool-05"],
Step #54 - "apply node-pool-local":         	            	  on ../../../modules/beta-public-cluster/cluster.tf line 644, in resource "google_container_node_pool" "pools":
Step #54 - "apply node-pool-local":         	            	 644: resource "google_container_node_pool" "pools" {
Step #54 - "apply node-pool-local":         	            	}

Signed-off-by: drfaust92 <[email protected]>

DrFaust92 · 2025-05-05T23:34:39Z

apeabody Ive attempted to make changes to accommodate this node type

DrFaust92 requested review from ericyz and a team as code owners December 21, 2024 18:18

DrFaust92 changed the title ~~feat: add support for hugepages_config + fix setting cgroup_name via "all""~~ feat: add support for hugepages_config Dec 21, 2024

apeabody requested changes Dec 24, 2024

View reviewed changes

autogen/main/cluster.tf.tmpl Outdated Show resolved Hide resolved

autogen/main/cluster.tf.tmpl Outdated Show resolved Hide resolved

apeabody self-assigned this Dec 24, 2024

DrFaust92 requested a review from apeabody January 2, 2025 15:19

DrFaust92 force-pushed the huge_pages branch from dcca780 to 580a3fb Compare January 2, 2025 20:42

DrFaust92 force-pushed the huge_pages branch 2 times, most recently from 99c20b9 to 51a780f Compare January 4, 2025 23:03

apeabody reviewed Jan 6, 2025

View reviewed changes

autogen/main/cluster.tf.tmpl Outdated Show resolved Hide resolved

apeabody removed their assignment Feb 3, 2025

DrFaust92 force-pushed the huge_pages branch from 51a780f to c629490 Compare February 27, 2025 02:50

DrFaust92 force-pushed the huge_pages branch from d380ab4 to f38654d Compare April 7, 2025 03:06

DrFaust92 requested a review from apeabody April 7, 2025 03:18

DrFaust92 mentioned this pull request Apr 7, 2025

feat(Hugepages_config): Support hugepages configuration in linux nodepools #2322

Closed

DrFaust92 marked this pull request as draft April 9, 2025 02:39

DrFaust92 force-pushed the huge_pages branch from 13c39f5 to 6ccbeb5 Compare April 25, 2025 23:24

DrFaust92 marked this pull request as ready for review April 25, 2025 23:28

DrFaust92 added 6 commits May 5, 2025 19:19

support setting hugepages config

8f38b8f

Signed-off-by: drfaust92 <[email protected]>

align tests

65a6cb0

Signed-off-by: drfaust92 <[email protected]>

CR comments

2a3fcfa

Signed-off-by: drfaust92 <[email protected]>

huge-pages

e6e7e27

Signed-off-by: drfaust92 <[email protected]>

try try()

9039d8c

Signed-off-by: drfaust92 <[email protected]>

attempt to fix test

9a4a35f

Signed-off-by: drfaust92 <[email protected]>

DrFaust92 force-pushed the huge_pages branch from 6ccbeb5 to 9a4a35f Compare May 5, 2025 23:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for hugepages_config #2223

feat: add support for hugepages_config #2223

DrFaust92 commented Dec 21, 2024 •

edited

Loading

apeabody commented Dec 23, 2024

apeabody left a comment

DrFaust92 commented Jan 2, 2025

apeabody commented Jan 2, 2025

apeabody commented Jan 2, 2025

DrFaust92 commented Jan 3, 2025

apeabody commented Jan 3, 2025

DrFaust92 commented Jan 4, 2025

apeabody commented Jan 6, 2025

apeabody commented Apr 3, 2025

DrFaust92 commented Apr 7, 2025

rawanbadawi commented Apr 7, 2025

apeabody commented Apr 7, 2025

apeabody commented Apr 7, 2025 •

edited

Loading

DrFaust92 commented Apr 25, 2025

apeabody commented Apr 25, 2025

DrFaust92 commented Apr 26, 2025

apeabody commented Apr 26, 2025

DrFaust92 commented Apr 29, 2025

apeabody commented Apr 29, 2025

DrFaust92 commented May 5, 2025

feat: add support for hugepages_config #2223

Are you sure you want to change the base?

feat: add support for hugepages_config #2223

Conversation

DrFaust92 commented Dec 21, 2024 • edited Loading

apeabody commented Dec 23, 2024

apeabody left a comment

Choose a reason for hiding this comment

DrFaust92 commented Jan 2, 2025

apeabody commented Jan 2, 2025

apeabody commented Jan 2, 2025

DrFaust92 commented Jan 3, 2025

apeabody commented Jan 3, 2025

DrFaust92 commented Jan 4, 2025

apeabody commented Jan 6, 2025

apeabody commented Apr 3, 2025

DrFaust92 commented Apr 7, 2025

rawanbadawi commented Apr 7, 2025

apeabody commented Apr 7, 2025

apeabody commented Apr 7, 2025 • edited Loading

DrFaust92 commented Apr 25, 2025

apeabody commented Apr 25, 2025

DrFaust92 commented Apr 26, 2025

apeabody commented Apr 26, 2025

DrFaust92 commented Apr 29, 2025

apeabody commented Apr 29, 2025

DrFaust92 commented May 5, 2025

DrFaust92 commented Dec 21, 2024 •

edited

Loading

apeabody commented Apr 7, 2025 •

edited

Loading