Implmemnted Feature Windows Gameserver Move Alpha to beta - #4560
Implmemnted Feature Windows Gameserver Move Alpha to beta#4560Sivasankaran25 wants to merge 27 commits into
Conversation
|
Build Failed 😭 Build Id: 1d89798a-9fcf-422e-a2a2-c65b60892f56 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: e35ec88a-3848-4a70-8f87-3fe1b11a06e5 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 0894d8f3-6aa3-484e-8c51-b7d089f35370 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: f680848d-d334-4731-a973-ddd02c1184e4 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: 441eecfa-987c-407a-9c93-4dfaa26a41b6 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: c486f2ce-f194-47ec-a6cf-7a9112176a76 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 03abe090-4b08-4114-85dc-109f562c5d16 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
igooch
left a comment
There was a problem hiding this comment.
GKE Autopilot does not support Windows Nodes https://docs.cloud.google.com/kubernetes-engine/docs/concepts/windows-server-gke#limitations. Try running tests on Standard only.
|
Build Failed 😭 Build Id: d9561616-5ef7-4585-9398-81a4d7bce436 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: d899272a-a35a-4aed-86fa-6eb15696a675 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: c4b35f9c-0c5d-421a-a377-e514d5730e33 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Curious to hear how the work here is going? I can't see the clusters, but what is failing here? |
The |
|
Could we use something like what we did here to skip autopilot? agones/test/e2e/gameserver_test.go Line 263 in 00f9151 |
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | ||
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | ||
| # | ||
| - name: e2e-runner |
There was a problem hiding this comment.
I'll reiterate this - we have tests that only run on autopilot, why do we need this extra build step? This should just be a single extra test that runs on standard GKE clusters, that tests the windows nodes.
|
|
||
| gs := framework.DefaultGameServer(framework.Namespace) | ||
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | ||
| "kubernetes.io/os": "windows", |
| - started-by-${_PARENT_BUILD_ID} | ||
| timeout: 5400s # 1.5h | ||
| queueTtl: 7200s # 2h // only one set of e2es should be running at once | ||
| logsBucket: gs://agones-build-logs |
There was a problem hiding this comment.
Don't remove the log bucket - that's why I can't see the logs.
| # | ||
| # Run Windows GameServer smoke test. | ||
| # Windows nodes are NOT supported on GKE Autopilot: | ||
| # https://cloud.google.com/kubernetes-engine/docs/concepts/windows-server-gke#limitations | ||
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | ||
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | ||
| # | ||
| - name: e2e-runner | ||
| args: | ||
| - ${_FEATURE_WITHOUT_GATE} | ||
| - ${_CLOUD_PRODUCT} | ||
| - ${_TEST_CLUSTER_NAME} | ||
| - ${_TEST_CLUSTER_LOCATION} | ||
| - ${_REGISTRY} | ||
| - ${_GS_TEST_IMAGE} | ||
| id: e2e-windows-smoke | ||
| waitFor: | ||
| - build-e2e | ||
| timeout: 900s |
There was a problem hiding this comment.
| # | |
| # Run Windows GameServer smoke test. | |
| # Windows nodes are NOT supported on GKE Autopilot: | |
| # https://cloud.google.com/kubernetes-engine/docs/concepts/windows-server-gke#limitations | |
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | |
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | |
| # | |
| - name: e2e-runner | |
| args: | |
| - ${_FEATURE_WITHOUT_GATE} | |
| - ${_CLOUD_PRODUCT} | |
| - ${_TEST_CLUSTER_NAME} | |
| - ${_TEST_CLUSTER_LOCATION} | |
| - ${_REGISTRY} | |
| - ${_GS_TEST_IMAGE} | |
| id: e2e-windows-smoke | |
| waitFor: | |
| - build-e2e | |
| timeout: 900s | |
| timeout: 900s |
| func TestWindowsCreateConnect(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| if framework.CloudProduct == "gke-autopilot" { |
There was a problem hiding this comment.
Since you have this, it should run as part of the standard set of e2e tests -- no need for the extra build step.
|
Build Failed 😭 Build Id: 2d8351fd-4531-43fd-8c89-bfbe5c8192dd Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 1a5bebee-8115-4f71-a368-fb94b3d61bac Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
There was a problem hiding this comment.
Thanks for moving Windows GameServers to Beta! The docs and support-tier changes look right, and the Autopilot handling is correct — the test is skipped on Autopilot and the node pool is only added to the standard cluster.
However, TestWindowsCreateConnect will fail consistently on the standard e2e cluster (where it actually runs) for two reasons. Fixes are suggested inline, plus one blocker that lives outside this diff is that no Windows image is built for e2e:
cloudbuild.yaml:188 builds and pushes the e2e simple-game-server image with WITH_WINDOWS=0, so its manifest is linux/amd64-only (examples/simple-game-server/Makefile only adds the windows_amd64 layers when WITH_WINDOWS=1). The test comment claims framework.GameServerImage is multi-arch with a windows/amd64 variant — that isn't true in CI, so a Windows-pinned pod has no runnable image and never reaches Ready. This needs WITH_WINDOWS=1 for the e2e image build step:
- 'WITH_WINDOWS=1'Inline suggestions
- Add the Windows node toleration and prefer the
SkipOnCloudProducthelper (test file). - Set the Windows taint on the node pool and restore the missing trailing newline (
module.tf).
Nice work overall — once the e2e image includes a Windows variant and the pod can tolerate/schedule onto the Windows node, this should go green.
| if framework.CloudProduct == "gke-autopilot" { | ||
| t.Skip("Windows nodes are not supported on GKE Autopilot") | ||
| } | ||
|
|
||
| gs := framework.DefaultGameServer(framework.Namespace) | ||
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | ||
| "kubernetes.io/os": "windows", | ||
| } | ||
| // framework.GameServerImage is multi-arch (linux/amd64 + windows/amd64), | ||
| // so no separate Windows-tagged image is required - the correct | ||
| // platform variant is selected automatically based on the node. | ||
| gs.Spec.Template.Spec.Containers[0].Image = framework.GameServerImage |
There was a problem hiding this comment.
🔴 GKE taints Windows nodes node.kubernetes.io/os=windows:NoSchedule, but DefaultGameServer sets no toleration, so the pod can't schedule onto the Windows node. Also prefer the existing SkipOnCloudProduct helper, and drop the multi-arch comment — the e2e image is built with WITH_WINDOWS=0 (see summary), so GameServerImage has no windows/amd64 variant today.
| if framework.CloudProduct == "gke-autopilot" { | |
| t.Skip("Windows nodes are not supported on GKE Autopilot") | |
| } | |
| gs := framework.DefaultGameServer(framework.Namespace) | |
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | |
| "kubernetes.io/os": "windows", | |
| } | |
| // framework.GameServerImage is multi-arch (linux/amd64 + windows/amd64), | |
| // so no separate Windows-tagged image is required - the correct | |
| // platform variant is selected automatically based on the node. | |
| gs.Spec.Template.Spec.Containers[0].Image = framework.GameServerImage | |
| framework.SkipOnCloudProduct(t, "gke-autopilot", "Windows nodes are not supported on GKE Autopilot") | |
| gs := framework.DefaultGameServer(framework.Namespace) | |
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | |
| "kubernetes.io/os": "windows", | |
| } | |
| gs.Spec.Template.Spec.Tolerations = []corev1.Toleration{{ | |
| Key: "node.kubernetes.io/os", | |
| Operator: corev1.TolerationOpEqual, | |
| Value: "windows", | |
| Effect: corev1.TaintEffectNoSchedule, | |
| }} | |
| gs.Spec.Template.Spec.Containers[0].Image = framework.GameServerImage |
|
Build Failed 😭 Build Id: f6275ab8-4205-4849-a395-7f559c635004 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: 5915080e-be9a-40a2-bc48-888ba68fb9ab Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
@igooch , @Sivasankaran25 - we just waiting to create windows nodes on the clusters for this, yeah? |
|
Build Failed 😭 Build Id: 3ded919e-20ab-412a-b289-85efa96f4422 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: a1865ae1-8638-4150-b98b-768841b449a6 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Succeeded 🥳 Build Id: 5b5b7b94-0ea1-4148-950b-951e139a581d The following development artifacts have been built, and will exist for the next 30 days:
A preview of the website (the last 30 builds are retained): To install this version: |
|
Build Succeeded 🥳 Build Id: e56c63ea-ca3f-4650-9fcf-75080f48b3c4 The following development artifacts have been built, and will exist for the next 30 days:
A preview of the website (the last 30 builds are retained): To install this version: |
|
|
||
| The following container operating systems and architectures can be utilised with Agones: | ||
|
|
||
| {{% feature publishVersion="1.60.0" %}} |
There was a problem hiding this comment.
Since we've done a release, let's update these versions.
| t.Parallel() | ||
|
|
||
| framework.SkipOnCloudProduct(t, "gke-autopilot", "Windows nodes are not supported on GKE Autopilot") | ||
| skipIfNoWindowsNodes(t) |
There was a problem hiding this comment.
🤔 why do we want this? This feels like the sort of thing that would hide issues with testing the windows machines.
TBH, I didn't look, and assumed the tests where passing. I'd recommend removing this, because if we don't have windows nodes on a specific gke standard version, these tests should fail.
|
Build Succeeded 🥳 Build Id: 8a160c8d-620d-4803-9128-7fa6c9ea5c82 The following development artifacts have been built, and will exist for the next 30 days:
A preview of the website (the last 30 builds are retained): To install this version: |
|
Build Succeeded 🥳 Build Id: fb1c6e27-8c25-4e91-81f4-77478ab8c53b The following development artifacts have been built, and will exist for the next 30 days:
A preview of the website (the last 30 builds are retained): To install this version: |
|
Build Failed 😭 Build Id: ea98727e-f680-44b6-9c6f-273de6741ba9 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 8314269d-5d41-4d96-8d2f-834f5ef5e139 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
What type of PR is this?
/kind feature
What this PR does / Why we need it:
Which issue(s) this PR fixes:
Closes #4523
Special notes for your reviewer: