game-ci / versioning-backend Goto Github PK

https://github.com/marketplace/actions/github-action-for-firebase

This is so meta ahah

Due to recent changes in the Unity archive page format, our automated system for retrieving new Unity versions has been
unable to detect and publish the latest releases. This has caused delays in making new Unity versions available for the
community.

Affected Versions

All Unity versions published after May 16, 2024, including but not limited to:

• Unity 6000.0.3f1
• Unity 6000.0.4f1
• Unity 2022.3.30f1
• Unity 2022.3.31f1

Proposed Solution

• Option 1: We need to update our crawler to adapt to the new format of the Unity archive page.
• Option 2: We can consider switching to the Unity symbol server endpoints for metadata retrieval, which appear
  to have a more consistent format and are less likely to change.

Symbol Server Endpoints:

• http://symbolserver.unity3d.com/000Admin/server.txt
• http://symbolserver.unity3d.com/000Admin/history.txt

Both files seem to have the same modified date/time and may be used for scraping and parsing metadata.

How to Help

We welcome contributions from the community to help solve this issue. Here are some steps to get started:

1. Analyze the new format of the Unity archive page and the structure of the symbol server endpoints.
2. Update the crawler code to parse the new format or switch to the symbol server endpoints.
3. Test the updated crawler to ensure it correctly retrieves the latest Unity versions.
4. Submit a pull request with your changes.

Interim Solutions

While we work on resolving this issue, we recommend using the last images that were published to maintain your CI
workflows:

• Unity 6000.0.2f1
• Unity 2023.2.20f1
• Unity 2022.3.29f1
• Unity 2021.3.38f1

You can find the list of available versions here.

Technical details and files we might have to update

• functions/src/logic/ingestUnityVersions/scrapeVersions.ts
  This file contains the logic for scraping versions from the Unity archive page. We might have to update this file to
  adapt to the new format of the archive page or switch to using the Unity symbol server endpoints.
• functions/src/logic/ingestRepoVersions/index.ts
  This file initiates the scraping process and updates the database. We need to ensure it correctly handles any new data
  format.
• functions/src/cron/index.ts
  This file schedules the routine tasks, including the ingestion of Unity versions. Ensure it calls the updated
  ingestion function.

Recent unity versions are not being published to docker hub anymore due to our retry logic. Right now, Windows based images are failing due to game-ci/docker#154 and we are reaching the number of retries limit for a given unity version, which prevents other working targets to be published.

https://game.ci/docs/docker/versions

Here are the currently failed images:

It is still unclear what is going on with the retry logic because some recent images succeeded to build at least the ubuntu based images, but others didn't.

Example of success: editor-2021.2.9f1-0.17.0

This version doesn't seem to have hit the current issue

Example of failure: editor-2021.2.11f1-0.17.0

This specific version seem to have hit the retry logic problem.