Plenty of new stuff - #51
Open
edelkas wants to merge 1164 commits into
Open
Conversation
edelkas
changed the title
…abstracted finding a player's PB and updating his rank
Minor changes to level names (for normalization) and data
Added a couple of fields to the mappack levels table to store the corresponding userlevel ID and whether it should be forwarded or not. This is intended for mappacks that contain already-published userlevels so they can be redirected to Metanet instead of storing a separate copy of the leaderboards locally. Added a method to automatically find the corresponding userlevel (by name), and also a way to compute distances between levels so that we can keep the best match found in case of a tie, as well as find non-exact matches to inspect them manually.
For mappack levels that are connected with a userlevel, I'm now forwarding new submitted scores to Metanet, as well as leaderboard requests. Replays get forwarded automatically already because their replay IDs aren't recognized. This has involved several subtleties that stem from the fact the game generates corrupt scores when submitting IL scores from within episode runs, which I've had to patch to ensure corrupt scores don't get pushed to Metanet boards. Also, this functionality requires to patch the requests before forwarding to Metanet, as well as patching the response before sending it back to the game (mainly to swap the level ID, but also perhaps the score, as mentioned). To aid in this, I've modified the Request class so that it's more easily editable, and now it no longer stores the original Rack env, which isn't really designed to be mutated.
Made some things a bit less strict. For instance, sheet names are now parsed via regex, and a few more errors are detected in advance. Also, exceptions are reported to Discord so I am alerted.
Arbitrary text can now be overlayed on the PNG screenshots. This requires ImageMagick since ChunkyPNG doesn't support text rendering, but it's costly to convert between both formats so we probably shouldn't do that for day-to-day screenshots. Used it for optionally showing bad matches in userlevels connected to mappack levels, so they can be compared with the naked eye, as an application.
The random function now returns a screenshot showcase of all selected samples, with their IDs overlayed.
Added a new table to store permanent score corrections that need to be applied to levels that were changed after release, specifically when gold was added or removed. This is also useful for mappack levels that differ from their userlevel counterparts only in unavoidable gold, so their scores can also be forwarded to Metanet after adjusting the scores suitably.
New slash command to browse the video library. In the process I noticed I was parsing challenge codes incorrectly, so I fixed it and made it more flexible in the process. Additionally, added a helper to find exact / partial / approximate matches in ActiveRecord classes, and another one to check if the event has been triggered by the botmaster.
When the userlevel report doesn't fit in a single message, it gets split in two pieces: one for the 0th rankings and another for the point rankings. Turns out when it gets splits I was ouputting the point rankings twice, but it had never happened until today.
On the occasion of the 2nd worldwide D-Day (Dalton Day), I've improved the command to gather the resulting mapping statistics, including adding a wrapper for graph creation, cleaning some of the graphs, fixing the bar plot, sending all attachments together, and being able to choose the year.
Assuming a mappack level with a corresponding userlevel version has only been changed by adding or removing gold, and assuming all added gold is forced and all removed gold was impossible, we can automatically calculate the score correction required to forward the scores to Metanet. The first condition can easily be checked, and we do so. The second one can't, so ONLY run this function when it's certain. In particular, filtering by date will help. Also added the ability to filter by inner ID, something necessary to prevent D-Day 2025 and D-Day 2026 level names from interferring with each other in the seeding process.
New version of D-Day UE mappack fixing two impossible gold levels. Added a function to disable forwarding manually, just for this case.
I was using the updated-at timestamp (instead of created-on), which caused each ticket to be reported after every new comment.
N+ games are now split by platform in Speedrun, which makes sense since they're different games, so we're tracking those now. Also improved logic so that new games work out of the box without needing to seed anything to the database initially like before. Group multiple embeds into a single message whenever possible.
Removed last usages of old APP_EMOJIS constants hardcoding app emoji values. Could be removed, but I'd rather keep it for later just in case. Added the trophy emojis for the newly added N+ (DS) version, and renamed N+ to N+ (XBLA).
Reordered official palettes in master file to match the baked order in the game, that way the palette index here is the palette ID in-game. Also improved the generator script in a number of ways: strictly parse official palettes only, robustized TGA parsing, and notably, a significant optimization in RLE decoding (avoid slicing, buffered reads, etc). Also improved logging. Also, I had been sampling the wrong pixel, the game actually uses pixel (32, 31) from each 64x64 block, I had (32, 32), which didn't really matter in practice.
Created a new db table for holding palettes, seeded with all 129 official ones from the master image. The corresponding class can parse the TGA files and also export them back, verified round-trip equality.
Added tons of information to the mappack digest so that it's actually useful for a potential tool: all the db fields, plus modes, tabs, score and player counts, and many more. Updated hourly automatically.
Instead of hardcoding the location of mappack files (levels, challenges, authors, scores) I've added a general getter that allows to retrieve the list of available files (simply as the list of paths), and another one that will actually read it. This is in order to support reorganizing the directory tree later, and the second getter is so that the files could even be stored compressed.
Turns out that the game stores TGA pixel data bottom-to-top, and samples the pixel (32, 32) this way. In general, if the image is 64 pixels high, as usual, this will be pixel (32, 31) from the top (i.e. what's conventionally shown by image editors), so what we had is mostly correct. But if the image isn't 64 pixels high the previous code breaks, because the pixel must be read from the bottom. Now I store data bottom-to-top.
Instead of keeping the raw files in the repo, they're all zipped now, mainly to improve downloading with the global tool later. Updated the digest accordingly.
The digest now includes an MD5 signature to prevent it from being updated when the core information hasn't changed, otherwise it'd be changing constantly and polluting our commit history. Additionally, I've clarified the download info so that it can be fetched by the tool automatically: download link, download size and md5 hash for integrity verification. The digest is now being pretty printed, which is useful mainly for the diffs. On top of that, I've corrected a few bugs here and there (different date formats, palettes were showing full path, etc). Finally, I've added a GitHub link creation function that abstracts that away. Used it in the digest and a few other places in the API.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is mainly a stylistic commit, but there's also a couple of minor functionalities added.
Main change
Added padding to name formatting and score formatting, so that most functions are aligned in columns, with two modes:
Smaller changes
Style
inne++ functions
Dev