rlbuddy
- 43 Devlogs
- 182 Total hours
A rocket league companion app
A rocket league companion app
A few fixes here and there. Some refactors, caching externally loaded playlists in savedata, and fixing a deadlock when clearing the items cache. And for some reason the decal option was missing from the items list picker.
I made a standalone web-based swapper for the people who don’t want to download an entire app. A lot of people had broken packages after the last update but didn’t want to install a dedicated program just for a few swaps. So, I used the asset swapper backend of rlbuddy to make this thingamabob which runs entirely in the browser!
First, I had to find out how to actually share rlbuddy’s asset swapper with other projects. Turns out Cargo has a cool thing called workspaces where you can have multiple crates in the same repo! So I moved the dedicated UPK/AES etc. stuff into a new crate and added it as a dependency in regular rlbuddy.
Then, I whipped up a small wasm-compatible eframe app with the template, and included the new crate as a dependency. I made a big state machine of the steps meaning it’s impossible to have an invalid state, which is really nice. The only problem was that Rust’s ownership system isn’t smart enough to deal with it, so the code is a bit repetitive in places but it’s definitely worth it.
The WASM restrictions were super annoying, mainly do to with filesystem access and requiring async. I ended up using the egui-async crate and spilling Binds all over the place within the state machine. I couldn’t even figure out how to get the actual swap procedure to run in a different thread.
I also spent like 30 minutes debugging why in the world asset swapping was just broken. Even when I verified the same binary data were going in to the Upk object in the webapp and rlbuddy, for some reason the output upk was just different. Turns out I used chosen_replacement.name instead of chosen_replacement.package.name(). Isn’t that great? Anyways, it works fine now and I’m planning on posting it on r/RocketLeagueMods and possible r/RocketLeague.
Not much in terms of super-cool stuff this time. Highlights are:
So no ultra-exciting things to talk about. anyhow is really cool though, it really simplifies error handling everywhere in the app, and I’m a big fan of its bail! macro. I’m also using Cow a lot more. Moo!
I got licensee version 34 (encrypted payloads too) to swap finally!
Okay, so this was awesome. Rocket League’s been having a bunch of updates lately, which breaks the packages of people trying to get free stuff. So, in a Reddit thread, I ended up dishing out a bunch of swapped asset files for people in this situation. There’s a new boost which everybody wanted really bad since it was really rare (but they decided to make it unlockable at rocket pass lv99 so it doesn’t even matter anymore), and it was UNSWAPPABLE for three reasons:
I’ll start with 3. I had written a tool which looks in the games cache for AES keys, and dumped them, and it worked great. What I didn’t expect is the filename to change between versions. So, I was super confused since yesterday since the AES key for the boost was just… missing. Thankfully, the geniuses at the RLMM discord got it and helped me figure out that the file I was reading from was just outdated. Great!
Anyways, not too big of a deal, just a big delay. Now it’s time for the real challenge: figuring out how the hell to handle this type of package.
First, I did some research. The twelve bytes of padding in files with licensee version >=33 before the summary are actually a nonce/aes iv, not just padding. Then, the header is encrypted with AES-CTR with the twelve first bytes of the iv just being the nonce. And the compressed chunks are encrypted too here: each one is also AES-CTRed with their own nonces in FCompressedChunkInfo.
Thing is, I don’t have an implementation to really go off anymore, meaning there’s a lot more trial and error. For the original implementation, there were one or two existing asset swappers (though they were vibecoded meaning more of a puzzle to figure out what they’re even doing), but this time I was mostly relying on emii’s fork of UEViewer. Anyways, after spending a while figuring out what exactly I had to masquerade (payload, key, swap nonces) I finally got it to work!
rlbuddy now loads additional playlists straight from the game’s cached webconfig, instead of hardcoding them all! This means that the list of playlists will basically never go out of date.
There are actually quite a few services which rely on game files (eg. auto stats api setup, custom maps, asset swapper). So, the first thing I did was to refactor it so make all of these load from the same global value.
Then, for playlists, I made it read from a specific file in the game’s http cache directory which contains a bunch of configuration. Then, it parses it into a JSON object, grabs all the values with Class PlaylistSettings_TA, and then extracts the title and playlist id!
There are still a bunch of built-in Playlists, since those have internal things like going from a casual playlist type to its respective competitive one or having the hard-coded list of singleplayer playlists. But, all the other ones, like ltms are dynamically loaded. Yay!
I made a ton of improvements to the asset swapper! And fixed a bug with the music player which is cool technically, so I’ll start with that. The player uses the GSMTC api, which is async, but egui doesn’t really like async. So, I made it so that every time you wanted to run some command on the current session, it would start up a new thread. This was less than ideal for a few reasons:
The main problem was 2: for example, if you skipped a replay too fast, the Play command from the end of the replay might actually reach Windows before the Pause command from the replay start, leaving your music paused.
So instead, it starts one dedicated worker thread which has a channel which receives a Box<dyn FnOnce(Session)>, and executes at most one command every 50ms (only one command per loop, fixed loop time of 50ms). It’s really interesting because you actually end up sending a closure through a channel, which feels funny but when you really think about it it’s just ownership semantics. But, that’s working great now!
On to swapper improvements! First, I moved most of the swapper module to anyhow handling (it’s heaven!), and I also modernized the upk api a bit since it was basically dragged directly from the proof-of-concept repo, meaning it was not really application-ready. Also, it previously looked at the game’s webcache to decode the AES keys for header decryption, but now it uses the wonderful Toga-Files repository along with a really nice caching layer for it to load items and their associated API keys.
Having access to the actual item data is also way better since you don’t have to manually type in a package name which is kinda weird for any normal person. Now, you can select it from a dropdown instead! I love UX.
rlbuddy now has item swapping! Basically, making one item appear like another to use items you don’t actually own. These hours were spent on a few different things. Firstly, I optimized the thing a bunch! There were a few places that were pretty slow since the original repo was mainly for testing. For example instead of decrypting the entire header and rechecking it to find the right key, it only loads the first 4 blocks and attempts to decrypt the first FString. If it fails, that means the AES key isn’t valid. Also, I improved the error handling with anyhow. It actually makes it so much easier to deal with things and was pretty useful when debugging the move over to the actual app. Now I’m just going to work on polishing the panel UI a bit more, since it’s still kind of ugly.
Apparently there are a bunch of different AES keys rocket league uses, not just one. You can get a list of them from the psynet config url, and then do some weird crc stuff to change them from “content keys” to aes keys. I discovered the rocket league mapmaking discord today and I’m blown away by the amount of information on there, and also apparently my whole thing about the unknown bytes when licensee version > 32 was straight up wrong, which is funny.
Anyways it works for all items now, which is great. I’m gonna work on actually putting it in rlbuddy now, hopefully that goes well
IT WORKS WHEN THE HEADER EXPANDS NOW YAYAYYAYAYAYYAYAYYAYAYAY
I’m gonna be honest, the problem might just have been that my testing package was corrupted. I was debugging my deserialization logic, which I thought might have some parsing issue near the header extraction, and noticed that once the encrypted region deserializer’s Cursor reached the end of the region, it was actually FARTHER than the package summary’s total_header_size - by 6 bytes! That’s pretty bad, and I don’t know how something like that even happened. Anyways, I repaired the game and brought over the shiny new upk and… it worked.
I’m pretty sure that I only accidentally did this while tidying up my workspace a few hours ago, but if this has genuinely been the thing holding me back for the past few days -
I also finally made the poc code public, at https://github.com/kalilamodow/rl-upk-testing/
IT SWAPPED IT SWAPPED IT SWAPPED AHAHAHHAHAHAHHAHAHAHAHHA FINALLY
Okay now I gotta make it work for other pairs of boosts. If WHEN that works, I’ll actually put it in rlbuddy.
I removed all the code relating to changing offsets to try to give myself a minimal workspace to tinker in, with as little side effects as possible. So, all the current code does is 1. open them 2. replace the name table entries and package guid. Surprisingly simple. Once I make it a bit more generic, I can extend it to support more swaps.
If I’m really unable to get it to work for more combinations, I’ll probably end up just publishing the poc repo and not adding swapping to rlbuddy at all.
This is a great motivation boost, though. I was honestly dreading coming home to work on it today because I was done with failing over and over, and considering giving up.
I got a name swap! It doesn’t work properly, but I’m glad the game didn’t crash. These hours have been by far the most frustrating because the only way to test is against the real game since it’s really picky, but the game takes like 30 seconds to start, and if it crashes, it gives a cryptic error log, and if it freezes, it doesn’t give any log at all. I spent most of the time messing with moving different offset values forward and backwards, and ended up reverting to a simpler version with no overwriting when the header size changes just to test things.
So far, I’ve mostly gleaned that name renaming is possible, but only up to a point. If the name grows too much, the game crashes, and if the name shrinks too much, the game freezes. Additionally, for some reason, total_header_size changes by the padded header size instead of the encrypted header size, which I still don’t understand.
It’s seriously weird. I serialize each name properly, and yet the game reads it wrong. This game really does need a new codebase. When UE6 comes out, either the game’s gonna be so good it doesn’t need mods, or it’ll be relatively easily moddable. Or neither, which is the most probable based on Epic Games’ track record.
Anyways, enjoy the beautiful screenshot.
I figured out what comes after export tables! And it took way too long, since I DISCOVERED IT MYSELF! At least, as far as I know. I couldn’t find any information online about this portion of the header, so it was basically luck that I noticed a pattern. Before, I was just copying everything after the export table and pasting them back in afterwards, but as you can guess, that stopped working if I adjusted anything within the encrypted region.
Compressed chunk info is a known part, and I figured out the AES padding pretty easily, so most of the time was spent trying to figure out what in the world those 24 bytes were.
The first four bytes in the compressed chunk info are how many chunks they are. Here, in the Bubbles boost package, there are two (little endian), with 24 bytes of ???. And in Flamethrower, there are three, with 36 bytes of ???.
I noticed that the ??? was always n chunks * 12 bytes (actually, it’s lucky that I didn’t test any which ended up having 0, otherwise I probably wouldn’t have figured it out). Since this probably had something to do with chunks, I made a guess that each chunk had some extra data stored in this array, 12 bytes of info per chunk. Using HxD’s highlight-bytes-and-get-the-decimal-value feature, I noticed a pattern where the first 8 bytes looked like an i64 offset and the last 4 looked like an i32 size, which the compressed chunk infos were also like for their matching counterparts.
So, I deserialized it as [i64; i32] * n chunks and was able to read it better. These additional chunk infos seem to kinda sorta match up to uncompressed header offsets, but I don’t really understand what they are (and they’re marked as such in the code, lol).
So, with that working (and some fiddling with broken serialization of TArrays which I’m surprised I didn’t catch earlier), I now understood the ENTIRE package header! Hooray!
Let’s look at another example. body_grain_SF, the Fennec, has 4 chunks. Guess how many bytes of ??? it has. That’s right, zero!
For some reason, a few packages just don’t have those things. After going through a bunch of them, I found that if licensee version was 32, there was no ???. But if it was 33, there WAS ???. I basically just added an if statement and it was fixed.
I’m going to try to work on figuring out the padding in between the package summary and the start of the name table, but if I can’t, I suppose I’ll finally get to actually doing package swapping. Later!
I JUST DESERIALIZED AND RESERIALIZED A UPK. OMAIGOD IM SO HAPPY. my brain is so tired. Alright so the payload still doesn’t get transferred but that’s pretty simple, it’s just Vec::extend.
What a challenge! Deserialization actually ended up being the easy part, which is opposite of what I expected from starting the project. It went (relatively) fast since there are a few existing implementations which I could kinda just “borrow” structure parsing from.
But, mine is unique in that it almost completely reserializes the package itself. I might’ve been able to “borrow” reserialization, but the impl I’ve been looking at is STRAIGHT ai slop and it’s more annoying to read that than to just do it myself. It was useful though, so I have to respect it.
The first problem: basically, the first byte of an FString determines how many parts it has, NOT how many bytes it has. Which wasn’t a problem for parsing, since I had some word-based helpers which worked so I didn’t think much about it. But, when reserializing, I made two mistakes: 1. forgot to add the null terminator (which shouldn’t even be necessary) and 2. I accidentally put the wrong offset.
I have no clue why there’s a null terminator for FStrings. The format is [n chars][the char bytes], meaning it should already be known how long the string is. Pretty stupid, but whatever. Anyways, when reserializing you’re supposed to include the null terminator in the string length marker. So, my logic was that I should offset it by two since unicode characters are two bytes. Bleh! I should’ve only incremented by one, since it’s in terms of utf16 chars rather than bytes.
Yay, that worked! I don’t know how to summarize the next problem, so I’ll explain the reasoning. The package header is file summary + tables + something else (we’ll get to that later). Anyways, there’s some mysterious padding in between the summary and tables. Parsing, it’s easy to skip over because the summary includes the tables start offset (name_offset, the name table is the first one). But when reserializing, you gotta keep those offsets intact. Well, I’m not totally sure if you actually have to keep them, but just for the sake of simplicity I tried to (which ended up adding more complexity, which I’ll see if I can remove now). So, I serialize/push the summary to the big Vec. Then, I make a new Vec and serialize the name table at (name_offset - summary_size), since the new Vec has to be encrypted and then appended to the big Vec. I do the same thing for imports/exports (placing the table at import/export _table_offset - summary_size).
Oh no! Since name_table_offset isn’t equal to summary_size, there’s some padding in between, which is expected. HOWEVER! The previous logic would skip over it in the table Vec creation, meaning the table Vec would have some 0-padding at the start. This padding needs to be in the SUMMARY part instead, and the name_offset should basically be the ZERO of the tables Vec!
I doubt I would’ve been able to solve this part without the beautiful tool HxD. It’s really easy to use and is always helpful when I’m figgling around with binary.
So, it does some more math, calculates the spacing, adds it to the big vec, and removes it from the tables Vec. Then, it re-encrypts the tables Vec and pushes it onto the big Vec! Okay, now that’s done. NOPE! There’s a mysterious structure for some reason at the end of the real table part? Yep, it’s that something else I said we’ll get to. I actually still have no idea what it is. From some research, I gathered it must be the dependencies table, but it’s really poorly documented. So, I ended up just cutting the bytes out from after the export table to the end of the tables part, and “pasting” it back before re-encryption.
I do hope this devlog was readable. Now I’ll get to including the actual package payload and hopefully simplifying reserialization offsets. See you soon!
I’ve been putting this off for a while, but I’ve finally decided I’m going to add asset swapping. There’re a few existing implementations, so I’m reading those as well as some pdfs about the upk format. So basically credit to this document and this implementation. All it can do for now is parse the file summary.
The implementation wasn’t too bad since I had Google as well as the C# implementation I just talked about. I basically have
trait Deserializable: Sized {
fn deserialize(reader: &mut impl Read) -> AnyResult<Self>;
}
Then implement that for the various parts (eg. FGuid, a generic TArray). Then, I use byteorder to read out numbers into each struct. For example,
#[derive(Debug)]
struct FCompressedChunkInfo {
uncompressed_offset: i64,
uncompressed_size: i32,
compressed_offset: i64,
compressed_size: i32,
}
impl Deserializable for FCompressedChunkInfo {
fn deserialize(reader: &mut impl Read) -> AnyResult<Self> {
Ok(Self {
uncompressed_offset: reader.read_i64::<LittleEndian>()?,
uncompressed_size: reader.read_i32::<LittleEndian>()?,
compressed_offset: reader.read_i64::<LittleEndian>()?,
compressed_size: reader.read_i32::<LittleEndian>()?,
})
}
}
I originally prototyped it in Python and then moved to Rust once I got tired of python’s byte shenanigans.
I did lots of performance improvements and also IT WORKS ON LINUX NOW!!!
Let’s talk about performance first. Anyways, it has been using around 4% and 220MB of ram, which isn’t really what you’d expect from an overlay! So, I used things like DHAT to profile the memory, and noticed that fonts were doing some shenanigans. So I removed egui_system_fonts and loaded directly from C:\Windows\Fonts\segoeui.ttf and RAM usage dropped to 30mb. That was what the previous devlog was about.
I then changed the default logic update rate from 10ms (100/s) to 65ms (15/s approx). This reduced idle cpu usage to under 0.2%. But, doing anything with the UI would spike it to 4%, which is still pretty bad.
So, I used a method I found in an egui issue thread where I’d count the frametime since the last render, then sleep until the target frametime was finished. For example, if the frame rendered in 2ms but I want 16ms frametime, it would sleep for 14ms. That way, I could limit how often the UI rendered! This made it stay under around 1% CPU forever!
Okay, Linux time! Rust is mostly cross-platform, so support was mostly dealing with dumbass design decisions. Since Music relies on GSMTC, I had to feature-gate the entire thing (as well as the windows/windows-future crates) behind cfg(windows) :(
I also had to fix Linux font loading (since like I said earlier, it’s hardcoded to segoe ui). So I install fontdb on Linux and search for some popular fonts from there to load instead.
Finally, I fixed up the storage paths since it could only load from AppData. I also made it use AppData\Local vs. AppData\Roaming instead of putting everything in Roaming.
I first got it working through WSL! ! It took some messing with my GPU drivers but I got it to render. Now was the real challenge: getting it to build on an ACTUAL Linux machine.
I had an old, basically broken Surface Pro 3 that I had put Debian+GNOME on for shits and giggles a few months ago. The perfect target! The main problem was getting Rocket League to even launch. I set up Legendary (goated launcher btw, check it out) and tried to run it. I installed Wine, got Wine Mono, and it failed with a weird cryptic error about the CreateWindow function. The obvious solution to a graphical error: disable EAC. Yay, it launched!
Then to get it to build. First I had to fix some filepaths since Linux is case-sensitive (nerds) while I don’t care about case on Windows. Then, rust-lld gave some weird error about simd or something which I didn’t care about. I just ran cargo clean and rebuilt it and it worked fine.
Then to run, and it actually worked first try! It’s cool because I went into it expecting to debug a million socket errors, but it actuallly connected and ran. Awesome sauce! Now Linux users can use rlbuddy. More minions to contr- I mean cool users! Cya in the next one! ;)
If you check out the rl perf overlay in the screenshot, you’ll be able to guess why this is the laptop I decided to put Linux on.
I’ve been debugging high memory for the past month or so. It was egui_system_fonts. Why would fonts take 170mb of ram????????????????????????????????? help!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I hate external dependencies!!!!!!!!!!!!!!!!!!!!11
I owe my life to DHAT now. go star https://github.com/nnethercote/dhat-rs
More gamepad overlay buttons!!! Yay!!!
In not-so-awesomely-exciting news, I also added a table of contents to the readme (as well as a generator for it). The script’s in Python (I’m sorry for not using Rust) meaning the repository is now 0.5% Python. Takeover incoming?
I overcomplicated the script a lot, but it’s really easy to read. There are three whole comments, aka 6% of lines in the whole file! It’s not technically interesting in the slightest. I was thinking of building a tree-like structure before outputting the string, but realized I can just use the header level (h1 vs h2 vs h3 etc) as the amount of spaces / 2.
I also refactored it not to use magic PanelIds for the default panel list. I kinda rushed that part when I made it, I’ll be honest. But now each Panel can implement fn open_by_default(&self) -> bool (returns false by default) and if it does it keeps it open when savedata doesn’t exist yet.
I don’t get how an entire hour n 27 minutes was spent on like 7 not-even-that-big commits. At least it was a good use of my math class. We reviewed limits for some reason (which everyone should already know for one because they took the prereqs?), and the teacher managed to extend her lecture for the ENTIRE class time. rlbuddy development saving the day, I guess (just like it will in your rocket league games btw, so you should download it)
I’ve mostly been working on smaller features, improvements, and bugfixes. A cool one I did is fixing weird glitchy movement with the gamepad overlay window!
Basically, the window needs to persist its position across disabling it/enabling it. And egui is immediate mode. So, every time it rendered the viewport, it would both set and get its position. So, moving it would update the current position, but because of the weird interop between setting the position in a ViewportBuilder and getting it from the viewport info, it would glitch back and forth, especially when my laptop is lagging.
I fixed it by only calling the ViewportBuilder‘s “with_position” method when the overlay had just been enabled (!was_enabled && currently_enabled). That way, it only sets it once externally, and lets it move freely while it’s open. Then, while it’s moving, it continues to update the saved position. And upon closing and then reopening it, it sets it again (but only once!)
Relatively simple fix but it’s kind of funny. I mainly noticed it when I added a feature that would auto-unmaximize the overlay when the user maximizes it, and it would go crazy and fly around the screen. Very fun!
I also fixed a pretty old bug where it would incorrectly detect the local player. Basically, the stats api doesn’t actually tell you who the local player is. So, we have to guess based on who the camera is targeting right now. Usually, it’s on the local player, but during goal replays or spectating a bot when joining a casual game, it would be improperly set at the start and then stay like that.
So, I made sure that it always updated as long as the match is 1. in progress 2. not in a replay. It also fixes a thing with inconsistent match toasts, since the toasts only fire if the scoring player is the local player (and if it thinks the wrong player is the local player, everything’s messed up).
I’m planning on revamping hotkeys next, to allow you to bind to any controller button and even do custom quickchats in-game. Later!
Time formatting doesn’t suck anymore! I made the implementation way better so I could make it support stuff even past weeks, like months and years.
The new implementation added around 60 lines, but it’s for a good cause. The previous implementation used a match statement with duplicated logic that only supported seconds, minutes, and hours, using a pluralize function and inline if-statements everywhere. If it were sent to any rational code-reviewer, they would likely have a heart attack.
Now it’s a bit more complex but less complicated. You have TimeUnits which format the unit themselves and TimeParts which format a value + a unit. Then, just use some math to build a Vec and format it into a string.
Boom! Beautiful, well-formed time formatting. To soak in the new beauty, I added days n weeks. I just had to add 14 lines! The previous one would have probably needed a bunch of stupid magic numbers and weird chains everywhere to accomplish the same thing.
Okay, I’m done using your time on reading a way-too-long write-up about fixing a crappy implementation that shouldn’t have been there in the first place.
I’m gonna ship right after posting this. If you get it, make sure to give it 10 stars in everything 😉. Haha, jk. Well, not really. Cya later.
Had to remove the tracker.gg widget :( unfortunately they don’t allow scraping and what I had wasn’t allowed anyways. Things that used to open the player info widget now just open tracker in a browser.
I finished the big refactor! I pushed around 25 commits to GitHub at once. I’m really gonna enjoy adding the next feature, basking in how much easier it is ;)
Apparently, I never made a service for session stats, so it currently can’t persist its one setting. Actually, writing this I realized that the setting isn’t actually used, so I can delete it. That’s great.
After finally kicking out all the annoying legacy panel code, I remembered that I never re-implemented the positioning or its related persistence code for the new panel system.
The persistence system was reworked too, something that I don’t think really got included in the previous devlog. Each service can implement save, and there are generic helper methods in a common module for saving/loading serde structs to a file in the app data directory. The actual app now uses this method as well instead of doing all the pathing logic itself, so the persistence module is pretty small and could honestly be moved into app.rs.
For new positioning, I store a list of open panels. Weirdly enough, it’s actually a lot easier than adding a position/is_open field to every AppPanel. So, I run the name through a Hasher to make a u64 which is easier to clone/copy and stuff and made a newtype around a PanelId. That way I could reuse the code from the previous paneling system to swap/manage open PanelIds, then just use find on the AppPanel vec to find the corresponding widget.
Also, I never mentioned it but I’ve been trying RustRover for the past few devlogs. It’s a lot more stable than Zed, but I sorta miss its snappiness and minimalism. If I end up finding an annoying bug in RustRover, I’ll probably switch back. However, it’s been quite a few days and I don’t think I’ve found a single one. JetBrains is truly awesome at tooling.
See ya next time. Probably will do my next ship soon :)
I’m MASSIVELY refactoring rlbuddy! This, I think is the second (and hopefully the final) major refactor to the codebase.
So, basically the current architecture is split into services and widgets. Each service has an update method, which does its stuff (draining subscribers, processing commands, etc). Since egui is immediate mode, this architecture works unlike callbacks/event-based which I actually tried to implement but couldn’t.
Every widget is just implements egui::Widget, that holds some state handles (Rc) and command senders (Sender), usually just one for its respective service.
Services are alive for the lifetime of the app, while widgets are the openable panels you see. Widgets display the data from service state, and talk back to it by mutating shared state or sending commands.
I think music control has been the best for me to wrap my head around to decide about changes, so I’ll explain it with that. Music control had a controller, a service, and a widget file. The controller was a thread pool manager (if you can call the questionable implementation that) around Windows’ GSMTC API to give a single-thread api for it (since the app is basically single-threaded). Then, the service is always alive, syncing a state struct between the controller and its own state handle. Finally, the widget is optionally there as a panel, with next/previous/pause buttons which send commands and the ui which reads service state to display. It also has a mutable service state reference for settings (the “pause during anthems” button).
The god object RlbuddyApp struct stores all of this stuff. So it has a music_control_service, music_control_widget, stats_api_service, all of those fields, which kinda sucked. On top of that, I had to maintain an enum of every panel, which was enumerated for the “open panel” popup and to render each one if it’s open.
you might think “hey… isn’t that a self referential struct? how does music_control_widget store a reference to music_control_service?” Well, music_control_widget internally only has fields for the music_control_service’s state handles which are Rc/Arcs. So it only needs a reference to the service for it’s Widget::new function lifetime, all is well.
Although it doesn’t look horrible, it was HELL to do anything. For example, refactoring the gilrs service out of hotkey and adding the gamepad overlay probably involved just as much fiddling around with Rcs in RlbuddyApp as adding the actual feature.
So, I decided to move to a new architecture: services + features! And, after thinking about it for around 8 hours (i slept), I decided on something a bit different. Services + panels! Yeah, basically the same as services + widgets, but I was gonna implement it GOOD this time, and make design decisions that wouldn’t bite me in the ass later on.
I wrote my first traits - Service, which requires update and optionally save, and Panel, which requires name() -> &'static str and ui (same as egui::Widget: :ui). Then, I’d change how components are registered: instead of keeping every service and widget as a field in RlBuddyApp, I’d make a Vec<Box<dyn Service/Panel» for each, and loop through it.
I spent about an hour trying to get Panels to automatically implement ui by requiring egui::Widget as a supertrait, which doesn’t work since widgets are stateful (need &self). So, I tried to require &mut Self to implement egui::Widget, but apparently something to do with type erasure doesn’t let that work with dynamic dispatch which was needed for the Vec<Box> thing.
I’ve moved a few features to the new architecture, but I’m still in the process of finishing up. Additionally, it has to work side-by-side with existing crappy monster-Panel-enum architecture, which sucks to maintain.
Okay, once again, I had to strip down a ton of detail (and my terrible wording but wtv) cause I hit like 5000 characters b4 realizing. See ya! May the Stardance gods increase devlog limits :)
rlbuddy has a gamepad overlay now so that you can show off your awesome bindings! Don’t even bother figuring out mine.
It’s kinda starting to become hellish to add new widgets/services, but I’m almost done for the next ship so I doubt I’ll refactor just yet [edit: im doing a super refactor now hehe].
I made the overlay with the raw egui::Painter rather than actual controls (since it would be pretty hard to make a controller out of Buttons or something). I also prefer to avoid embedding static assets, otherwise I might have made an actual one in MS Paint or something.
I also can’t seem to find a good egui version. I’ve been on 0.34 almost the whole time, which works well but doesn’t support transparency in extra viewports. But, 0.35/0.36 broke external viewports. So, until then, we get an opaque gamepad overlay :( but the main app is still transparent, so it’s not horrible.
Rolled an 8 and a 9 on the Stardance rng today! I’m so proud of myself.
I moved the mmr server to alpine linux! Before, it was regular raspberry pi os lite on the Pi. But, it was behaving strangely, randomly rebooting and things, so I decided to try installing Alpine. The lightweight distro has been on my radar for a while now, but this was my first time ever actually trying it.
I imaged it normally with the raspberry pi imager, and put it on the pi, did setup-alpine, and all was well. But, I put in the wrong wifi password at first, which broke everything else. I tried changing the wifi config and doing something with wpa_supplicant (thanks google ai overview), but nothing fixed it so I ended up doing a fresh install.
This time, I put in the right password and everything went great. I set up ssh, node, pm2, etc. and moved the server files onto the pi through sftp. Then, I got it running over my local network and set up authentication.
So, it was time to set up the Cloudflare tunnel again. I followed this neat guide, changing values as needed (for example, my pi is aarch64, but the guide was x64, so I spent a while figuring out why the pi kept giving elf errors). It took a really long time to set up the openrc startup script because 1. cloudflare rotates the setup token relatively frequently and 2. I accidentally put 2 –token arguments. And since I was using vi, I didn’t even notice and kept trying to figure out why cloudflared crashed on startup. But I eventually found out and removed the duplicate and it worked!
Then I finished setting up pm2 and put the pi in its home next to the router, and mmr.kmdw.dev is back up exposing your lobby’s skills. See you in the next one!
Yes I know part of the token is in the image but it’s used already so it doesn’t matter
rlbuddy can load custom maps now! Either you can download them and then import from a zip, or you can download them from bakkesplugins straight from the app!
Importing it from a file is relatively easy. I checked the code for Lethamyr’s custom map loader to see what exactly it did, and for Lethamyr maps it:
I knew Rust, being a low-level language would let me not have to extract the zip fully first. Additionally, Bakkesplugins maps don’t contain info.json or preview.jpg, they only contain a udk file, so I had to make sure it would work with that. So, the import code I wrote basically:
It worked great. I made a really cool looking widget with stupid manual layouting behind it for map selection. First, it would add the preview image (or a black background if none). Then, it would get the screen rect of the image, and draw a semitransparent black background onto it (for contrast). Finally, it drew the actual contents in that screen rect. So, I got pretty-looking tiles with preview image backgrounds!
I also added a progress bar for import status. I benchmarked each main part of the import process and found that decompressing/extracting the upk file took around 2 seconds, for which the ui thread would be totally frozen (big nono)! So, I moved the work onto another thread, and also made it decompress/load in 100 total chunks, updating a progress bar each time.
I then moved on to the map downloader. Bakkesplugins doesn’t have an API, so it relies on scraping the website. I didn’t actually use any external crates for gathering information, though. I wrote a parser which takes a list of rules, each with a start and end search string, and looks for each of them in sequence till it fails to find a rule. I tried to make it completely zero-copy, but something it needed was to turn an [Option; N] into a Option<[T; N]> if every element was Some, but the only way I could figure out how to do that was with an iterator. It’s still really fast though. So, it parses search results, including map titles and urls straight out of the html string without needing an HTML parser or regex library.
Once it was able to load results from the search page, I went to sleep and came back today. Now I had to get it to actually download the ZIP files and preview images, and import them straight into the app. It first has to download/parse the plugin info page since the ZIP url isn’t available in the search results. Then, it requests the ZIP. I had some trouble with this, since I wanted to show a progress bar for the download status. I first tried a function from this gist, but it didn’t work. So, I made my own which used the same chunking strategy as the import status loader. Turns out, ureq blocks requests once they hit 10MB by default, which was probably why the gist version didn’t work. But, mine was faster anyways, so I kept it. It’s even cooler since I made it myself.
Then, I wired up a couple more commands to let it import straight from archive bytes instead of from a file path, and it finally loaded! Once that worked, I made it pass along the map title, author, and description as well. Then, I extended it to parse out the thumbnail image url and download that, then pass that along to the importer as well.
Anyways, that was super fun. I would add a lot more detail, but Stardance limits devlogs to 4000 characters and I’m approaching scarily close to that. I’m planning on just polishing the downloader interface a bit more, and then I’ll do some bugfixes for the next release. See ya!
rlbuddy now can show people’s avatars cross-platform! It uses a web api I put on Cloudflare Workers which uses platform-specific requests to get avatar urls (see the previous devlog), which the app just fetches and renders.
The http part was pretty easy, since I already have wrappers for cached HTTP apis, so I just had to define types for that and hook it up. But, once it was time to display, I ran into a weird problem where egui said there was no image loader installed for jpegs! Even using all_loaders + default didn’t work. Turns out, I have to actually include image as a separate crate and enable all the filetype features I want in there.
Once I got it to load, it was time to mess with styling again. egui is really annoying with images, so I spent a while trying to get them to size right first. The default avatar used while they’re loading or for Epic/Switch players is from Fandom, like the rank icons. However, even though the icon is 90x90 pixels, the webp is 128x128 pixels, it just has a bunch of transparent padding.
So, I initially messed with egui to do some weird ui allocation and calculation to get it to render right. But, turns out I didn’t have to do all of that. All I had to do was allocate a 28x28 space (the target size), scale that up by the 128/90 ratio, then paint the image onto the new size. Worked great.
Then, I got to fight with grid layouts again. For some reason, wrapping the avatar and player details in a horizontal totally exploded all the layouting. So, I ended up making avatar and details separate columns. But, since the “player” header text is a bit larger than the icon width, there was too much padding between the avatar and the player details.
So, I did more messing around with egui manual layouting, and ended up allocating a 24x(body text height + 2) space for the header, which would give it the default spacing. Then, I’d copy the Rect, shift the left bound over by 12 pixels, then paint the header onto there. That way, the header is left aligned with the avatar and the avatar + details look like they’re in the same column, even though it’s actually a bunch of stupid manual layout magic behind the scenes.
Made an avatars api for Steam, PSN, and Xbox (Epic doesn’t support profile pictures and I have no clue what goes on in Nintendo-land, even Tracker doesn’t show theirs). It’s on Cloudflare Workers instead of my pi since it’s basically just api integration.
I totally overcomplicated finding out how to get Steam profile pictures. Like, it’s insane how easy it actually is. A few weeks ago, I got an old version of Steam Rocket League and decompiled it, trying to look for where it fetched avatars from. After some experimentation and debugging, I learned that when Psyonix builds Rocket League, they give it something that implements OnlineSubsystem for whatever the target platform is. So, Steam gets the Steamworks OnlineSubsystem, Epic gets the EOSOnlineSubsystem, and so forth. And the game doesn’t need to know implementation details, just that it implements OnlineSubsystem, I think the term for that is dynamic dispatch in programming languages.
Anyways, once I learned that Rocket League doesn’t do any avatar handling itself, I looked at the Steamworks SDK. Basically, it routes the avatar request through the open Steam client. So after some experimentation, I got it to fetch profile pictures with C++ and the Spacewars test game. But, that obviously wouldn’t work on a server, since the Steam client can’t really run on a Worker.
So, I tried to use the Steam Web API. I got an API key, but when I tried, it said I was forbidden. According to the documentation, the GetPlayerSummaries endpoint was at partner.steam-api.com, which was only for publishers.
About a week later, so yesterday, I had the genius idea to use api.steampowered.com (the public API domain) instead of partner.steam-api.com. Just swapping them out. And it worked. So, I now had a way to get Steam avatar urls.
Xbox and PSN were actually really easy. Xbox has an undocumented API at https://peoplehub-public.xboxlive.com/people/gt(GAMERTAG) which returns json containing the avatar uri, and I used the psn-api package with a new PlayStation account to get PSN profile pictures.
Then, I just put all of them in a Worker. So, rlbuddy will be able to show avatars soon enough!
Recently I was going through game archives and came across the BepInEx tool, which is basically a code injection framework for Unity games. I was really curious as to how it actually launched when the game launched, and found that they use a tool called Unity Doorstop, which adds winhttp.dll to the game’s binary directory. So, when the game tries to use things from winhttp, it goes to the new winhttp.dll instead of the system32 winhttp.dll. From there, the custom dll just forwards all the function calls. However, the custom dll is modified so that on its first interaction, it launches whatever Doorstop is configured to launch, and the mods can be loaded. That’s at least as far as I understood, but it made sense, and I thought, “what if I could launch rlbuddy automatically like this?”
So, first, I tried to test it with a custom executable and library. I created a basic executable and DLL in C, and got the executable to load and run some code from the DLL. Then, I created another DLL, and got it to “impersonate” the original DLL. So, I’d rename the first dll to something like “mylibrary_original.dll”, rename the faker dll to “mylibrary.dll”, and the original executable would call the original, but it goes through the faker instead. And it worked! I had successfully created an actual doorstop-like faker.
Now it was time to actually try this on Rocket League. So, I did a dumpbin /import on RocketLeague.exe, and found a few good targets - IPHLPAPI.dll, a system dll for networking stuff, and xinput1_3.dll, one shipped with the game for talking to controllers. RocketLeague.exe only imported two functions from each of them, meaning my fake interface wouldn’t have to be too big.
So, first I tried to spoof IPHLPAPI.dll. The code was pretty simple, just a couple of global objects to hold the function. So, when the game loaded the DLL, it loaded the fake DLL. Then, when it tried to call some function in the DLL, such as GetAdaptersAddresses, the fake DLL would be the actual receiver, it would log something to a file, and then it would call the REAL GetAdaptersAddresses by dynamically loading the system IPHLPAPI.dll and return it. And… it worked! I opened Rocket League and the log file was created and had the log “GetAdaptersAddresses called!”
Then, I launched it with anticheat enabled, and EAC immediately shut it down. I guess that was to be expected, but it was pretty annoying. So, I decided to try one more time, this time with xinput1_3.dll. This was more interesting because it was lazy loaded, and functions were imported by ordinals (2 = XInputGetState, 3 = XInputSetState), so I thought something might be different.
So, I modified the original faker to be xinput1_3, replacing the function names and definitions. Since Rocket League imported them by ordinal instead of just the function name, I had to create a .def file as well, and define the ordinals there. Then, I did the same surgery on the binary folder, and after a few small mistake fixes, it worked! But, I still hadn’t tried with EAC enabled. So I launched it with anticheat, and it gave me a different error this time - that it couldn’t determine xinput1_3.dll’s file version.
That was a different error! I opened the original xinput1_3.dll’s file properties, and it had copyright, file version, product version, and a couple other random things that were missing in my faker DLL. So I made a .rc file and compiled it together with the faker DLL. Then, most of the properties matched up. So, I tried it, and it seemingly worked! And then EAC said that it couldn’t verify the file again.
In the end, I didn’t manage to get it working. I should’ve expected EAC to block me from the beginning, but it was pretty fun, and if I ever want to do something similar with a game without anti-cheat, I can do that now!
rlbuddy is pretty closed to being released though, I just want to polish a few things and make sure the docs are fleshed out.
rlbuddy doesn’t use the default eframe persistence feature for savedata anymore, which is pretty cool. There were a few reasons for this:
Anyways, actually getting the data directory was pretty interesting. I’ve basically decided to only support Windows, since the music player relies on Windows APIs anyways and I don’t think the Stats API even works on Linux.
Initially, I checked out the source code for the dirs and directories crates, which led me to the dirs-sys crate (which actually isn’t even on crates.io). It used unsafe raw Win32 SHGetKnownFolderPath with the Roaming folder guid. However, since I was already using the windows-rs crate, I tried to find the WinRT equivalent. I ended up enabling the Storage_Search feature to use the API which would let me get it. Of course, this also didn’t work, because of something to do with how Windows apps are supposed to be registered.
So, I ended up just grabbing the APPDATA environment variable and using that, then joining rlbuddy\data.json to it. It’s pretty underwhelming, but it more than gets the job done. Now, whenever the app panics, I won’t lose all of my settings anymore! Pretty nice.
Anyways, that’s what mainly got accomplished. I also tried to set up a GitHub Action to build it and make a release, but I kept hitting errors so I gave up and decided to upload a release next time I get the urge to actually update the GitHub repository.
There’s also a few fixes, like the window title which has actually had a typo since refactors from around a month ago, and adding a missing ! which broke toasts when in training.
Overall, only an hour or so was tracked, but a heck of a lot got done. See you all next time!
I added a small toast notification that activates when you score in training, like bakkesmod used to add it to text chat. It also activates when you hit the crossbar so you can feel even more remorseful that your beautiful musty didn’t go in :)
I actually got the program to segfault while making it. egui’s external viewport api is kinda funky… there are two modes, immediate and deferred, and immediate is supposed to be easier to use. However, when I tried to use immediate mode, it panicked saying that the egui backend was implemented incorrectly. So I switched to deferred, and it gave me the segfault. There were 3 things I had to do:
I stored all active toasts in a Vec, and render them in the external viewport every frame. When one should disappear, it uses Vec::retain to remove old ones. However, this doesn’t do any cleanup, meaning the viewport would go bonkers and crash the application. So, I had to close the viewport first.
Calling ui.ctx() from the viewport made a segfault, probably my fault but I couldn’t figure out how to resolve it so I ended up just sending Viewport::Close from the main loop.
Even if I sent Viewport::Close from the main loop, it still gave the weird error. So, I resolved it by adding a small delay from after it should close, which is when it sends the Viewport::Close command, and when it’s actually removed from the Vec of toasts.
Well, I did all that, and it was working, but there was a really weird bug where it would lag the whole system while opening/closing. So, I switched from immediate to deferred viewport mode, and it stopped happening!
I hit my first powerful airdribble musty with it on, it was around 105kph, and a total fluke. I wish I had gotten it on recording.
I replaced the Spotify integration with a generic one that uses the Windows GSMTC api, meaning it’s way snappier, since it’s not making network requests every time it wants new data! The only reason it needs data is to fetch playback info, and it’s easy to do that since the windows crate gives a bunch of event listeners, meaning I also don’t need to poll anymore.
Unfortunately, the integration from browsers etc. is kind of buggy so thumbnail loading is relatively inconsistent, but by adding some caching and timeouts it works flawlessly :)
I also redid the UI since the old one was frankly quite ugly.
Since the playback listener doesn’t update that frequently for progressbar changes, I also made it fake progress when music is playing and the ui updates, that way even if the latest data is stale, it still looks like the progress bar is moving.
I’m now working on better session stats, I’ll probably end up replacing the all-time stat tracker with more in-depth session-only ones
rlbuddy finally has an MMR graph! I’m planning on adding more stat displays soon, like W/L, goals for/against, saves, etc. but this is a really big milestone since it’s a feature that a lot of players actually use (I basically only play casual so it’s not really for me 😅).
Anyways, this also required me to rearchitect how matches are stored, and now there are two representations: a session match, which contains lots of information about it like detailed player skill information, and a stripped match, which is minimal and stores very little because it actually gets saved to disk! So, match history also gets saved now! This also opens up the doorway to a Deja-Vu-like thing which tells you if you’ve played with someone before and when!
It uses the egui_plot library to show the graph. Funnily enough, basically every default option for a displayed plot was the opposite for this, and so the plot builder ended up using almost every method.
Also, once again, hackatime is wrong because Zed likes to uninstall extensions randomly :(
The Stats API finally provides the actual playlist id, so rlbuddy isn’t limited to guessing between 1v1/2v2/3v3/4v4 anymore! This brings in some cool extra features like displaying the rank each player is in for the given playlist as well as even better Discord presence!
I looked through BakkesMod as well as some psynet endpoints to grab the list of playlist ids to associate with each actual playlist.
There isn’t really an image to go with this, so here’s the monster enum of playlists :)
I rewrote the API admin page with Svelte! Before, it was a quick-and-dirty raw html/css/js solution, but now it should be a lot less annoying to work on.
I improved authorization a lot, now instead of adding the pw query parameter for every request you put in a password once and it actually uses the Authorization header. Unfortunately, the browser EventSource doesn’t support headers except for in cookies, meaning I had to use the custom eventsource library for bootstrapping since the bootstrap API uses SSE.
There’s basically a whole Vite project in src/webadmin, with its own tsconfig.json and everything. Because of the way it’s set up, development needs Vite running in build watch mode which builds to where the main app serves it out of. It makes deployment a lot easier, though.
I didn’t really add any actual features to it, except for a log filter. However, it should be way easier to add more features in the future, like a api tester!
rlbuddy has player lookup now, so you can check for smurfs without needing to boot up a heavy browser while playing. Unfortunately, this has to get its information from TRN because it’s the only source of this information, the rocket league apis don’t provide it. There’s some cool refactors behind the scenes, though. I generalized almost all HTTP requests into its own struct, so it removed like 200 lines of boilerplate code into a single generic module. It automatically handles caching, using new threads, ignoring if there’s already an inflight request, loading states, and automatic serde_json deserialization. The Epic id, uncensorer, ranks, and new TRN apis all use it now! Also, the button to open in TRN from the player list opens the player info dialog instead (though if you need to, you can open in TRN from there).
Note: I’ve actually had way more hours than it says, but I reinstalled windows and forgot to set up hackatime again so all that’s gone :(
mmr-api-v2 finally has an admin page, so I don’t have to ssh into my pi just to check some debug information anymore! It is kind of developer-oriented but I think it looks and works well enough for what it is for.
In the actual rlbuddy app, there are some cool new features! Facilitated by lots of refactors, of course (turns out, the matches service was bigger than the actual rocket league service, so I finally separated them). Spotify integration now has a previous button as well as a skip-to-song-in-current-playlist button. Match history was a feature that I’ve kind of ignored, so I gave it a much needed touch-up so its easier to view and browse.
A pretty cool thing is that that’s been added a rocket-league-playerid-to-epic-games-id translation endpoint. This means that you can see Switch players’ TRN profiles through their linked Epic account, since TRN doesn’t let you check based on Switch name alone.
Finally, there are some small quality of life improvements, like less deadlocks, better anthem pausing, and the open panel list persisting.
Turns out, for spotify integration, I can’t actually let other people use my spotify development mode app. It’s limited to 5 users which I have to invite by email. And to get a quota extension, I need 250,000 existing monthy active users, which I’m slightly short of (by about 249,999). So, I changed the login flow so that each user has to make their own Spotify app, and log in using the client ID from there. I didn’t change the current PKCE flow since that didn’t really need to be changed, but setup unfortunately takes a little more work now.
Also, it has transparency now.
It’s been a week since the last devlog, and lots has changed!
The code was getting pretty messy since I had kept adding new features without really doing major refactors. So, I finally took it upon myself to actually organize everything and come up with a proper architecture.
Each feature has its own module, containing services and widgets. Services have all the state, for example the Spotify service stores the current song and settings and the Rocket League module exposes a Matches service for the current match and past matches, which the match and past match widgets actually show.
Services all have update functions which get called every 10ms. There are two ways for services to talk to the rest of the app: events and state. State is straightforward, it’s usually just an Rc with a custom generic struct I made to manage whether the widget can read/write to it. Events are enums which the update method returns, and can get passed to other update methods. For example, the Stats API service doesn’t have its own widget, it only connects to the stats api and returns match events which other features can use, like Spotify using replay start/end events to play/pause music.
The dashboard part isn’t as ugly as it was before, and has actual cards for every panel instead of just having horizontal lines in between. Even though it sounds pretty small, it’s actually huge for the prettiness of the app!
Along with this, the behaviour of managing panels also changed. There’s an actual close button, and instead of having one button per panel at the bottom, there’s a small picker to open/close them. Also, you can also move them up and down now
This is a pretty small feature. It lets you select the path to the rocket league executable, detects whether it’s already enabled, and turns it on if it’s disabled. It does this by just replacing a line in the stats api configuration.
There were a few bugs which would make the API disconnect randomly, but they’ve been ironed out. This also made it so the app doesn’t have to deal with failure behaviour and can just keep asking until the API responds.
Although it looks like a just few elements and a button, theres a lot going on behind the scenes. I’m not embedding an entire webview to run the web playback SDK this time, so it just uses the Web API to keep resource usage nice and low.
Since it’s a native app, it has to jump through some hurdles to authorize, using the PKCE extension version since the client secret can’t be stored securely. It uses the webbrowser crate to open the Spotify auth page, and starts a tiny little oneshot HTTP server on a raw TCPListener. The redirect is set to the HTTP server, and then as soon as it receives the request, it sends a simple HTML page and continues. From there, it receives the authorization code and can continue as normal :)
Other than that, it’s just normal REST api interaction. I did learn a ton about thread safety (I hit so many deadlocks), because unlike the stats api, for example, which just sits in a different thread and hurls events over a channel, it actually has to work with the user.
It’s also able to pause during goal replays, and that was pretty annoying to structure. Eventually, I decided to just use a channel where the stats api could send replay start/end events through the match widget. Looking back on it, the solution was extremely simple 😅
I am planning to add a bit more, like shuffle/repeat controls as well as a previous track button, but this feature turned out pretty nicely!
rlbuddy can now show Rich Presence in Discord, so now all your friends can see you destroying your opponents ;)
rlbuddy has had major ui overhauls to shift it more towards an actual overlay rather than just an app that displays ranks. there are new features like a settings menu as well as major architectural overhauls, with more coming soon!
Got it working as an actual overlay!! Right now it just displays ranks, though
I got a stable MMR/ranks api working!!
https://github.com/Kalilamodow/mmr-api-v2
Basically, it’s an http server that opens a websocket to the actual rocket league game api in the background and spoofs a real client connection. The socket only stays active for 5 seconds at a time though so I don’t kill psyonix’s servers. That way, rlbuddy can get ranks without getting blocked by tracker network!
I originally tried to do it on Cloudflare, but the workers outbound websocket didn’t work since you can’t send custom headers with them. I ended up having to use my old raspberry pi and set up a homeserver behind Cloudflare tunnels, and I’m actually planning on moving the rest of my stuff that used to be on a VPS onto the pi.