The demo UI has been finished and it works quite well! I have a few more things to add like a looping map so that the Character doesn’t just walk off the end of the world and die plus some more main menu stuff I think too. anyways progress has been fast so far so expect more soon!

LOTS OF DEBUG UI COMEING YOUR WAY!

I have added lots of UI stuff to display internal values such as costs as stuff. I have also added the ability to adjust motion matching settings from in the demo so when I upload it to itch.io all of you can play with the system as much as you want and tell me how bad you broke it. to do wise its a bit more UI (some of it doesn’t work or I want it to look better) and then the data points kinda like the unity gizmo stuff I have been showing you here on star dance. anyways its like 1 AM I should have gone to bed HOURS AGO so I’mma go brush my teeth and work some more tomorrow…. my keyboard is starting to fail :-< Goodnight y’all. or good morning to the folk around the world like in Norway or something

I finally made a git hub repository. I also made a new unity project and moved everything into it to clean everything up for the demo I am starting to work on. I also made Bone Rotation matching but it doesn’t contribute all that match. I should be ready to ship/ Have a demo ready soon

OK OK IT WORKS ENOUGH OK DONT TOUCH IT

I’m a fraud you guys. I couldn’t figure out the velocity thing so I…. I USED AI! I SWARE I HAVENT BEEN THIS ENTIRE TIME…. ok heres what happened I couldn’t figure it out so I had some AI going through some Motion Matching related thesis stuff and explaining it to me and then I ended up giving it a small chunk of my matcher code use to generate the cost and it pointed out my problems….. so 1. The way I was calculating local position was wrong which is funny because when I do it the other way it also breaks but then that leads into the second part. 2. I was inverting data and that was what was keeping the system working for some reason awhile back and even when I tried removing it a bit ago it broke which was really weird. anyways if you change both of them they fix each other so YAY I guess… sorry for the AI usage. it didn’t re write my code for me though I hand fixed my problem which I can proudly say all of the code is all hand typed…. (other then stuff I copy and pasted and or reused from other code I wrote… TRUST ME I NEVER WANT TO REWRITE UI SYSTEMS please unity fix the uitoolkit its bad)

Working on velocity matching. its kinda working now better then it was but its really weird that it slows the character down so much but its better then random snapping poses

HOLY CRAP IT WALKS!!!

I can’t believe it. I am so FLIIPING happy ITS MOTION MATCHING! THAT IS ACTAUL REAL MOVEMENT AND NOT JUST WIGGLES!!!!! AAAAAAHHHHHH!!!!!! HECK YEAH!!!!!!! ok so in reality its still not fully working. I had to disable bone velocity matching completely to get it to work but I think I figured out why it doesn’t work. for trajectory matching I realized that the character is scaled to 1.83 instead of just 1 so that will affect the matching and I have to implement an offset (and I did it for the bone positions too but I never did it to the velocities) anyways I think I will fiddle with it till that work epeshly since STARDANCE WAS EXETENDED UNTILL OCTOBER 30TH THANK YOU HACK CLUB!!!!

Video is of it working, not working and me playing with settings.

WE HAVE HIT 100 HOURS LOGGED!!!!

wow the amount of time I spent on this is CRAZY! anyways we have had little improvements on the matcher I have been trying to get it to give me a full animation but it refuses instead kinda just wiggling its arms up and down pretending to be a bird OR snapping between poses that have NOTHING to do with walking.
I have spent the last few hours attempting to calibrate it to work as well as writing a better dataset debug script so then it will be easier to visualize what every frame of animation contains Data-wise so then I can attempt to manually match frames and see what the matcher ranks them vs what it normally matches so the I can maybe weight it or bias it a bit. we only have a week and bit left LET LOCK IN!!!!!!!!!!!!!!!!!!!!

we have done it!

MOTION MATCHING FROM SCRATCH!

the last week has been crazy. I have found more data from a unity demo of a long discontinued motion matching asset and I have retargeted the animation in blender and brought it into unity with proper root motion. since this was my first time using more then one animation clip with EVERYTHING I expected bugs and errors AND THAT I GOT. I had to go in and rewrite a really small chunk of how the dataset maker compiles the trajectory data. I had to make the matcher actually handle more then one clip as well as dig into a null reference error it was giving me so I had to dig into it more and learn that the dataset was still flawed and go back and do more to the editor AND THEN ONCE the data was good now the matcher scales the character to like 300% or something and I never even told it to touch scale so now I have to properly set up multi-animation unity playables since before I didn’t know that you couldn’t just set all the blend weights to one and hope for the best. now finally after all of that. it appears to match and hold onto a frame. I have a few last features to implement into it that being velocity matching so then it wont just hold a frame but understand that arms move and swing. as well as debug info for both me while testing as well as for you my dear hackclubbers so when I ship you all can see a bit into the inner workings of it.

thank you for coming with me on this journey I never expected it to go this long but it feels good to have at least one person interested in what I’m working on for once and not just get shrugged off. - Ash ❤️

Video is of the matcher working
photos are of the scaling issue

sheesh I’ve done a bit in the last week.

So we have

• added the ability to match bones in the matcher
• got the matcher to handle multiple animation clips

and I guess that’s it… I honestly thought I did more… on paper it doesn’t actually seem like that much.. huh. OH WELL! I also found a MASIVE collection of animation from a unreal engine demo project so I’m probably going to extract that and retarget TONS of that over the next few days because imma say it the problem with the matcher is the lack of good data at this point.

Pink/Purple - Animation frame data position
Red/green - predicted trajectory

OK OK OK

IT WORKS!!!!!

so we got bone positions to work. I’m done for the day. anyways I literally just had to flip my subtraction around… yes… its stupid…. anyways went from this: RootPositions[i][j] - BoneTransform.position to this: BoneTransform.position - RootPositions[i][j]. ok mic drop… I did say I would ship by the end of the month of august and clearly that didn’t happen but it will (hopefully) be this week

Pink sphere is the foot pos from data set
Blue is the other models foot pos not from dataset
then red is root position

(Also 2 devlogs in one day! I’m on fire!)

IM SO CLOSE!!!!!!!!!

LIKE LATER TODAY (maybe can’t promise anything or the programing gods will kill me) I WILL HAVE MOTION MATCHING IN A GOOD DEMOABLE STATE FOR ME TO SHIP! I have got the bone data to work and actually spit out something useful by changing my whole approach and instead of sampling curves (which could work I just gotta deal with all the weird problems) I have created a unity scene in memory it spawns in the rig samples all the bone transform.positrons making unity do all the math and hard work before it returns me world positions of all of the bones I told it to track. the only problem I am having now is that I got to convert those world positions back into local position so that they can work anywhere (and yes I just converted them from local position to world position and this might sound counter intuitive but I need to change the parent of the local position) I remember this being easy for trajectory matching but maybe since this is WAYY more sensitive to rotation I need to try a different approach. the final stretch is approaching us dear hack clubbers. I got the rest of today then school starts tomorrow.

LETS! LOCK! IN! YEAAAAAAAAAAAAAAAAAAAAAAAAAHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH!!!!

btw the pink sphere is the foot potion… or what should be the foot position. Blue is the actual foot position. Red is root Position

Note to self… don’t freehand code… plan everything out on paper because I’m just experiencing Murphey’s law at this point :-<

so…. the data that I didn’t know if it was right or not was incorrect… VERY incorrect (kinda) so that data was in local space I need it in world space but to convert it you either need to do the conversion through a transform (which requires a transform to reference) or do what I am doing (since I don’t have a easily accessible transform (unless I’m stupid and don’t know of one)) so I am going up the paths and for each object extracting their position and transforms (in local position) then adding it up until I eventually hit the root which would be in world space so now its converted (I’m doing the same for rotation too with more math) anyways my code for this entire project was never pre planned and I never normally pre plan but this is one of the times where once it works I’m going to figure out how it works (because who am I kidding I don’t really know) then I’m writing all of it down on paper and optimizing the loops and everything before jumping back into the editor to re-write the bake() function from scratch leading to MUCH BETTER READABILTY and hopefully speed because it takes a good 5 seconds from my for loops and brute force. anyways if you cant tell this is VERY BORING WORK and I really just want it to work SO BADLY because once I get everything in world position the next steps should be really easy but I shouldn’t say that because at this point if I do I might jinx myself

WOW JUST WOW!!! So I toured the Vicon Set-up at my local univercity and it was amazing. I loved every moment of it. they even put markers on me and captured some data (not any I can use for motion matching but still WOW THAT TECH IS COOL)

In more on topic news I got the dataset maker to output bone data into the database. is it the correct data… idk but I will figure that out!

(Photo of me standing forward is just a general photo with the markers. photo of me looking away from camera with hand in air is me showing off all the retroreflective markers (and EMGs) then there is a photo of the compiled dataset)

THERE WE GO! finally we can select what bones we want to track from a nested dropdown. now I just need to implement it into the baking logic so I can get a finished dataset out of dataset maker. perfect timing too. I have gotten a few emails back from the university and they are willing to give me a tour of the Vicon system and there may be a chance one day to capture data on it so hopefully everything works out and I can capture on the same tech that AAA game studios use which would be a dream come true and would blow my mind.

There we go. I think I finally got it working. JEEZ UNITY NEEDS BETTER DOCUMENTATION ON THE UI TOOLKIT LIKE WOW ITS THAT BAD…. anyway I got the custom lists working now I just gotta add the actual bone selections then write in the logic to add everything to the datasets. if you look at the photo that’s roughly how it will look to use the dataset maker. not sure if I mentioned this in the last devlog but I finally messaged my local university to see if they will let me either tour or use their vicon system so I’m hoping something comes of that because that would be really cool. anyways its really late I pulled another programmer and stayed up till midnight working. see yall later.

Pretty boring stuff happening right now. I have been working to implement bone positional and velocity tracking into the dataset builder but since I wanted to use a custom class to do it the unity UI Toolkit decided to pick a fight and be really annoying to do it in. if anyone wants to write a custom class in unity and have it show up in a list view use this: link I also am working on processing some mocap data but its not going so well right now and I am getting pretty tired with dealing with these markerless systems so we will see if my local university will let me use or even tour their 10 cam Vicon setup (cross your fingers for me) anyways I still have one mocap session to prosses so I will probably do that tomorrow since I can just let it run while I am doing other things. also I just watched these videos where this guy is building open sourced optical mocap cameras and I think I might just funnel all my hack club hours into pcb credits to build a few of them.
part 1 vid: https://youtu.be/kYVqL_DqBis?si=CPKxKiLa8cxf3M1t

Part 2 vid: https://youtu.be/WJ13C_1hDIA?si=hMlQ8u5ofEBf-cFS

#MOTION MATCHING SYSTEM V1 DEVLOG
ok its been a few days and we have gotten a few things done including

• started work on feet and hand bone tracking (the secret sause to motion matching)
• I finished making the large Charuco Board so I can calibrate FreeMoCap.

so its coming along… slowly but that’s ok. the hardest part of all of this is the motion capture though since my setup is really bad (I wish my city had a capture studio somewhere even if it was small)

sheesh its been awhile since I last posted a devlog.

MOTION MATCHING SYSTEM V1 - HIP

Better matching

Change Log/Updates

• Matching changed from world space to local space
• started work on recording motion capture

I got more consistent matching now thanks to changing how it store the data in the dataset and how it matches to it. originally the future trajectory positions where stored as world space values in the dataset and the matcher would match it to the players world space values. if you know how transform systems work in game engines or blender you might know that because of that I had to be in the EXACT same position in the world that the data was recorded (you can see this in my last devlog where I kinda wiggle around in a few areas and get flurries of matches) in which… doesn’t work very well for this because if you had to record your self walking forwards backwards and side to side for ever single value in the game world that would take you FOREVER! so I switched it to local space which is basically where you are in the world based off of something. so like instead of 0, 0, 0 being the root of the world it would be where the character is so the trajectory positions are based around the character transform… do you know what I mean? I hope so because I cant think of a better way to explain it at the moment.

Right now the biggest thing holding me back is the actual motion capture data that I am feeding into this thing because the recording I downloaded… is really bad. so I have been experimenting with a free open sourced software called FreeMoCap which is a marker less triangulation based motion capture software. I will report back when I get some good results out of it (hopefully soon)

TODO

• hand and feet position and velocity matching
• 6 to 8 camera motion capture data
• finish Charuco board for FreeMoCap camera calibration

Video is of the new matching in action the red (and 1 green) points are the players future predicted trajectory the pinkish purple points are the animation fames current trajectory. they should be aligned most of the time

FIRST MATCH!!! WE DID IT! This past week I started working on code to compare and chose a frame to play (the actual matcher) and it works… Kind of? YES IT MACHES… Is it a good match? HECK NO!!!! BUT IT MATCHES!!!! AND THAT JUST MADE 9 MONTHS OF WORK WORTH IT AS I HYPERVENTILATE WATCHING MY MATCHER STUGGLE BADLY LAGING EVERYTHING BUT IT WORKS! now the next step is making it work… properly. the video is of it trying to match. i kinda move around a bit to get it changing data but you can tell when it finds… something because the character will lock onto a frame or start playing the animation.

WE HAVE A WORKING DATASET! Hello hack club! Didja miss me? NO?! WELL TOO BAD!!!!! I now got my motion capture data outputting a dataset with trajectory samples and root positions which is a VERY big step forwards. so as I have mentioned in the past motion matching is literally what MOTION MATCHING is doing. Matching the motion. I have had player predicted trajectory working since before stardance even started but only recently have we started to make progress on the motion capture data that we can MATCH to the players predicted trajectory. the set up I have going on for running the motion data extraction is really cool if I do say so myself. I have a custom editor menu where I can add animation clips and tell it what times in the future I want the trajectory to sampled from then it will compile everything into a scriptable object. plus I also added cool loading bars! anyways we are getting very close to having actual motion matching made from scratch. wow. 9 months this has took so far. its been fun. well… technically not done yet so lets keep up the good work and build the matching algorithm