#Update 22
#Started Coding — Part 7

Hopefully the final stretch! In my next session, I am going to squash this last annoying gray-screen bug on the video feed (fingers crossed it’s just a loose camera wire or a Windows firewall issue).
Once the video is perfectly stable, the real fun begins: integrating MediaPipe! I’ll be adding actual eye tracking so the AI can fuse my iris movements with my head movements. This will give the system an exact, pinpoint location of where I am actually looking, rather than just where my head is pointing.
I also need to verify the MAX30102 heart rate sensor against my Apple Watch to see if the readings are actually accurate, or if my heart rate just naturally spikes while troubleshooting hardware lol. Getting so close to the finish line with this AI fusion model! :)

Update 21

Started Coding — Part 6

Another super long session today, but this was a major breakthrough! I completely overhauled how the system thinks.

The ESP32 was doing way too much math and crashing the video stream, so I moved all the heavy lifting to the Python script on my laptop. I also fixed the crazy false alarms on the crash detection (turning my head is definitely not a 10G car crash lol) and stopped the video feed from freezing whenever I moved my head too fast.

The code also talks to me to map out exactly where the center road and mirrors are based on my specific seat, and actively warns me if I look away for more than 2 seconds.

Next up: Fixing one last gray-screen video bug and finally starting the AI fusion model to combine the head and eye tracking! :)

Update 20

Started Coding — Part 5

Oops, I think I got carried away, 13 hours is crazy lol.

In this session, I was able to fix all the code, and work on all the parts of the project! The project works perfectly now, and I even got video stream onto my laptop :)

The ESP can now decide whether the driver is looking at the road and/or the side mirrors or he is distracted. This is a major breakthrough because now I can work on the stream directly.
Next up: AI fusion model.

Update 19

Started Coding — Part 4

I think I just need one more session before I am done with this!

In today’s session, I fixed all abnormal values, wired it everything together and added a callibration time to the gyroscope (so it knows when the driver is looking at the road and when not). Now, everything works pretty well together, but I still need to work on the AI model which fuses it together :)

Update 18

Started Coding — Part 3

This is my 4th, almost 10 hour session in a row, and honestly saying, I have no idea where time goes once I start working on this project.

I worked more on the code, fixed most of the abnormal values, yet somehow I still sometimes get a heart rate of 12 bpm (i am well and alive) from the sensor :( Apart from that, I was able to start work on the personalised AI model, which fuses all the readings from the sensors to create a personal baseline.

I will work more on trying to get as accurate readings as possible, but for now I think I have to work on the AI model (I have attached how I think the AI model will function and display the values).

Update 18

Started Coding — Part 3

I’m lowk addicted.
I got a new heartrate sensor for my project because I JUST found out, all that time where I though something was wrong with my code, it was just that the previous black heart rate sensor was known to have I2C issues :/
Anyway, I got a new one and soldered it (pretty clean ngl, I’ve learned so much this project). I have also assembled the glasses, with zipties, for a first prototype!! I still get abnormal values sometimes though, so that is something I will have to fix :)

Update 17

Started Coding — Part 2

I completed the code from last session, including the indiviual codes for the ESP32S3, sensors and the ethernet module. I had to rewrite a lot of the code from last session because of how inefficient it was :/

Then, I wired all the components together like my sketch from a while ago, but I was getting I2C issues with the heart rate sensor :( I’ll review the code again, and perhaps replace this heart rate sensor for a new one. I am very excited though because the project is finally coming to life!!!

Update 16

Started Coding — Part 1

I took a break from stardance to work on my school midterms. However, WE ARE BACK BABY!!
Today I started work on coding the ESP32S3 microcontroller. Coding is much more time consuming than making PCBs but I am starting to enjoy writing all these codes. It feels I am connecting puzzle pieces, where with each right one, I am able to a real picture of how the project will turn out.
In today’s session, I worked on trying to get the ESP32S3-Cam feed, along with the values from the sensor to my laptop through ethernet. Using ethernet allowed for a more seamless data transfer (internet speeds where I live are pretty trash). Next session, I have to work on making the AI for this thing which will help create a personalised baseline!

Update 15

KiCAD; Making My First Ever PCB! — Part 2

Today, I was finally able to complete reading the paper on the PCB editor to learn everything (might have over learnt it) about it. I also spent some time on an ESP32S3 pins sketch to make sure I had the wiring correct before I felt ready to make the final PCB!

Then on the PCB editor, I spent about an hour to make wire the ESP32S3 to the sensors and the W5500. I have now completed both of the PCBs needed for the project :)

Update 14:

KiCAD; Making My First Ever PCB! — Part 1

Knowing nothing about the PCB editor yesterday, to making my first PCB today was quite a fun journey. I started the session by reading a guide to the PCB editor in KiCAD, which helped me learn all the basics I needed to make 1 out of the 3 total PCBs!

Even though, I will have to learn a little more about the editor before I can make the PCB for the left temple (ESP32S3, W5500 & MAX 30102) and the front side (MPU6050), I am happy that I was able to complete the PCB for the right temple (Battery, TP4056 & TPS63020) of the glasses :)

Update 13

KiCAD; Making Footprints but Accurately! — Part 2

Starting from where I left yesterday, I began work on the remaining footprints. TPS63020 proved to kinda difficult to make the footprint for because it followed unconventional pin spacing but I was able to find a workaround using my newly attained knowledge(I had to read a little more :/ lol)

I am now done with all the footprints including the battery and the switch. I have placed these footprints in the PCB editor but I unfortunately have no idea how to make the editor work :( I’ll read about it tomorrow and hopefully make the PCB by tomorrow!

Update 12

KiCAD; Making Footprints but Accurately! — Part 1

I started the session off by reading a paper on how to use the PCB editor in KiCAD. However, mid-way through, I realised that the footprints I custom-made for my parts are not accurate :( So instead of working on the PCB, I spent the rest of the session to remake the footprints 1:1.

I was able to make the footprint for the ESP32S3, MPU6050, MAX30102, TP4056 & W5500. This leaved behind a footprint for the battery, the button (no clue how that works) and the TPS63020.

Hopefully, in the next session, I’ll be able to complete the footprints for the remaining components, however given that I don’t currently own the TPS63020, I’ll have to research more on its dimensions.

Update 11:

KiCAD; Introducing Power!

To start the session, I redesigned the ESP32S3’s footprint (I forgot to save the last one). I then moved on to make the footprint for MAX30102 heartrate sensor.

Then, I also included the symbol designs for the battery, battery charger and the voltage regulator. The schematic is fully wired now comprising of the sensors, esp, ethernet and the power :) I have placed some of the footprints in the PCB editor on KiCAD but honestly, I have no idea how the PCB editor works.

Next session I will read some papers on the PCB editor in KiCAD and start working (finally lol) on the actual PCB!

Update 10:

KiCAD; Fixing all errors & assigning footprints!

I started the session off by running an Electric Rule Checker test, which unfortunately gave me 4 errors :( Not sure what to do, I read a paper on wiring in KiCAD, which even though was VERY boring, I was able to pinpoint all 4 errors down to their causes. Thankfully, after debugging for a bit, I was able to get rid of all errors!

I also started assigning footprints to all my sensors and components (had to custom make one for ESP32-S3 because they did not have my exact model), so that I can finally move towards making the PCB come to life :)

Update 9:

KiCAD; We Started Wiring! — Part 1

I started this session off by reading some papers on how to wire components in KiCAD. I am getting bored of reading so much lol but reading these papers are really helpful in trying to understand how the wiring works :)

Following the reference drawings I made, I was able to connect all the components through wires (or so I thought because the first ERC test showed 19 errors lol). I still have 4 errors left to fix, which I will hopefully do tomorrow before I can move on to the PCB!

Update 8:

Learning KiCAD; but I finally understand it! — Part 2

Starting off where I left last session, I built the scematic designs for all the remaining components (i.e., W5500, ESP32-S3, MPU6050). The ESP32-S3 turned out quite tedious because of the 40 number of pins on it :/ but that wasn’t as bad the W5500 component design vanishing out of no where :(

I was able to complete all the components though, which is a plus! I’ll now have to learn how to wire scematics so I can finally wire all the components together and build the PCB :)

Update 7:

Learning KiCAD; but I finally understand it! — Part 1

After yesterday’s uneventful KiCAD session, I looked for more knowledge on KiCAD. That’s where I found a presentation by Oxford University on KiCAD. Reading through the paper was helpful in actually understanding the basics and learning how to design my own components!

Hopefully, later today I’ll be able to finish all 4 component designs and place them in one schematic :)

Update 6:

Learning KiCAD — from the basics!

I started the session off by reading a paper on KiCAD and PCBs. The paper gave me an overview of how PCBs work and what exactly is KiCAD (I knew nothing about either before this). Then, I tried to make the schematic for the project in KiCAD, but I could not find the exact model of my ESP32-S3 :( Looks like I’ll have to manually make it.

Hopefully, by tomorrow I’ll be able to get the models of ESP32-S3, MPU6050, MAX30102 and W5500 in KiCAD!

Update 5:

Hermes Vision; Drawn to Scale! — Part 2

I started a 1:1 scaled drawing of Hermes Vision yesterday. Today I was able to complete the full drawing including the left-outward side, the right-inward side and the right-outward side of the glasses. This complete design will hopefully help out in the making of the CAD for the glasses frame.

I was also drew a wiring diagram of the whole project, providing a reference for when I want to make the PCB for these glasses!

Update 5:

Hermes Vision; Drawn to Scale! — Part 1

On my tab, I started a 1:1 scaled drawing of Hermes Vision. The process was fun but tedious because I had to measure the length of each component by hand using a meter rule (I don’t have digital vernier calipars lol). In around 1.5 hours, I was able to complete the front alongwith the left-inward side of the glasses!

Hopefully by tomorrow I’ll have the complete drawing so I can move on to properly and accurately designing the CAD for my glasses frame :)