RoboSnoop

Devlog3_RoboSnoop_10/06/26
FINALLY aligned the second slider with the first and completed the code for the calculation of the angle for the index finger (unfortunately we have been lied to, A-level mathematics is not completely useless):

#Find vector AB
  const BA = {
    x: A.x - B.x,
    y: A.y - B.y
  };
#Find vector  CB
  const BC = {
    x: C.x - B.x,
    y: C.y - B.y
  };
#Calculate the dot product
  const dotProduct =
    BA.x * BC.x +
    BA.y * BC.y;
#mod AB = (x^2 + y^2)^0.5
  const magnitudeBA =
    Math.sqrt(BA.x * BA.x + BA.y * BA.y);

  const magnitudeBC =
    Math.sqrt(BC.x * BC.x + BC.y * BC.y);
#dot product = modAB * mod BC cos theta (manipulate to calculate theta)
  const angle = Math.acos(
    dotProduct /
    (magnitudeBA * magnitudeBC)
  );

  return angle * 180 / Math.PI;
}

hands.onResults((results) => {

  if (!results.multiHandLandmarks.length) return;

  const hand = results.multiHandLandmarks[0];
#5, 6, 7 represent the different joints in the finger
  const indexAngle =
    getAngle(hand[5], hand[6], hand[8]);

  setAngleValue(indexAngle);

The next steps would be adding 3 more sliders for each of the fingers (and aligning them so that they aren’t floating everywhere), and hopefully accounting for another degree of freedom about the knuckle joint (right now there’s only one degree I calculated for so only one value of theta, but there’s 3 degrees of freedom about the index finger (not accounting for adduction and abduction) and I have to figure out how to work around that or incorporate it into RoboSnoop and ultimately Project Handy

• Snoopsawg

Devlog2_RoboSnoop_10/06/26
I figured out how to attach a slider to the RoboSnoop OS, but I only got it to move based on the position of my hand along the x axis of the screen instead of based on the angle at which my fingers curl.

<script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js"></script>

<video id="camera" autoplay playsinline></video>
<canvas id="output"></canvas>

<script>
  const slider = document.querySelector("#slider");
  let thumbAngle = slider.value;

  slider.addEventListener("input", function () {
    thumbAngle = slider.value;
  });
</script>
<script> 
  const hands = new Hands({
    locateFile: (file) => {
      return `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}`;
    },
  });
  hands.setOptions({
    maxNumHands: 1,
    modelComplexity: 1,
    minDetectionConfidence: 0.5,
    minTrackingConfidence: 0.5,
  });
  hands.onResults((results) => {console.log("Found the handdd!", results)});
  const camera = new Camera (video,{onFrame: async () => {
    await hands.send({image: video});
  }, width: 700, height: 500});
  camera.start();
  hands.onResults((results) => {

if (!results.multiHandLandmarks || !results.multiHandLandmarks.length) return;

const indexTip = results.multiHandLandmarks[0][8]; // index fingertip
const min = Number(slider.min);
const max = Number(slider.max);

const value = min + indexTip.x * (max - min);
slider.value = value;
thumbAngle = value;
});

</script>

Devlog1_RoboSnoop_OS 10/06/26
Today I finished the code to begin the OS: inserting a webcam, adding a filter and headings to the system. Later on I’ll have to update the hand-tracking system so that it can follow along with angles for each finger and update the UI with bars which fluctuate based on micro-movements.

  body {
    background-color: #000000;
    background-image: radial-gradient(circle at top, rgba(28, 32, 64, 0.35), rgba(0, 0, 0, 0.95) 60%);
    color: rgb(255, 255, 255);
    font-family: "Headfont", Arial, sans-serif;
    text-align: left;
    position: absolute;
    left: 20px;
    top: 2px;
  }

  #container {
    margin: 0px auto;
    width: 700px;
    height: 500px;
     border: 5px solid rgba(107, 132, 255, 0.9);
     box-shadow: 0 0 18px rgba(107, 132, 255, 0.78), 0 0 44px rgba(53, 77, 255, 0.42), inset 0 0 24px rgba(0, 0, 0, 0.75);
    position: relative;
  }

  #videoElement {
    width: 700px;
    height: 500px;
    background-color: #000000;
     filter: saturate(4.4) contrast(2.1) hue-rotate(250deg) brightness(1.03) grayscale(0.12) drop-shadow(0 0 10px rgba(105, 145, 255, 0.48));
  }

  #overlay {
    position: absolute;
    inset: 0;
    background:
      linear-gradient(rgba(0, 0, 0, 0.18) 50%, rgba(255, 255, 255, 0.04) 50%),
      linear-gradient(90deg, rgba(40, 50, 120, 0.10), rgba(70, 95, 235, 0.07), rgba(10, 10, 10, 0.16));
    background-size: 100% 6px, 100% 100%;
    mix-blend-mode: screen;
    opacity: 0.7;
    pointer-events: none;
  }

  #overlay::before {
    content: "";
    position: absolute;
    inset: 0;
    background:
      radial-gradient(circle at 20% 20%, rgba(95, 130, 255, 0.38), transparent 24%),
      radial-gradient(circle at 80% 30%, rgba(70, 105, 240, 0.30), transparent 22%),
      radial-gradient(circle at 50% 75%, rgba(190, 210, 255, 0.16), transparent 26%);
    mix-blend-mode: screen;
  }
</style>

<div id="container">
  <video autoplay playsinline id="videoElement"></video>
  <div id="overlay"></div>
</div>

<script>
  const video = document.querySelector("#videoElement");

  navigator.mediaDevices
    .getUserMedia({ video: true })
    .then(function (stream) {
      video.srcObject = stream;
    })
    .catch(function (error) {
      console.error("Whoopsie:", error);
    });
</script>