Devlog 12

So I added the payments page and fixed a lot of bugs.
I have only slept 6 hours (3:30 prev night and around 2 at evening) for the past 35+ hours. I am only one coffee deep rn.

So lets talk about the bugs and all in a little detail.

• The latest bug I was facing was, I am using an paginate plugin in the backend mongoose aggregate paginate v2 so it doesn’t allow complex queries and stuff and there is a option countQuery which is to be set to the same aggregate used in aggregatePaginate, let me show you with the code

const payments = await Payment.aggregatePaginate(paymentAggregate, {
  page,
  limit,
  countQuery: Payment.aggregate(pipeline)
});

• I had no idea this thing existed because the docs are not well properly documented for the plugin.

• First of all I was thinking why isn’t the search working my code is completely right, then I came to the conclusion with the help of AI also debugging with me ofc that aggregatePaginate is the problem then I found out that i can use countQuery.

• Also before this, the search in products table wasn’t working, as I am using a template underneath so I checked the implementation, nuqs package is used for searchParams and for the table I’m using Tanstack Table which also had search filter so I had to get the value from the tanstack table search filter instead of from nuqs which also took me a while to debug.

• Then before that bug, I had a bug where the table weren’t getting updated if actions were performed on the same page without refresh like delete or in client page create or update, so the issue was I was invalidating queris serverSide instead I had to invalidate clientSide.

• Also some minor backend things, invoice get route filters had email / name differently i combined them to one search.

• The index in products on sku + businessId was considering null as one so I added

{
  unique: true,
  partialFilterExpression: {
    sku: { $type: 'string' },
  },
}

• Also in frontend, the fetch wrappers I had were converting all the body into json, so i fixed that If formData is present send it as formData not json.

• Also I’m uploading files and sending the request via server actions in Nextjs, there was a default limit of 1mb so I added config for it in next.config.ts, it is a experimental config though.

I think that is It, Just couple more hours of the project left then I will ship it.

Devlog 11

So I added / edited Clients, Products, Invoices Pages

• Each one of them uses Tanstack Table, Query and Form
• It took my time to understand the structure and all.
• There is nothing to say as the time was taken by the coding part, the type errors I faced and all
• Invoice form was a little complex as I had to add a combobox with debounced search to add products and link to a client.

In this devlog there is really nothing much to say as It was all trial and error as this tech is new to me.

Devlog 14

• Added a debounced username uniqueness check for welcome page and settings page

• Changed few things to make some pages feel less AI like the auth pages, they look cool now.

• Changed readme for both the projects to make it feel more simpler.

• Provided .env.example for both the repos

• Removed some dead code.

• This devlog is for small changes I made.

Devlog 13

• Added responsive design for mobiles
• Fixed few questions were a duplicate of another.
• Initialiased a initial state sync for the UI
• Removed persistance middleware and ensured localstorage cleanup for game and party stores
• Added loading indicators for game start and logout
• The agents card reset button was not working so fixed it
• Valorant font was sometimes hard to read so using Oxanium now
• Removed uppercase styling from some places
• Added server side validation for updating user details
• Implemented migration for anon users to linked accounts.
• Added notifications to oponent on player disconnection.

Devlog 11
So i fixed a bug in backend, zod url validation is strict and also requires the protocol https:// to be in the url but now I am using string with regex.

For the NextJS part

• Sign In, Sign Up page fully done with Backend For Frontend (BFF) architecture.

• Improved fetch wrappers a bit.

• Also in login api route in nextjs the cookies weren’t getting transfered to the browser so fixed that as well.

• Added the home page (/businesses) also added the create business page + a success page with a Confetti canvas animation

• Added api routes in nextjs for fetching user businesses (get) and creating a user business (post).

I’ve been working for almost 8 hours on this for two days, so tired rn goal is to finish this by tomorrow or I can say today as 1:20 am rn.

Devlog - 10

So now that I’m continuing this project, I decided to use a admin dashboard template and adjust it according to my needs as the main thing I wanna show is my backend skills.

• So first of all I was using bun in the backend, I switched to pnpm (personal preference)
• Now I am using tsdown for building the express app.
• There was some issue in extended Request type so I fixed it.
• Also I was using a mongodb cluster free tier, now I am using a mongodb replica set on docker.
• Also the global rate limiter was applied on everything not just /api/v1 so fixed that.

Also I renamed both the repos to something short -> Bms-Backend & Bms-Frontend.

Lets talk about the frontend that is on NextJs.

• I am now using a template which includes a dashboard not fully complete for my need so I will be doing that, it had sentry, clerk and some other stuff, I removed that as It removed a cleanup script.
• Added the login form and attached it with the backend.
• I was previously using axios, but now I am using fetch and I am using a architecture called BFF (Backend for Frotend). So the client components call the nextjs backend and that backend calls the express Api whereas server component directly call the express Api so I created fetch wrappers for client calls and server calls.
• Added a logo for the app.
• Added proxy.ts and its logic

Devlog 09

So, I combine two stardance projects (backend, frontend) into one (fullstack), at the time I thought I could ship both together but I had to have both repos connected in one project, I didn’t know at the time and I then abandoned the project and this project was the backend & I added the frontend repo to it so all the fronted devlogs are gone.

In this devlog I will write the changes made to backend after the previous devlog + I will summarised what was done in the fronted so far and after this devlog, I will write the devlog as normal as you guys will catch up to speed.

Backend (Express)

• In zod using z.number() or .date() were resulting in error if passed as a string as data would always get converted to string before reaching backend so i added z.coerce -> z.coerce.number() or .date
• Added github actions workflow to automatically pull changes and build with docker + fixed perms issue in Dockerfile.
• Added redirecting of / to /docs

Frontend (NextJS)

• Auth Pages are done (Login, Signup, Forgot-Password, Reset-Password) with zod and react-hook-form validation
• Pages are made using shadcn ui components
• Home page is done with listing of business that user is a part of
• Sidebar, header, Create-business-form and BusinessCard, invoice table components are also commited

Devlog 12

So I think this one is the last before ship, I promise. 😂🥹

• Fixed some bugs here and there
• Added readme
• Added some bg and showcase photos
• Added a layout for the home pages (lobby, settings, leaderboard, match history) where the top navbar and the friend sidebar is given to all.
• I think that’s it.

I got no friends, or the friends I have aren’t like best of friends So I wasn’t able to test the game on scale. Feel free to open a issue in github or message me on slack

Devlog 11

So I have done lots of stuff and this is the last devlog before the first ship.

Little sleepy rn as its 2:35 am my time, so I will wrap up fast.

• Added pino in Nextjs (Its a logger)

• Added caching wherever possible in nextjs using the cacheLife

• Added auth refresh event so if the name is changed it doesn’t need socket reconnect

• Optimised some code in socket server and also nextjs.

• Games getting saved to DB and there is a career history page now.

• Mostly did fixes of stuff and redesigned some elements in the lobby page.

Like there is nothing else to talk about I’m mostly fixed stuff.
I gotta sleep, have to ship the project too maybe I do it after waking up because readme is not ready yet.
I think I’m gonna do after waking up so tired rn

Devlog - 10

I will keep this devlog sweet and short as I wanna ship this project as soon as possible as It has taken a lot of time now.

• I added some more rate limiting
• remove game Id / party id payload where neccessary from client in socket server as game/party id can be retrieved from player stored in redis.
• I’m directly getting friendships from db instead of relying on nextjs api in the socket server to show the initial friends list with presence (online/offline status).
• Removed TTL from party and game as they may get deleted mid game (prev ttl was 10 mins).
• I have also added token refreshing in socket server as socket server only verifies token once so now when the token refreshes (only for when user details are mutated) socket disconnects and connects, I know this is a bad logic I will change that but 10h 37m will be logged so I don’t wanna waste any more time , so it will be in next devlog

• Using AVIF instead of PNG for Agents Banner & Icon
• Added new fields to user (tagline, banner, usernamedUpdatedAt, nameUpdatedAt)
• Cleaned up many of the files in the frontend party
• Added logic for If user changes their avatar it should delete the old image in cloudinary
• Added Match ending design
• ENV are now accessed from ENV.ts
• Removed console.logs etc from nexjs
• Fixed some bugs.

Devlog - 9

NextJS

• I added upload avatar in the settings page, user uploads an avatar and crops it accordingly , max file size limit is 5mb also the image gets uploaded to cloudinary and the secure url of the image is stored in db
• Changed the lobby UI to make it less cluttered
• Now there are drawers for gamemode settings and party stuff
• Changed the friends sidebar to be attaced to the right wall and closed and when you open it , it comes out of the screen not occupieing any space.
• Made a UI for FREEFORM questionMode
• Added a animated starting screen.

Socket Server

• Added a new questionMode inside the Guess Agent gamemode now you can have freeform questions or preset
• Preset is same as before you will have a preset of questions to choose from and all its all like before
• Freeform is different you will have two buttons instead of questionPanel, you will have make guess and consume no. This gamemode is meant to be played in a vc like discord or any of your choice you can ask questions to the opponent direclty if there is a no you can just consume no and if you wanna guess you can guess. its like a offline game but you can count the nos and guess and have a agent panel to deduce
• Added a game starting phase where some details are created to show to the frontend.
• Added game cleanup functions, not perfect but will do for now
• Added a new event CONSUME_NO to consume no for questionMode
• Made some changes to type to adjust for freeform question mode

EXTRA

• Added rate limiting in some parts
• In socket server rate limiting is token bucket and in nextjs rate limiting is sliding window counter (somewhat strict rate limiting in guest login and magic link)
• The rate limiting is handled by the same redis used for game states and stuff
• Fixed some bugs here and there.

I know this devlog doesn’t feel personal and I only talk about the stuff I did, its because I wanna Ship this project as soon as possible and get done with it and move on to something else, because projects like this never end and you have to decide when to stop.

I think next devlog will be the last devlog before the first ship

Devlog - 8

Big devlog, here we go again.

Lets talk about nextjs first.
So I added magic link to the app as the feedback I received was I don’t wanna use my google account as I don’t the trust the website enough to use it and I also want to use all the features so guest won’t work. This pushed me to add somewhat of a email login. I dont like password login so I wanted to add passwordless login and it wasn’t a debate I just picked one which is magic link.

• I am using resend for sending emails and i already have a domain for the app so using that for send email with auth@mydomain
• I created a template in resend dashboard itself so i can direclty use it in my nextjs app.

Now lets talk about the big thing which made this a long devlog !!! Drum Roll please 🥁🥁🥁 I decided to restructure many parts of my socket server because of scalability as my previous architecture wouldn’t have survived If i added more gamemodes and all.

What were the flaws

• Room was acting like if it was the party and the game
• Current gamemode is available is guess the agent so all the connecting modules were designed based on that which would have made adding other gamemodes impossible.
• There was no generic game module it was just Guess the Agent

What I did

• Changed room to game and separated game ( previously room ) from party and added a separate events and module for party and party gets stored in redis as well.
• Inside the game module added a core folder where core mechanics (listener, types, service) of each game regardless of gamemode lives.
• Each gamemode has its own folder inside game containing the logic specific to that game.
• Redis player now has a partyId element too.
• Improved player presence (connection and disconnect) and using switch statements in places where if the mode is different some logic might be different.

Why did it took so much time and What I learnt.

Because when I started the restructure I didn’t know how my architecture would look like or what would work ( still haven’t tested anything ). So I researched some stuff but some stuff you decide along the way.
I learnt that no matter what the project is and what you decide at the start that you are gonna build there will always be a time where the plans fail and we have to decide should we go through with it or suffer in the future or should we just focus right now and work hard so that in the future it all comes back to us with a lots of happiness!!!

Cheers,
Neerav

Devlog - 7

Another Long Devlog Time

So I was setting up my project to be deployed on the nest container that I have since nest only gives public ipv6 I couldn’t host my nextjs app on vercel because then I wouldn’t have been able to connect to my postgres database which was on nest as vercel dont have ipv6 support so I moved my nextjs to nestjs and It caused me hell lot of problems and bugs.

• First of all had to make sure nextjs, postgres and socket server were running on same docker network.
• My nest container was crashing when building the nextjs for docker.
• Had to move building process to my laptop then push the image to ghcr.io
• And some silly mistakes that took me long time to fix.

Lets talk about the project what I update in that

• So first of all made a presence for managing friends online status.
• Added logic to connection and disconnection events of player socket. As when players connects through socket it should know whom his friends and what friend requets he got and what friends are currently online and also if he is reconnecting (For now 1minute reconnect window) the socketId should be updated in the needed area (room and player redis store).
• When the socket disconnects, set the socketId in player redis store to null . After the reconnect window is over check if he is in a room, if yes remove him from the room and in both cases delete the player record from redis .
• Also lets expand on connection syncFriends the socket requests data from an internal api route only accessible through the socket for what friendship records are there and formats it in

  friends: [], 
  requests: [
    incoming: [], 
    outgoing: []
  ]

Then socket server retrieves all the playes using their playerid from redis via mget and then returns the same object structure but in friends its adds a online: true/false to the player objects

• Also removed polling for friendships as now socket does them all. So there is less db calls now.
• Fixed few bugs too.

Now let me tell you a funny story. When testing stuff in prod, I was confused why stuff wasn’t working because it was working fine during development then i remembered I didn’t update the .env file in prod. It may have cut some years from my lifetime.

Devlog - 6

This is gonna be a long devlog.

Lets start with the frotend

• I committed the prev work from the previous devlog as mentioned there.
• Added welcome, settings page, Added friends sidebar with realtime functionalities.
• Added a friends store to store incoming friends invites.
• Fixed some file structure to have protected routes inside (protected).
• And mostly this covers it for frotend

Now lets talk about backend that made this devlog of 10 hours

Backend

• I first connected the socket with the player from the db via jwt tokens.
• Stored some player data like id, username inside socket.data.
• Then I changed my room mapper implementation to use player Id instead of socket Id and messed up in some places where socket Id was still getting passed to room mapper which made me spent so much time looking for what is the issue why isn’t my room details getting sent to me and I was looking in nextjs frontend for the issue. I spent so much time finding this bug
• I added some more events for player invites and added a new redis key:value pair -> player:playerid to store player details like id, username, socket id, roomId, lastHeartBeatAt also removed lastHeartBeatAt and reconnectToken from room player.
• Added events for invite, server event of invite:sync and made realtime invite work which took me forever.
• Made some changes in connection event so it handles reconnection

Finally let me tell you something when visting the website if the thing looks small the implementation of that thing takes so much time.

Bro Literally the friends sidebar took so much code to make in the frontend and in the backend and in the socket server as well. like that small piece of the page took me so much time to make which also made this devlog So long.

Incountered so many bugs along the way. They were all silly mistakes some were logic error but most were silly mistakes as there are so many files in the project now its hard to find some things.

Devlog 5

• So I am now using drizzle with postgres and I also took some time to tinker with it and understand it a little better.

• I am also using better auth with google signIn and anonymous login.

• In the user i have username and name , username is for adding friends or locating a user, name is to be displayed , username is unique, name isn’t.

• Created the db model with friends support and matches history.

• We can add friends in the game also added user settings to customise stuff.

• Project is starting to look like something. All the drizzle + backend part that i have done in the period is committed and pushed but the frontend part still remains to be committed.

Devlog 4

• So first of all I switched from using npm to pnpm and updated the docker file accordingly for the socket server.

• Then I started working on adding users / db functionality in the nextjs app.

• I created login and register page with two options for signIn -> google or guest.

• I started tinkering with prisma and better auth and the prisma8 has just come so I started setting it up and had made the schema and had written up some services too but then when I tried to logIn i got few problems which I researched and searched and found out that betterAuth hasn’t adjusted to prism8.

• So I did what I should have done before, I switched to drizzle. Around 3-4 or more hours were wasted!.
  Now I’m using drizzle and have setup the schema and tables and also better auth in it and the signIn are working.

• Now I have to create routes or server Actions to retrieve data or send data. Now the games will be saved and stored once finished and user will able to see it in match history.

Devlog 3

Backend Socket Server

• Made few changes to setting type and added a questionPool in gameState
• Settings now have a option to change totalQuestions and a randomly generated questionPool from the totalQuestions
• Removed spectator logic as not planning on pursuing it right now
• Also added a heartbeat event which supplies heartBeat every 5 seconds or so and if there is no heartBeat provided by the player for over 30 seconds the player gets removed from the room

Now Frontend

• Redesigned the lobby page and home page
• If you open the game for the first time it asks for the name which is currently stored in localstorage for a guest user.
• When users are added then people would be able to login using google and save their details in database.
• For now nothing is stored , only gameState are persistent for the duration of the game in redis and after few minutes of inactivity in the room , the room expires via ttl.
• Also added SEO Optimisations.

Future Plan

• I will either work on adding more gamemodes or adding the user authentication stuff using postgresql and prims direclty in nextjs
• After the auth has been added then I will publish the project so that it has atleast some complete features and users can see their stats and prev games results

Devlog 2

Making this devlog after a long time of doing all this work, the frontend part is still uncommited in github though

Lets talk about backend first ->

• I first of all was using playerId for the routes to verify the user, but now I’m using socketId
• And also for reconnection I’m now using a reconnectToken which is created on the socket server and given to the frontend to it can pass it when reconnecting.
• Also I added a ROOM_KICK event.
• And I added docker setup too but its still buggy
• Changed Winning logic so that first correct guess wins

Now Frontend

• I designed the pages using AI because I am not a good designer rather I am a good coder you can say, because if there is a already made design, I can make a similar one using tailwind and shadcn and other UI libraries
• I added socket setup for the frontend which goes as follow
• src/socket -> index.ts, emitter.ts, events.ts
• src/components/SocketProvider.tsx
• Socket Provider has the listeners
• emitter.ts has small functions that can be directly used inside tsx instead of importing socket
• There are 3 pages currently -> /, /lobby, /play