Determinist
- 14 Devlogs
- 76 Total hours
A deterministic password generator available from your terminal and your browser!
A deterministic password generator available from your terminal and your browser!
Finally, I figured out how to make the site work offline…
After working on the PWA for way longer than I expected, it finally works as intended!
After I’ve figured out how to use next-pwa, I encountered an issue: the site was being cached and loaded correctly in the browser but not when installed as a PWA. I rewrote multiple parts of the code for it to work, yet it did not. In the end, it was the unsecured connections that were at fault.
Discovering that, I have learned to use something new again: Localtunnel, which allows me to use a valid (not self-signed) HTTPS connection while testing. This uncovered even more issues with my code, the most obvious being the fact that fonts were not cached at all.
This was because I was getting my fonts by connecting to fonts.google.com. This was a really easy thing to fix though, as Next.js already has a next/fonts/google library that had the fonts I used, and now everything works well.
Quite a few issues, but apparently it works…
Well, making Determinist a PWA certainly took way longer than expected. I thought it would be an hour at most, I mean Next.js literally has a guide that tells me what to do step-by-step. It was in fact NOT easy. I encountered a ton of problems.
First off, I had to write a manifest which had the app name, icons etc., the obvious things. What was not obvious however was that I had multiple ways of doing that. I could write a JSON file, or write a JavaScript file that exports an object. I chose the latter as it was the version that the Next.js guide was using.
Then came the icons. Oh, the icons. I made my icons and put it in the public/ directory with the correct names so that they match the manifest. Cool, works when I share it but there is no icon when I add it to my home screen on my iPhone. It said I’m missing a file called apple-touch-icon and apple-touch-icon-precomposed. Cool, I duplicated the icons and I renamed them. No 404s, but it still did not work. I look for a solution. For hours. I look at Next.js docs, YouTube, StackOverflow, Reddit, GitHub issues and there is nothing that would fix it. Finally, I found out I need to link the icon in the head of the website. Great, but still doesn’t work. Turns out, the problem was that I was following the Next.js guide which told me to use HTTPS to test the PWA, but iOS does not trust self-signed keys so it did not download the icon.
After that, the generation broke when I was connecting to the site with the IP and not through localhost. I noticed that submitting (pressing the generate button) redirected me which should not have happened due to the line e.preventDefault(). Then I saw Next.js attempting to connect a websocket, and a bunch of 403 errors that weren’t there before. I spent quite a bit of time debugging that, and it was due to not adding my IP to allowedDevOrigins in the next.config.mjs file despite it working perfectly fine before.
As you can see, the image I decided to include in my devlog is the icon for Determinist. I actually did not think of putting the D and the E together at first, but when I did, it reminded me of a lock (hope you can see it!) and I chose to keep it as it kind of fits. And I didn’t want a solid color background, so I decided I’ll just add the name repeating. It was fun making it, and it was also a very good way to practice using Illustrator.
Well, it took me more than a week, but the v3 algorithm is on the website!
I have published version 0.5.2 of the Python CLI, which fixed a few issues I found (you can see them on the GitHub page!), and added actual functionality to the default presets. You can also report issues on the GitHub page, and it is highly appreciated.
I added the v3 algorithm, after a bit of tinkering with the error handling. My problem was that the error handling wasn’t really dynamic. It couldn’t handle the invalid input errprs from v3, so I made a separate component. I also had to learn how to use HKDF on the client side, which was actually easier to do than I expected, although the actual usage of HKDF was a bit more complicated than it is in Python.
After the first week of school, I have finished work on the CLI, and Determinist 0.5.1 is released!
presets create commandg prefix from the commit hash is now removedFirstly, I just started school, so I got very busy lately and that is why I haven’t been doing much work. As a compensation, v3 of the algorithm is now available on GitHub!! This is version 0.5.1, which is a pretty big jump if you look at the versions before. However, there is a reason for that: active development of Determinist is going to stop once version 1.0.0 comes out, with all the new features and fixes being ported to the website. This will come out in the next week or so. Determinist will still be updated here and there if I miss some features or people have suggestions and issues, but I will be mainly focusing on other projects.
The preset creation was actually pretty easy. I figured I’ll reuse the options from the generate command so I don’t have to write them manually again, and it worked amazing! I had to add to the PresetHandler to make it work, and that was really simple as well. I first thought of adding strings together, but then I realized I could just create dictionaries that my TOML library can convert to TOML and write them to the preset, so I didn’t have to do all the writing myself. The bug fixes weren’t interesting at all, honestly. I messed around with Determinist, found bugs and issues and I fixed them. My favorite one was debugging errors with the preset commands however. I found a bunch of things worth fixing, and in the end I made another helper function, as well as changing how check_file_type works.
Updates here and there, and the v3 algorithm is finally complete! All of this and the content of the rest of the devlogs will only be released with the final 1.0.0 release of Determinist.
generate prompt only prompt users if the prompts are applicable. This means you will no longer be prompted for the settings of v1 when you use one of the other two algorithms.site_name option is omitted, the prompt will display Master pass instead of the correct prompt.Well, of course the development of v3 came with some challenges. Firstly, I had to figure out how to actually generate enough bytes to pass rejection sampling (making sure no bytes are out of range when mapped onto the character set), as the previous implementation was that I multiplied the password length by 32. This worked, but wasn’t pretty and in very rare and unlucky cases, there weren’t enough bytes. I fixed this by using something called HKDF, which stands for HMAC-based Extract-and-Expand Key Derivation Function. I’d really prefer to use HKDF. Anyway, this function was really good for this task, as it can take the Argon2 output and create a long, pseudo-random key that can go through rejection sampling, and finally end up in the password. In case this generated key is not enough, HKDF will generate more of it, essentially giving us infinite bytes to then map to our characters. This was especially good because in the same loop I could implement…
This should have been easy, but it took me a day to think about how to do it. The challenge: take a dictionary ({"lowercase": 2, "uppercase": 3, "special": 1, "digits": 0}), and include at least as many characters of each type as their value in that dictionary says, and if it’s 0 then don’t include it at all. Fine, I said, but I didn’t actually know how to implement this. After a day of thinking, it finally clicked: I implement this in the rejection sampling loop, as that is where my characters get added. Introducing the _construct_pass function: it looks for the already added characters, determines the type of the current character, counts how many are missing, and if a character type is missing some characters or there aren’t any left, the character is added, and otherwise it is discarded.
g prefix appearing in the version command output before the git commit hashLong time no see y’all, sorry for the wait! I’m on vacation so I had less time to work on the project, but I did bring some amazing improvements.
The website is officially up and works identically to the CLI, now with presets! Just load your TOML presets into the website, and they will be saved for you to reuse! Important note: they are saved locally, therefore you will not see them on different devices unless you import them there too.
There actually weren’t a lot of challenges for once. The biggest one was that I forgot to work on the project while on vacation, but that’s not horrible. And then came React, JavaScript and everything else… firstly, the preset loading was actually really easy, I did however fight with React about passing values from a child component to the parent. And then it turned out I forgot to make it actually pleasant to use, as I had to select the preset file every time I wanted to load it. Now you can save, select and load your presets on the website. However, React had other plans.
I finally learned how React’s useEffect hook works, and using it is pretty easy now that I get what I’m doing. Along with this, I also properly learned about using functions inside useState for lazy initialization which I needed for localStorage. Oh, localStorage was a pleasure to learn and use… Well, the operations with it are actually fairly easy, it’s like an object, and my first thought of initializing a state with it was to use useState({ ...localStorage }), which did work, but it introduced something else: a build error after I merged the changes into main.
Turns out, localStorage is undefined when GitHub Actions wants to build my website, therefore it gave me an error saying so. What I had to do is check if the window object is equal to 'undefined', and if not, then spread localStorage and initialize the state.
Finally, the website managed to build properly, but when I tried to use it on my phone, the output of the generator was outside my screen… so I had to go back and change how the UI looks after I’ve already merged everything into main. Amazing. Well, the good new is that it works, and next up is the v3 algorithm, as well as QoL changes, bug fixes, and more in both the CLI and web versions.
The UI is mostly complete, and I will be releasing the website tomorrow!
<select> instead of the regular <input>, so you don’t have to type them outThere really wasn’t anything too hard about making the UI. You might ask, “why did you spend over 6 hours on it then?”. I have an answer for you. I tried using Figma for designing the UI but it was just too boring for me, so I decided to start coding it and figure it out as I go. I’m pretty satisfied with the result, but with everything, it can always be improved.
The second problem was that while I was coding, I was also listening to music and I spent a bunch of time skipping songs because I didn’t feel like listening to them. This took my attention along with singing a bit too much, so my progress was slowed…
The last thing to slow me down was my general inexperience with React and JS, although I got over that quick because the frontend is way easier to work on.
Well, it’s official. I had to use AI for the first time :(
JavaScript does not have the same random implementation as Python does. Python uses MT19937 while JS uses Xorshift128+. This means we can’t use JavaScript’s random number generator, not to mention that we can’t even use it to shuffle arrays, as it lacks the method. And since I am relatively new to JS, I had two options: get a 15 year old module work that does exactly what I need it to, or make it with AI. I tried getting that package to work. I really did. Unfortunately, the algorithm it uses seems to be outdated, as the results differed in tests. That is why I used AI. I really didn’t want to, but I had to. Well, let’s look at the positives, at least!
Pretty big milestone: V1 is finally working on the website!
V1 is complete and works perfectly! I had to figure out a way to enter objects, and it was quite difficult. See, JavaScript doesn’t really have a dictionary type like Python does, so I had to use an object, which is similar enough and I can look for keys with in, just like in Python. The tricky part was creating such an object from the user’s input. I chose the most user friendly way I could think of, and made separate inputs for the key and value, and you can dynamically add more with an “Add Character Pair” button. I really wish there was an easier way and maybe there is a template out there, but I didn’t consider it until writing this very sentence. Anyways, tomorrow I’ll spend more time refining V1, the inputs and I’ll start working on V2.
I’ve started working on the website finally!
I’ll use GitHub Pages to host Determinist until I can get my own domain and personal site up, after which I’ll transfer it there. The plan is to make the generator client sided by using WebAssembly to access Argon2 in the browser without requiring a backend. I wanna do this so that no information leaves the user’s computer, which is a key part of keeping Determinist privacy focused. I chose to use Next.js because I used it before, and it has Static Site Generation which is essential if I want to put the site on GitHub Pages.
Yes, that’s about it. I spent most of my time debugging errors and learning basics as I’ve only used JavaScript for making components, not to make more complex algorithms like Determinist’s is. I am a fast learner though, so I expect to be able to recreate Determinist V1 relatively quickly. Starting with WASM on my first serious web project where I’m still learning JS maybe wasn’t the most amazing idea, but hey, I like challenges and learning by doing.
Thank you! Developing Determinist has been lots of fun and I’ve learned a lot of valuable things. Your support and feedback is greatly appreciated, it motivates me to work every single day. It’s a project I’m really passionate about, as I wanted to create this for years. Thank you all for your support, and I hope you’ll enjoy using Determinist whether it’s in your terminal or the web.
See you in the next devlog!
Developer experience and cleanups
version command shows what version Determinist is on, what commit it was built from, and the Python version it uses.I was testing the generation of the default preset, and I forgot to press tab when deleting the Determinist config directory. I only realized that when Hyprland started screaming at me about missing configurations. I thought “that’s weird”, so I checked the command I ran. rm -rf ~/.config/. I deleted my entire configuration directory… Luckily, I had backups, but at least I’ve learned that I should double check my commands.
To absolutely no one’s surprise, Typer didn’t work as intended, once again. It kept reporting the wrong error when running presets subcommands. Turns out it was the callback I wrote to list the presets. It actually ran the check every time a presets command was run, giving the wrong error when a non TOML file was detected in the presets directory. It was a quick fix, but now I’m quite comfortable with Typer.
Determinist v0.2.2 is here and available to install, but I certainly did not thing this would take another 10 hours…
generate command.
presets command lists out all installed presetspresets save with a path added at the end will save the selected preset to your config folderdelete after presets with the name of a preset will delete that permanently
presets default and adding the preset name will set that preset as default--help is actually helpful now!prompt a subcommand of generate. I ran into an issue while implementing this…You see, Typer cannot tell the difference between a positional argument and a subcommand, meaning that it took prompt as the master_pass argument for generate. This is obviously not at all what I intended, so I needed a way to figure out how to solve this. I didn’t want to make prompt an option, since that would’ve complicated things a lot more than they needed to be. Therefore, I changed master_pass and site_name to required options, which means they need to be typed out manually. Fear not however, because if you forget to do that, Determinist now prompts you for it!
Another problem was with types, again. Typer apparently doesn’t like Union types, which I would’ve used for the preset option. What I wanted to do is make the user be able to type out the name of the preset, select one if none is entered in the command, or do everything manually. Problem: that’s 3 types: None, str and UNSET, which is basically a different spelling of None, it jut signals that the presets option was passed without an argument, as a flag. But Typer decided to ruin my day and only let me use two types, so I removed it all and made --preset a boolean, therefore a flag.
That’s about it for now, if you like the idea of having an easy way of remembering your passwords, check out Determinist!
Feedback
Works great! Good luck in voting!
Shipwright walkthrough
Feedback
As this is a python project that is packaged into wheel, please make the package available on pypi (The package index for pip/python) - https://pypi.org/. GitHub release of wheel file cannot be accepted. Sorry for the inconvenience and feel free to resubmit when you have fixed this.
Shipwright walkthrough
Welcome back to the second devlog of Determinist! It’s been a while, so let’s recap what happened…
The first official release of Determinist is here! You can check out the GitHub page to learn more.
Quite a few changes here.
I made my CLI with Typer, which was a challenge to say the least. This is my first time using it, and looking back, it wasn’t THAT bad.
All I had to do is refactor generator.py completely, and make my types compatible (which I worked REALLY HARD on for V1).
Typer doesn’t allow me to use things like lists and dictionaries, both of which are essential, but I managed to change everything in a way that’s mostly straightforward. I can split strings into a list and use JSON for dictionaries.
This is pretty funny. I wrote a series of tests for version 2 of the algorithm before I started implementing Typer. After than however, all tests passed except one. I was supposed to use assertUnequal(), but used the opposite instead, and that was I caught an error in my code that would’ve went unnoticed.
The tests saved me another time, when I noticed that changing the order in which the charset character types are written changes the output, which is not what I need because it’s essentially the same input, and it should be more like a toggle.
The fix was making the charset canonical. The algorithm goes through the list, writes the correct characters into a string, and uses that in the salt instead of the list of the charset.
Generator class was removed, as it served no purpose.Hey everyone! This is the first devlog of Determinist, a deterministic password generator.
After a few hours of messing around, I’ve managed to write the first working version of the generator algorithm. (Flowchart attached)
You can specify the parameters such as password length, special character frequency, etc. as well as the master passphrase and the salt (site name in this case), and the script will generate a pseudorandom password for you.
I’ve hit quite a few roadblock along the way, such as the salt being too small. This is a feature of Argon2, so for now you can only use custom salts that are bigger than 8 bytes.
I’ve also included a few unit tests as proof of concept. The generator algorithm will be improved in later versions, and a CLI tool will be available.