DNS-Tracker

Slack id finding

The Programm now checks the follows the links in the slack profiles to check if the may redict to GitHub (because some hackclubbers habe www.github.com instead of GitHub.com, and some other use there own url shortener so that my program broke). And due this the number of found slack ids is the first time higher then the not found (310 to 276).

Also if a found some new records and the author/commiter hasn’t been indexed, the program prioritize them so that I can hopefully send the message with the mentions.

Link

If the records I an A, AAAA, CNAME or ALIAS the link in the slack message is clickable so that you can directly open the page.

I also added a link message to the GitHub repository where the change came from.

Deleted records

In my last devlog I said that I want to detect deleted and removed records. I tried that, but it is easier to say then to implement.

Changes

Slack id finder

I updated the GitHub to slack id finder, it now also search for messages, because the chance that someone sendet a link to his/her GitHub account is really high. Before this change I found from the 603 users only 70. After the change I found from 488 users, 234 which is near at 50% (some users are missing, because I couldn’t check does due the rate limiting from Slack).

Records parsing

Due a bug in the parsing logic more then half of the records in the repo are ignored. Like this one:

test:
- type: CNAME
   value: hackclub.com.

The bug is -, the parser has read it as an list instead of an map so I missed 1018 records, it is now fixed.

Ignoring

I also added the possibility to ignore specific repos. Like this one. I use this to test my code locally without spamming the #dns-tracker channel.

Next

The next thing is to detect deleting records on any possible way (removing the lines, reseting head, merging, branch deleting, etc.)

The image contains an proposals from @parth one of the Stardance developers, who adds an new domain staging.stardance.hackclub.com

I have done a lot in the 13 hours

What I added

I build a database migration system so that if I change some tables (which I do more then a should), a migration script is generated and on the next start executed so that I don’t have to manually update the db.

Mentions

Since the start of this project, I always wanted to mention the user, in my dns update message, which has done the changes. Which is easier to say than to make. Before this devlog I tried to do this by Slack APIs which didn’t work, so I thought where all of Hackclubs Users have there Slack Id and there GitHub account: Hackatime. I experimented which there API to find a way to get the slack id from an GitHub-account, which is sadly not possible. So I got back to Slack API and found a way:

1. If I want to get an User I request the Slack API with the account name
2. Then I go through every user id that I found and check the profile of it, if there GitHub profile matches my requested name, then I have the user id
   When a new changes is detected I look in the DB and check if there is a connection.

There is only one problem: The api I use is heavily rate limited, so that I can currently check one user every 5 minutes, but it works.

(Yeah I know 13 hours is to much, but I needed to finish this feature)

Since my last devlog the bot found some new interesting records (before #hackclub-leeks knew them), like the one in the image.

What I added

I changed the logging so that it also logs to a file and is colorful in the console.

What I tried to add

I wanted to make it so that in the message the person is mentioned that added/changed a record. The problem is: There is no way to for my to get a Slack user from its GitHub account. I know there is a profile field for the GitHub account, but I am not able to search for it. So I have some options:

1. I could index all users on Slack, by 124.265 users with 100 requests per minute it takes around ~21 hours (and I don’t think that the FD would like that)
2. Then I found a API. It is called “Slack SCIM API”, and it is perfect, because I can request all users that match a rule (like field GitHub == something). Until I found out that only Admins can use it.
3. I could do it manually. It will take some time (currently 664 users have contributed or forked the hackclub/dns repo) and that is not the only reason why I will not do it

Other things

Also due to some unknown reasons the bot sometimes gets killed by the OOM Killer and I couldn’t figure out why it needs so much ram.

Btw, 3h 10m where coded on my new MacBook Air (from flavortown) and because of the new keyboard layout it takes four times longer to write every line of code and the @ is killing me

Updated the slack message so that it is more clear what changes.
Also made the app more efficient in checking if the main repo or fork had been updated. (It makes 2 http requests which together only take around 1 second). And if nothing has changed, nothing will be added to the github rate limit.

It also runs on Nest and post messages in #dns-tracker.
There are still some bugs with detecting deleted records (in forks)

I created a program that scans the Hackclub/DNS repository, including its forks, to look for new entries or updates.
(Also i think i have the first devlog of stardance) (not the first, but the second)