Updates

Quite a bunch of updates this time, a lot of work has been done, and i feel like I was nearly killed by fd.

Okay so Shiptalkers is now opt in.

Most of this devlogs time was spent:

• Adding a message locator system, basically maps out where every message is instead of storing content of it
• Process messages without storing message content
• Backfill message content for users who opt in
• Add a custom shiptalkers id assigned to messages so we dont store sender id of each message.
• To opt in you can send “Opt In” in the shiptalkers channel

Apart from that I migrated from in file SQL to a proper migrations system.

General updates

• Added mobile support

• Massive cleanup of AGENTS.md
  0 Full fetch and scrape all messages run in parrallel now.

• Archived channels card for stats page.

• Now persists thread rescans, added a thread timestamp index, persisted thread high-water marks.

• Reduced repeated history-wide scans

• Added durable consent backfill progress and fixed backfills blocking fresh message checks

• Fixed a per message update of the shared database counter row. It now updates per statement

• Score recomputation is now hourly instead of passly.

I think thats about it.
Main point of this is that we got scraping to be way faster, and we got opt in requirement.