Devlog #6: Introducing Hierarchical Storage Management (HSM) and R2 Support

Over the past day, I have been working on a major redesign of Minamo’s storage architecture.

From Single Storage to Multi-Tier Storage

Minamo now supports configurable storage tiers, including:

• Hot tier (tier 0)
• Warm tier (tier 1)
• Cold tier (tier 2)

Minamo can automatically manage data movement between tiers based on configurable policies.

New Storage Management Architecture

To support this, I introduced several new components:

StorageManager

Instead of directly interacting with a specific backend, object operations are now managed by StorageManager, which decides:

• Where new objects should be written
• Which tier should serve read requests
• When objects should be migrated

Cloudflare R2 Backend

Minamo now includes a native Cloudflare R2 backend.

This enables using R2 as one of the storage tiers.

Background Migration and Storage Optimization

A new HsmScheduler was added to handle background tasks:

• Tier migration
• Storage eviction
• Periodic heat score calculation

To make these decisions smarter, Minamo now records access patterns through a dedicated access_logs.db.

This allows Minamo to gradually evolve from rule-based storage movement toward more intelligent storage scheduling.

What’s Next

The next development direction will focus on improving compatibility and expanding storage backend options.

Planned tasks:

• Add Hack Club CDN support as a new storage backend
• Improve configuration documentation and examples
• Continue improving HSM scheduling algorithms

Minamo is gradually becoming a more complete storage management system.

There is still a long way to go!

Commit

Devlog #5: Why is it called Minamo?

Someone recently asked me whether Minamo was named after Mixamo.

It actually wasn’t, but I realized I had never shared where the name came from, so here’s the story.

Minamo is derived from the Japanese word みなも (水面, minamo), meaning “the surface of the water.”

The idea behind this name comes from the relationship between the surface and what lies beneath it. From above, the surface of a lake appears simple, calm, and clear. However, beneath that surface exists a much more complex world that is hidden from view.

This is also the philosophy behind Minamo. For clients, Minamo provides a simple and consistent S3-compatible interface — just like a calm water surface that is easy to interact with. Behind that interface, Minamo handles the complexity of storage systems, including different storage backends, tiering, caching, and data management.

Minamo aims to keep the surface simple while allowing the complexity underneath to remain invisible, creating a clean and reliable experience for clients.

I’m currently adding Cloudflare R2 support to Minamo, and I was wondering: what if I also added Hack Club CDN as an optional backend?
Since it offers free storage, it could be a pretty cool addition. What do you all think? Would love to get some feedback!

Devlog #4: Layered Config, HTTP Range & Multi-Backend Foundations

Over the past couple of days, I’ve been working on one of the biggest refactors since Minamo entered its MVP stage.

This update introduces a layered configuration system, HTTP Range support for S3-compatible downloads, a redesigned CLI, and a cleaner runtime state architecture.

To keep the project moving forward, I ended up working on Minamo almost everywhere I could — on my desktop, in GitHub Codespaces, through remote SSH sessions on a Linux server, and even from GitHub Mobile whenever I had a few spare minutes.

Unfortunately, development time spent through remote SSH sessions and GitHub Mobile isn’t tracked by Hackatime, so those hours don’t appear in my statistics.

Also, because this refactor involved lots of experimentation and incremental changes, I squashed many small commits into a single clean commit before pushing. The history is much cleaner now, even though it hides quite a bit of the work that went into this update.

Highlights

Layered Configuration System

The previous environment-variable-only configuration has been replaced with a layered configuration system:

CLI arguments
    ↓
Environment variables
    ↓
TOML configuration files

This makes deployments much more flexible while still allowing temporary overrides from the command line.

The refactor also introduces:

• ConfigManager
• Automatic configuration scaffolding
• config.example/ templates

HTTP Range Support

Minamo now supports HTTP Range requests for S3-compatible downloads.

Clients such as rclone and the AWS CLI can now download only the required portions of large objects instead of fetching entire files.

Supported formats include:

• bytes=start-end
• bytes=start-
• bytes=-suffix

The implementation correctly returns 206 Partial Content, Content-Range, Accept-Ranges, and proper 416 Range Not Satisfiable responses.

CLI & Runtime State

The CLI has been redesigned around subcommands:

• minamo init
• minamo run

Runtime-generated state has also been separated from user configuration, making deployments cleaner and preparing the project for future storage backends.

What’s Next?

The next milestone is adding additional storage backends.

Current roadmap:

Layered Storage Scheduling

Alongside new storage backends, I’m also redesigning Minamo’s storage scheduling architecture.

The long-term goal is to support a layered storage model, where frequently accessed (“hot”) data stays on fast local storage while less frequently accessed (“cold”) data can be transparently moved to lower-cost backends such as OneDrive.

Since Minamo hasn’t seen production workloads yet, this design is still largely based on theoretical considerations rather than operational experience. I fully expect it to evolve as the project matures and real-world usage reveals better trade-offs.

Cloudflare R2

This will likely be the first backend after the local filesystem.

Cloudflare R2 is surprisingly developer-friendly: it only requires a credit card for verification, includes 10 GB of free storage, and charges no egress fees. For many personal and small projects, I think it’s one of the best object storage services available.

Microsoft OneDrive

After R2, I plan to implement a OneDrive backend.

I wouldn’t recommend using a personal OneDrive account as production object storage. It isn’t designed for that, can be less reliable, and is relatively easy to hit rate limits (HTTP 429).

However, for personal projects or storing infrequently accessed “cold” data, it’s an incredibly cost-effective option and fits Minamo’s planned hot/cold storage architecture quite well.

Commit

Devlog #3: Designing a New Configuration System

I am currently working on gradually refactoring Minamo’s configuration system.

Previously, Minamo relied mainly on MINAMO_* environment variables for configuration. As more features and storage backends are introduced, this approach becomes increasingly difficult to maintain and does not clearly separate normal configuration from sensitive information.

The new design will move towards a TOML based configuration system. Configurations will be split into multiple .toml files. This keeps the configuration structure clearer.

I chose TOML because it is easier to represent options and allows users to directly review and edit their configuration.

Configuration files will be ignored by git and initialized from config.example/ on the first run, ensuring that users have a clear starting point without accidentally committing private information.

I am also implementing a new ConfigManager to unify configuration loading, with the following precedence:
CLI arguments > ENV variables > CONFIG files
This keeps the flexibility of temporary overrides while providing a stable configuration system for deployments.

These changes are still being gradually implemented and have not been released yet. The goal is to make Minamo’s architecture more suitable for long-term usage and easier to extend in the future.

Devlog #2: Local Disk MVP Complete

Today marks an important milestone for Minamo, the Local Disk MVP is now complete.

The first goal of this project was never to build a full storage platform overnight. Instead, I wanted to establish a solid foundation: a correct, well-structured, and S3-compatible server with clean separation between the API, service, metadata, and storage layers.

With the MVP completed, Minamo now supports:

• S3-compatible object and bucket operations
• Multipart uploads
• Presigned URLs
• AWS Signature Version 4 (header & query authentication)
• AWS-chunked streaming uploads
• A pluggable storage backend architecture, with Local Disk as the first implementation

Compatibility has also been continuously validated against the official AWS SDK, giving me confidence that the foundation is ready for what’s next.

That said, this is only the beginning.

The original vision for Minamo has always been much larger than a local storage server. The next stages will focus on turning Minamo into an intelligent storage gateway capable of routing data across multiple storage providers, automatically managing hot and cold data, introducing caching, and making storage both more efficient and more affordable.

There’s still a long road ahead.

Thanks to everyone following the project so far. More exciting updates are coming soon!

Commit

Devlog #1: Building Minamo

I’m excited to share the very first development update for Minamo.

Minamo is an S3-compatible storage gateway designed to reduce storage costs while remaining fully compatible with existing S3 clients. The long-term goal is to intelligently manage object placement, keeping frequently accessed data on fast storage while transparently moving cold data to lower-cost storage.

The project is currently about 50% complete.

Why Minamo?

The idea came from developing my AI chat application, Arona Chat.

While building its attachment system, I noticed that most uploaded files are accessed frequently for only the first few days before becoming “cold” data. They still need to be retained, but keeping everything on expensive hot storage unnecessarily increases operating costs.

That led to a simple question:

What if an S3-compatible gateway could automatically keep hot data on fast storage while moving cold data to lower-cost storage?

That’s how Minamo began.

Development Strategy

Rather than building the complete multi-tier storage system from the beginning, I’m developing Minamo in stages.

The first milestone is a Local Disk-only MVP, focusing on implementing a correct S3-compatible server before introducing additional storage providers.

To make future expansion straightforward, the project already adopts a layered architecture:

• HTTP / S3 API Layer (FastAPI) — request handling, XML responses, AWS Signature Version 4 verification
• S3 Service Layer — business logic and S3 error mapping
• Metadata Layer — provider-independent SQLite metadata
• Storage Backend Interface — pluggable abstraction with a Local Disk backend for the MVP

This separation allows future storage backends to be added with minimal changes.

Current Progress (~50%)

The MVP is actively under development.

Implemented so far:

• PutObject / GetObject / DeleteObject / HeadObject
• ListObjectsV2 (prefix, delimiter, pagination)
• Bucket operations
• Multipart Upload
• Presigned URLs
• AWS Signature Version 4 (header & query authentication)
• AWS-chunked streaming uploads

Compatibility is continuously verified against the official AWS SDK, with 22 boto3 compatibility tests currently passing against a live server.

The remaining work mainly focuses on completing the MVP and polishing compatibility before introducing additional storage providers.

Looking Ahead

Alongside finishing the Local Disk MVP, I’m designing Minamo’s future storage orchestration system.

The long-term vision is an intelligent storage gateway capable of automatically managing data across multiple storage tiers, including high-performance local storage, cache layers, and low-cost cold storage, while remaining fully compatible with the S3 API.

Hopefully Minamo can help make self-hosted object storage both more affordable and more efficient.

Thanks for reading! If you’re interested in the project, feel free to follow the project. I’ll share another devlog once the Local Disk MVP is complete and development moves into the multi-storage stage.