GitHub: https://github.com/theycallmeloki/MiladyOS

Problem:

GPU Provisioning SaaS/dApps do not expose the server side sys-admin tooling that is required to operate large GPU clusters because that is their moat.

MiladyOS is an open-source container management toolkit that supports wan-wide operations as a thin alternative to Kubernetes deployed on federated GPU clusters. MiladyOS is designed to be simple to use and operate, making it ideal for small to medium-sized organizations or teams that want to leverage containers without the complexity of managing multiple clouds/clusters using kubernetes.

Key features of MiladyOS include:

Privileged container orchestrator: MiladyOS can let you run mining, inference, large scale pre-training, and actual full finetune / from scratch training jobs.

Simple to use and operate: When you connect multiple machines on the same network that have MiladyOS installed, they talk to each other and can understand the compute units that can be configured for training/inference use in a protocol configurable way so it can be stitched together for larger jobs.

WAN-wide operations: MiladyOS supports WAN-wide operations, which means you get a virtual overlay network that lets you assign jobs to compute units across the internet. The IP address of the individual nodes along with job metadata are shared in a decentralized p2p protocol to prevent centralization at all costs

Gateway: Everything you need to get up and running with ingress is batteries included, just run the project, point your domain, configure credentials, and get access to your entire fleet from your codebase.

MiladyOS is a promising alternative to Kubernetes for organizations that want to leverage containers without the complexity and overhead of Kubernetes mixed with federated cluster orchestrators. Artificial complexity is the root of most evil and that makes MiladyOS a lightweight, efficient, and easy-to-use container management toolkit that provides all the essential features needed for managing containers in a cluster with tens of GPUs spread across geographically

MiladyOS is an open-source project, specifically on the Viral Public License, meaning that it is free to use and modify.

The ultimate goal of MiladyOS would be to restore the former glory of GPU mining days, in the modern day Gen AI landscape, so you a fellow GPU operator can maximize returns on your investment, more so that whan the current web2/web3 marketing landscape is able to offer for the same compute by leveraging on chain settlements based on scraped prometheus metrics.

As an aspiring neochibi web3 niche internet microcelebrity status esoteric hipster sys-admin indie maker, it brings me immense joy to be able to work on cutting edge consumer facing GPU orchestration tech and hope to take MiladyOS to great heights ^_^