An attempt to unify all compute devices at one's disposal and run local models - smolcluster

I’m currently building smolcluster, a project focused on demystifying how distributed learning actually works under the hood, both for training and inference. This initiative distills complex information into digestible content for anyone interested in learning more about these algorithms, like FSDP