Low accuracy (~50%) with SSL (BYOL/MAE/VICReg) on hyperspectral crop stress data — what am I missing? [R]

I’m working on a hyperspectral dataset of cabbage crops for nitrogen deficiency detection. The dataset has 3 classes: Healthy Mild nitrogen stress Severe nitrogen stress I’m trying to use self-supervised learning (SSL) for representation learning and then fine-tune for classification. What I’ve done