Ahmed Doghri Logo Image
Ahmed Doghri

fedcal

Federated learning looks clean until every client has a different population. fedcal makes that skew explicit and improves worst-client accuracy from 0.667 to 0.694.

fedcal, a non-IID federated learning benchmark

The Average Client Is A Fiction

FedAvg is easy to explain because the clients disappear into one neat average. Real clients do not disappear. They have different shifts, different label balances, and different ways to punish a global model.

fedcal builds that mess into the benchmark. Six clients get different feature shifts and class skews, then the repo compares plain FedAvg, FedProx, and a conservative local calibration step.

Personalization With A Seatbelt

The local step only moves a client bias if validation accuracy actually improves. That matters because personalization can overfit fast when the local slice is small.

Client updates are clipped, the logistic model is readable, and the whole benchmark stays dependency-free. It is not a full federated platform. It is the failure mode, isolated.

The Number

FedAvg gets 0.829 macro accuracy and 0.667 worst-client accuracy. FedProx plus conservative calibration gets 0.815 macro accuracy and 0.694 worst-client accuracy.

4 tests pass locally and on GitHub Actions across Python 3.9, 3.11, and 3.13.

Tools Used

Python
Federated Learning
FedAvg
FedProx
Client Calibration
pytest
GitHub Actions CI