# ModelZoo

FederatedScope provides many built-in models in different deep learning fields, including Computer Vision, Natural Language Processing, Graph, Recommendation Systems, and Speech. Furthermore, more models are on the way!

To use our `ModelZoo`

, set `cfg.model.type = Model_NAME`

. And you can configure the model-related hyperparameters via a`yaml`

file.

```
# Some methods may leverage more than one model in each trainer
cfg.model.model_num_per_trainer = 1
# Model name
cfg.model.type = 'lr'
cfg.model.use_bias = True
# For graph model
cfg.model.task = 'node'
# Hidden dim
cfg.model.hidden = 256
# Drop out ratio
cfg.model.dropout = 0.5
# in_channels dim. If 0, model will be built by data.shape
cfg.model.in_channels = 0
# out_channels dim. If 0, model will be built by label.shape
cfg.model.out_channels = 1
# In GPR-GNN, K = gnn_layer
cfg.model.gnn_layer = 2
cfg.model.graph_pooling = 'mean'
cfg.model.embed_size = 8
cfg.model.num_item = 0
cfg.model.num_user = 0
```

For more model-related settings, please refer to each model.

## Computer Vision

**ConvNet2**

ConvNet2 (from`federatedscope/cv/model`

) is a two-layer CNN for image classification. (`cfg.model.type = 'convnet2'`

)`class ConvNet2(Module): def __init__(self, in_channels, h=32, w=32, hidden=2048, class_num=10, use_bn=True): ...`

**ConvNet5**

ConvNet5 (from`federatedscope/cv/model`

) is a five-layer CNN for image classification. (`cfg.model.type = 'convnet5'`

)`class ConvNet5(Module): def __init__(self, in_channels, h=32, w=32, hidden=2048, class_num=10): ...`

**VGG11**

VGG11 [1] (from`federatedscope/cv/model`

) is an 11 layer CNN with very small (3x3) convolution filters for image classification. It is from Very Deep Convolutional Networks for Large-Scale Image Recognition. (`cfg.model.type = 'vgg11'`

)`class VGG11(Module): def __init__(self, in_channels, h=32, w=32, hidden=128, class_num=10): ...`

## Natural Language Processing

**LSTM**

LSTM [2] (from`federatedscope/nlp/model`

) is a type of RNN that solves the vanishing gradient problem through additional cells, input and output gates. (`cfg.model.type = 'lstm'`

)`class LSTM(nn.Module): def __init__(self, in_channels, hidden, out_channels, n_layers=2, embed_size=8): ...`

## Graph

**GCN**

GCN [3] (from`federatedscope/gfl/model`

) is a kind of Graph Neural Networks from Semi-supervised Classification with Graph Convolutional Networks, which is adapted for node-level, link-level and graph-level tasks. (`cfg.model.type = 'gcn'`

,`cfg.model.task = 'node'`

)`class GCN_Net(torch.nn.Module): def __init__(self, in_channels, out_channels, hidden=64, max_depth=2, dropout=.0): ...`

**GAT**

GAT [4] (from`federatedscope/gfl/model`

) is a kind of Graph Neural Networks from Graph Attention Networks. GAT employ attention mechanisms to node neighbors to learn attention coefficients, which is adapted for node-level, link-level and graph-level tasks. (`cfg.model.type = 'gat'`

,`cfg.model.task = 'node' # node, link or graph`

)`class GAT_Net(torch.nn.Module): def __init__(self, in_channels, out_channels, hidden=64, max_depth=2, dropout=.0): ...`

**GraphSAGE**

GraphSAGE [5] (from`federatedscope/gfl/model`

) is a general inductive GNN framework, from Inductive Representation Learning on Large Graphs. GraphSAGE learns a function that generates embeddings by sampling and aggregating from the local neighborhood of each node, which is adapted for node-level and link-level tasks. (`cfg.model.type = 'sage'`

,`cfg.model.task = 'node' # node, link or graph`

)`class SAGE_Net(torch.nn.Module): def __init__(self, in_channels, out_channels, hidden=64, max_depth=2, dropout=.0): ...`

**GPR-GNN**

GPR-GNN [6] (from`federatedscope/gfl/model`

) adaptively learns the Generalized PageRank weights so as to jointly optimize node feature and topological information extraction from Adaptive Universal Generalized PageRank Graph Neural Network, which is adapted for node-level and link-level tasks. (`cfg.model.type = 'gpr'`

,`cfg.model.task = 'node' # node or link`

)`class GPR_Net(torch.nn.Module): def __init__(self, in_channels, out_channels, hidden=64, K=10, dropout=.0, ppnp='GPR_prop', alpha=0.1, Init='PPR'): ...`

**GIN**

GIN [7] (from`federatedscope/gfl/model`

) generalizes the Weisfeiler-Lehman test and achieves maximum discriminative power among GNNs from How Powerful are Graph Neural Networks? which is adapted for graph-level tasks. (`cfg.model.type = 'gin'`

,`cfg.model.task = 'graph'`

)`class GIN_Net(torch.nn.Module): def __init__(self, in_channels, out_channels, hidden=64, max_depth=2, dropout=.0): ...`

## Recommendation System

**MF models**

MF model [8] (from`federatedscope/mf/model`

) has two trainable parameters: user embedding and item embedding. Based on the given federated setting, they share different embedding with the other participators. FederatedScope achieves`VMFNet`

and`HMFNet`

to support federated MF, and both of them inherit the basic MF model class`BasicMFNet`

.

```
class VMFNet(BasicMFNet):
name_reserve = "embed_item"
class HMFNet(BasicMFNet):
name_reserve = "embed_user"
```

## Speech

Coming Soon!

## References

[1] Simonyan, Karen, and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition.” *arXiv preprint arXiv* 2014.

[2] Hochreiter, Sepp, and Jürgen Schmidhuber. “Long short-term memory.” *Neural computation* 1997.

[3] Kipf, Thomas N., and Max Welling. “Semi-supervised classification with graph convolutional networks.” *arXiv* 2016.

[4] Veličković, Petar, et al. “Graph attention networks.” *ICLR* 2018.

[5] Hamilton, Will, Zhitao Ying, and Jure Leskovec. “Inductive representation learning on large graphs.” *NeurIPS* 2017.

[6] Chien, Eli, et al. “Adaptive universal generalized pagerank graph neural network.” *ICLR* 2021.

[7] Xu, Keyulu, et al. “How powerful are graph neural networks?.” *ICLR* 2019.

[8] Yongjie, Du, et al. “Federated matrix factorization for privacy-preserving recommender systems” *VLDB* 2022.