Skip to content

Commit

Permalink
Update ShareBottom
Browse files Browse the repository at this point in the history
  • Loading branch information
xpai committed Nov 5, 2024
1 parent ebb37a1 commit 4892b6b
Show file tree
Hide file tree
Showing 12 changed files with 22 additions and 19 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ Click-through rate (CTR) prediction is a critical task for various industrial ap
| 50 | NeurIPS'22 | [APG](./model_zoo/APG) | [APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction](https://arxiv.org/abs/2203.16218) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/APG) | `torch` |
| 51 | KDD'23 | [PPNet](./model_zoo/PEPNet) | [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://arxiv.org/abs/2302.01115) :triangular_flag_on_post:**KuaiShou** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/PPNet) | `torch` |
|<tr><th colspan=6 align="center">:open_file_folder: **Multi-Task Modeling**</th></tr>|
| 52 | MachineLearn'97 | [SharedBottom](./model_zoo/multitask/SharedBottom) | [Multitask Learning](https://link.springer.com/article/10.1023/A:1007379606734) | | `torch` |
| 52 | Arxiv'17 | [ShareBottom](./model_zoo/multitask/ShareBottom) | [An Overview of Multi-Task Learning in Deep Neural Networks](https://arxiv.org/abs/1706.05098) | | `torch` |
| 53 | KDD'18 | [MMoE](./model_zoo/multitask/MMOE) | [Modeling Task Relationships in Multi-task Learning with Multi-Gate Mixture-of-Experts](https://dl.acm.org/doi/pdf/10.1145/3219819.3220007) :triangular_flag_on_post:**Google** | | `torch` |
| 54 | KDD'18 | [PLE](./model_zoo/multitask/PLE) | [Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/10.1145/3383313.3412236) :triangular_flag_on_post:**Tencent** | | `torch` |
|<tr><th colspan=6 align="center">:open_file_folder: **Multi-Domain Modeling**</th></tr>|
Expand Down
File renamed without changes
4 changes: 3 additions & 1 deletion model_zoo/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,5 +42,7 @@
from .xDeepFM.src import xDeepFM
from .PEPNet.src import PPNet
from .TransAct.src import TransAct
from .multitask import SharedBottom, MMoE
from .multitask import ShareBottom, MMoE, PLE
from .EulerNet.src import EulerNet
from .WuKong.src import WuKong
from .GDCN.src import GDCN
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
## Shared-Bottom
## ShareBottom

| [Overview](#Overview) | [Configuration](#Configuration) | [Implementation](#Implementation) |


### Overview

Shared-Bottom is the classic multitasking model based on a shared bottom. The model is published in the following paper:
Share-Bottom is the classic multitasking model based on a Share bottom. The model is published in the following paper:

+ Rich Caruana. [Multitask Learning](https://link.springer.com/article/10.1023/A:1007379606734), in Machine learning 1997.

**Model structure:**

<div align="center">
<img width="50%" src="https://cdn.jsdelivr.net/gh/xue-pai/FuxiCTR@main/docs/img/SharedBottom.jpg">
<img width="50%" src="https://cdn.jsdelivr.net/gh/reczoo/FuxiCTR@main/docs/img/ShareBottom.jpg">
</div>

This figure comes from the paper of [MMoE (KDD 2018)](https://www.kdd.org/kdd2018/accepted-papers/view/modeling-task-relationships-in-multi-task-learning-with-multi-gate-mixture-).
Expand All @@ -33,7 +33,7 @@ The `model_config.yaml` file contains all the model hyper-parameters as follows.

| Params | Type | Default | Description |
| ---------------------- | --------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| model | str | "SharedBottom" | model name,  which should be same with model class name |
| model | str | "ShareBottom" | model name,  which should be same with model class name |
| dataset_id | str | "TBD" | dataset_id to be determined |
| loss | list | ["binary_crossentropy","binary_crossentropy"] | loss function for each task |
| metrics | list | ['logloss', 'AUC'] | a list of metrics for evaluation |
Expand Down Expand Up @@ -76,7 +76,7 @@ The `model_config.yaml` file contains all the model hyper-parameters as follows.
│ ├── dataset_config.yaml # 数据集配置文件
│ └── model_config.yaml # 模型配置文件
├── src # 模型代码文件夹
│ └── SharedBottom.py # 模型代码
│ └── ShareBottom.py # 模型代码
├── fuxictr_version.py # fuxictr加载及版本检查文件
├── README.md # 使用说明
└── run_expid.py # 执行脚本文件
Expand All @@ -95,6 +95,6 @@ The model is tested with the following dependencies.
Running the model on the tiny data:

```
python run_expid.py --expid SharedBottom_test --gpu 0
python run_expid.py --expid ShareBottom_test --gpu 0
```

Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ Base:
feature_specs: null
feature_config: null

SharedBottom_test:
ShareBottom_test:
model: SharedBottom
dataset_id: tiny_mtl
loss: ['binary_crossentropy','binary_crossentropy']
loss: ['binary_crossentropy', 'binary_crossentropy']
metrics: ['logloss', 'AUC']
task: ['binary_classification','binary_classification']
task: ['binary_classification', 'binary_classification']
num_tasks: 2
optimizer: adam
learning_rate: 1.e-3
bottom_hidden_units: [512,256,128]
bottom_hidden_units: [512, 256, 128]
tower_hidden_units: [128, 64]
hidden_activations: relu
net_regularizer: 0
Expand All @@ -30,8 +30,8 @@ SharedBottom_test:
net_dropout: 0
batch_size: 128
embedding_dim: 128
epochs: 50
epochs: 1
shuffle: True
seed: 2023
monitor: 'AUC'
monitor_mode: 'max'
monitor_mode: 'max'
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
'''
parser = argparse.ArgumentParser()
parser.add_argument('--config', type=str, default='./config/', help='The config directory.')
parser.add_argument('--expid', type=str, default='SharedBottom_test', help='The experiment id to run.')
parser.add_argument('--expid', type=str, default='ShareBottom_test', help='The experiment id to run.')
parser.add_argument('--gpu', type=int, default=-1, help='The gpu index, -1 for cpu')
args = vars(parser.parse_args())

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from fuxictr.pytorch.layers import FeatureEmbedding, MLP_Block


class SharedBottom(MultiTaskModel):
class ShareBottom(MultiTaskModel):
def __init__(self,
feature_map,
model_id="SharedBottom",
Expand All @@ -40,7 +40,7 @@ def __init__(self,
embedding_regularizer=None,
net_regularizer=None,
**kwargs):
super(SharedBottom, self).__init__(feature_map,
super(ShareBottom, self).__init__(feature_map,
task=task,
loss_weight=loss_weight,
num_tasks=num_tasks,
Expand Down
1 change: 1 addition & 0 deletions model_zoo/multitask/ShareBottom/src/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .ShareBottom import *
1 change: 0 additions & 1 deletion model_zoo/multitask/SharedBottom/src/__init__.py

This file was deleted.

3 changes: 2 additions & 1 deletion model_zoo/multitask/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
from .MMoE.src import MMoE
from .SharedBottom.src import SharedBottom
from .ShareBottom.src import ShareBottom
from .PLE.src import PLE

0 comments on commit 4892b6b

Please sign in to comment.