Skip to content

I've got different metrics, I wonder the hyperparameters for metrics posted in paper. #2

@PolarisRisingWar

Description

@PolarisRisingWar

This is my result in test_metrics.json:
'macro_prec': 0.3009602427482605, 'macro_rec': 0.20516671240329742, 'macro_f1': 0.22117026150226593, 'micro_prec': 0.4229486087450312, 'micro_rec': 0.2341717429750442, 'micro_f1': 0.3014443629372929, 'jacc': 89.72397011976976, 'loss': 1.490009605884552

While the result in paper is:
image

And my hyperparameters are (some used the default hyperparameters, some used the ones posted on paper):

{
	"seed": 42,
	"do_train_dev": true,
	"do_test": true,
	"do_infer": false,
	"vocab_limit": null,
	"vocab_thresh": null,
	"weight_scheme": "tws",
	"tws_thresh": 10.0,
	"train_bs": 32,
	"dev_bs": 512,
	"test_bs": 512,
	"infer_bs": 512,
	"max_segments": 128,
	"max_segment_size": null,
	"num_mpath_samples": 8,
	"hidden_size": 200,
	"opt_lr": 1e-4,
	"opt_wt_decay": 1e-4,
	"sch_factor": 0.2,
	"sch_patience": 3,
	"num_epochs": 100,
	"pthresh": 0.65,
	"thetas": [1,2,3],
	"lambdas": [0.25,0.75],
	"dropout": 0.5
}

The results are different from the metrics posted on paper. So I wonder what are the hyperparamaters or other configurations used in your setting different from my configuration?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions