phuongntc/Multi_EvalSumViet2

Vietnamese Summary Evaluator (videberta-base + trunk + 3 heads)

What is this? A multi-criteria evaluator that predicts three scores in [0,1]: faithfulness, coherence, relevance for a (document, summary) pair.

Architecture. Vietnamese encoder backbone → mean pooling → shared MLP trunk → three linear heads.

Training (short).

• Multi-task regression (MSE) + in-document pairwise hinge ranking (doc_id grouping).
• Tokenization: pair (doc, summary), truncation='only_first', pre-trim summary to 256, max_len=512.
• See training_args.json, loss_config.json, and arch_config.json for details.

Files

• config.json, model.safetensors (+ tokenizer files): backbone encoder
• trunk.pt, head_faith.pt, head_coh.pt, head_rel.pt: lightweight heads
• arch_config.json, training_args.json, loss_config.json, package_versions.json
• modeling_summary_evaluator.py: loader & pair-encoding helpers

Quickstart

from huggingface_hub import snapshot_download
import importlib.util, os
repo = snapshot_download("summary-evaluator-export", repo_type="model")
spec = importlib.util.spec_from_file_location("mse", os.path.join(repo, "modeling_summary_evaluator.py"))
mse = importlib.util.module_from_spec(spec); spec.loader.exec_module(mse)
model, tok, device = mse.load_for_inference(repo)
enc = mse.encode_pair(tok, ["Văn bản gốc..."], ["Bản tóm tắt..."])
import torch
with torch.inference_mode():
    y = model(torch.tensor(enc["input_ids"]), torch.tensor(enc["attention_mask"]))
print(y)  # [1, 3] in [0,1]