---
language: en
license: other
tags:
  - finance
  - risk-relation
  - retrieval
  - encoder
  - feature-extraction
  - stock-prediction
pipeline_tag: feature-extraction
---

# Financial Risk Identification through Dual-view Adaptation — Encoder

This repository hosts the pretrained encoder from the work **“Financial Risk Identification through Dual-view Adaptation.”**  
The model is designed to uncover **inter-firm risk relations** from financial text, supporting downstream tasks such as **retrieval**, **relation mining**, and **stock-signal experiments** where relation strength acts as a feature.

> **Files**  
> - `pytorch_model.safetensors` — model weights  
> - `config.json` — model configuration  
> - `README.md` (this file)

---

## ✨ What’s special (Dual-view Adaptation)

The model aligns two complementary “views” of firm relations and adapts them during training:

- **Lexical view (`lex`)** — focuses on token/phrase-level and domain terms common in 10-K and financial news.  
- **Temporal view (`time`)** — encourages stability/consistency of relations across reporting periods and evolving events.

A **two-view combination (“Best”)** integrates both signals and yields stronger retrieval quality and more stable risk-relation estimates. Ablations (`lex`, `time`) are also supported for analysis.

---

## 🔧 Intended Use

- **Feature extraction / sentence embeddings** for paragraphs, sections, or documents in financial filings.  
- **Retrieval & ranking**: compute similarities between queries (e.g., “supply chain risk for X”) and candidate passages.  
- **Risk-relation estimation**: aggregate cross-document similarities to produce pairwise firm relation scores used in downstream analytics.

> ⚠️ Not a generative LLM. Use it as an **encoder** (feature extractor).

---

## 🚀 Quickstart (Transformers)

```python
import torch
from transformers import AutoTokenizer, AutoModel

MODEL_ID = "william0816/Dual_View_Financial_Encoder"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True)
model = AutoModel.from_pretrained(MODEL_ID)

def mean_pool(last_hidden_state, attention_mask):
    # Mean-pool w.r.t. the attention mask
    mask = attention_mask.unsqueeze(-1).type_as(last_hidden_state)
    summed = (last_hidden_state * mask).sum(dim=1)
    counts = torch.clamp(mask.sum(dim=1), min=1e-9)
    return summed / counts

texts = [
    "The company faces supplier concentration risk due to a single-source vendor.",
    "Management reported foreign exchange exposure impacting Q4 margins."
]

enc = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
with torch.no_grad():
    outputs = model(**enc)
embeddings = mean_pool(outputs.last_hidden_state, enc["attention_mask"])

# Cosine similarity for retrieval
emb_norm = torch.nn.functional.normalize(embeddings, p=2, dim=1)
similarity = emb_norm @ emb_norm.T
print(similarity)
```

## 🖇️ Citation
If you use this model or the dual-view methodology, please cite:
```bibtex
@misc{financial_risk_dualview_2025,
  title        = {Financial Risk Identification through Dual-view Adaptation},
  author       = {Chiu, Wei-Ning and collaborators},
  year         = {2025},
  note         = {Preprint/Project},
  howpublished = {\url{https://huggingface.co/william0816/Dual_View_Financial_Encoder}}
}