C/C++教程

The Art of Prompting: Event Detection based on Type Specific Prompts

本文主要是介绍The Art of Prompting: Event Detection based on Type Specific Prompts,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!

Motivation

  • 之前的研究表明prompt可以提高模型在事件检测方面的性能,包括
    • 使用特定structure
    • 使用每种事件类型特定的query
    • 原型 trigger
  • 这些尝试启发对不同prompt效果的探究

Settings

作者在3种setting下做了实验:

  • Supervised event detection
  • Few-shot Event detection
    • 两个数据集\(D_{base}, D_{novel}\),前者规模大,后者小。后者用来作为第二个域。
  • Zero-shot Event detection
    • 只用第二个域的数据作为测试,不在\(D_{novel}\)上面fine-tune

Event Type Prompts

  • Event Type Name
    • 使用Event的名字作为prompt
  • Definition
    • 如 Attack:violent or physical act causing harm or damage
  • Prototype Seed Triggers
    • 在判断事件类别\(T\)时,选取\(K\)个trigger word,他们作为\(T\)的触发词的次数占他们出现总次数的比例最大,并连接起来。
  • Event Type Structure
    • 加入事件的参数
  • Continuous Soft Prompt
    • 参考经典论文 Prefifix-tuning: Optimizing continuous prompts for generation
  • APEX Prompt
    • 综合prompt,包含:event type name, seed triggers, and definition
    • 如下图所示

A Unified Framework for Event Detection

  • Context Encoding

    • Sentence \(W={w_1,w_2,\dots,w_N}\)

    • event type prompt \(T={\tau _1^t, \tau _2^t,\dots,\tau _K^t}\)

    • 拼接成一个序列

    • \[[CLS] \tau _1^t, \tau _2^t,\dots,\tau _K^t [SEP] w_1,\dots,w_N[SEP] \]

    • 序列通过BERT得到对应的向量\(\mathbf{W},\mathbf{T}\)

  • Event Type Aware Contextual Representation

    • 计算每个token的event type \(t\) aware contextual representation \(A_i^t=\sum_{j=1}^{|T|} \alpha_{ij} \cdot \mathbf{T}_j, \text{where } \alpha_{ij}=\cos(\mathbf{w}_i, \mathbf{T_j})\)
  • Event Detection

    • \(\tilde{y}_i^t = U_o ([\mathbf{w}_i;\mathbf{A}_i^T;\mathbf{P}_i])\)
    • \(\mathbf{P}_i\)是token i的pos编码向量
    • \(U_o\)是预测事件检测的分类头

Experiments

Method Supervised ED Few-shot ED Zero-shot ED
State of the art 73.3 35.2 49.1
(a) Event Type name 72.2 52.7 49.8
(b) Defifinition 73.1 46.7 45.5
(c) Seed Triggers 73.7 53.8 52.4
(d) Event Type Structure 72.8 50.4 48.0
(e) Continuous Soft Prompt 68.1 48.2 -
Majority Voting of (a)-(e) 73.9 52.1 48.7
(f) APEX Prompt 74.9 57.4 55.3
这篇关于The Art of Prompting: Event Detection based on Type Specific Prompts的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!