Paper Interpretation丨Structured Information Extraction in Zero-Shot Scene

Abstract: In the field of information structured extraction, predecessors generally needed to complete information structured extraction based on manually labeled templates. The paper proposes a zero-shot solution based on graph convolutional network, which can solve the problem that the training set and test set come from different vertical fields.

This article is shared from Huawei Cloud Community " Paper Interpretation Series Sixteen: Information Structured Extraction in Zero-Shot Scene 16108b1392c5af", author:

Summary

In the field of information structured extraction, predecessors generally need to complete information structured extraction based on manually labeled templates. The paper proposes a zero-shot solution based on graph convolutional network, which can solve the problem that the training set and test set come from different vertical fields.

Figure 1. The vertical fields of training and inference data sources are different.

Problem definition

Figure 2. Intuitive understanding of OpenIE and ClosedIE.

Relatin Extraction

• Close Relation Extraction (ClasedIE)
  RR stands for a collection of categories, including no categories, and the model can directly assign categories to each entity.
• Open Relation Extraction(OpenIE)
  RR represents a set of categories. The model classifies two categories to determine whether an entity is the key of another entity.

Zero-Shot Extraction

Zero-Shot can be distinguished by difficulty as follows:

• Unseen-Website Zero-shot Extraction
  That is, different layouts in the same vertical field, for example, are all web pages from movies. It's just that the web page layout used in the reasoning test is different from the training.
• Unseen-Websiste Zero-shot Extraction
  That is, different layouts in different vertical fields, for example, the training is from the webpage of the movie, and the webpage of the recruitment website may be used in the reasoning test.

The solution proposed in the paper is actually to discover all the key-value pairs in the graph network. Since the task of discovering the key-value itself is format-independent, it serves as a cross-domain format structure analysis.

concept

• relation: refers to key
• object: refers to value
• relationship: refers to key -> value

Encoder (feature construction)

The construction of node information is completed by the graph GG, which includes a series of nodes NN (entities) and the edges E (Edges) between nodes.

Build relationships between entities based on design rules

In the following cases, edges between nodes will be constructed (key-value pairs are often up and down or left and right relationships):

• Horizontal situation: horizontal neighbors, and there are no other nodes in the middle;
• Vertical situation: vertical neighbors, and no other nodes in the middle;
• Same level situation: nodes at the same level;

Use graph network to model the relationship between entities

Based on Graph Attention Network (GAT) to model the node relationship, the initial (input) characteristics of the node:

• Visual features: the visual description of the node in the webpage;
• Text features: OpenIE averages the features of pre-trained Bert, and CloseIE counts the frequency of occurrence of the node string (seems to be more cross-domain friendly);

Pre-training mechanism

The paper designs an auxiliary loss function L_{pre}Lpre​ to supervise three types of classification: {key, value, other}. At the same time, in order to prevent the training process from overfitting, the graph network weights in the OpenIE task will not be updated after the pre-training is completed.

Relationship prediction network

OpenIE

Determine whether a pair of nodes meets the requirement that the string content of the first node is the key of the string content of the second node:

• Use the candidate pair identification algorithm to obtain potential string pairs;
• The original input features of the two nodes + GNN output features + the relationship features of the two nodes are used as the input of the classifier;
• Fully connected network for classification;

ClosedIE

Cross-entropy multi-class classification

experiment

• Indeed, cross-cutting tasks are more difficult.
  
• CloseIE: Indeed, the more URLs, the better the effect.
  
• Confirm the influence of various factors on the effect of the network model.
  

Click to follow, and learn about Huawei Cloud's fresh technology for the first time~