Course field-oriented image text aggregation method and system

一种文本、领域的技术，应用在计算机领域，能够解决难课程领域图像和文本描述自动化匹配等问题，达到促进全局相似度的提升、提高准确率的效果

Active Publication Date: 2021-08-06
XI AN JIAOTONG UNIV
View PDF3 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the huge difference between course domain images and natural images, traditional natural image text aggregation methods cannot deeply mine the information contained in course domain images, so it is difficult to effectively perform automatic matching between course domain images and text descriptions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] Next, the technical solutions in the embodiments of the present invention will be described in contemplation in the embodiment of the present invention, and it is clear that the described embodiments are intended to be in an embodiment of the invention, not all of the embodiments of the invention. Based on the embodiments of the present invention, there are all other embodiments obtained without making creative labor without making creative labor premises.

[0060] It should be understood that when used in this specification and the appended claims, the terms "include" and "including" indicating the description of the features, overall, steps, operations, elements, and / or components, but does not exclude one or Multiple other features, overall, steps, operations, elements, components, and / or their set presence or addition.

[0061] It should also be understood that the terms used in the specification of the present invention are merely the purposes of describing the par...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a course field-oriented image text aggregation method and system, and the method comprises the steps of taking object features and an adjacent matrix as input to construct an object relation graph corresponding to an image, and updating the features of nodes in the relation graph through a graph convolutional neural network; taking a set of feature vectors of all the language chunks as feature representation of the whole text description; taking the object feature set and the step language block feature set as input, and constructing a local similarity matrix between the image-text pairs; calculating the global similarity between the image and the text description in the whole course field; and training parameters of the image-text matching model through a gradient descent method, obtaining a section of text description with the highest global similarity through the learned parameters, and taking the text description as a description text matched with the image, so as to realize image text aggregation. According to the invention, the features of image and text description in the course field can be effectively extracted, so that schematic diagrams and text knowledge fragments in the course field are aggregated, and cross-modal knowledge fragments are automatically constructed.

Description

Technical field [0001] The present invention belongs to the technical field of computer, and more particularly to an image text aggregation method and system for the course. Background technique [0002] In recent years, with the rapid development of Internet technology, various knowledge-related website platforms continue to emerge, users who have acquired knowledge through the Internet have increased rapidly, thus promoting various types of knowledge resource explosive growth in the network. A variety of courses and text knowledge containing knowledge concepts are created by different platform organizations, which are distributed in various websites. The complement between these multi-modal knowledge helps achieve efficient expression of complex things, knowledge, and concepts. [0003] At the same time, the multi-mode knowledge is in line with the multi-channel information processing mode of the human brain, which can effectively improve the learner's cognitive efficiency. Sin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/32G06K9/46G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V10/25G06V20/62G06V10/44G06V30/10G06N3/048G06N3/045G06F18/2132
Inventor 张玲玲吴文俊杨宽刘均郑庆华魏笔凡胡欣王绍伟王佳欣
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap