The invention discloses a course field-oriented image text aggregation method and
system, and the method comprises the steps of taking object features and an adjacent matrix as input to construct an object
relation graph corresponding to an image, and updating the features of nodes in the
relation graph through a graph
convolutional neural network; taking a set of feature vectors of all the language chunks as feature representation of the whole text description; taking the object
feature set and the step language block
feature set as input, and constructing a local
similarity matrix between the image-text pairs; calculating the global similarity between the image and the text description in the whole course field; and training parameters of the image-
text matching model through a
gradient descent method, obtaining a section of text description with the highest global similarity through the learned parameters, and taking the text description as a description text matched with the image, so as to realize image text aggregation. According to the invention, the features of image and text description in the course field can be effectively extracted, so that
schematic diagrams and text knowledge fragments in the course field are aggregated, and cross-
modal knowledge fragments are automatically constructed.