2D and 3D image registration method and device

[0029] The embodiment of the present invention provides a 2D and 3D image registration method, the flow chart of which is as follows figure 1 shown, including the following steps:

[0030] S1. Acquire a 2D image and a 3D image to be registered.

[0031] S2. Input the 2D image and the 3D image into the pre-trained 2D image feature extraction network model and the 3D image feature extraction network model respectively, and obtain 2D image features and 3D image features containing spatial information, and the 2D image feature containing spatial information The image feature has the same spatial dimension as the 3D image feature containing spatial information.

[0032] figure 2 The network structure of the feature extraction model S21 in S2 is shown:

[0033] S211, 2D image feature extraction network model S211, indirectly extracting 1D features of 2D images containing spatial information from the 2D image, the 2D image feature extraction network model S211 specifically includes

[0086] S3131. Feature combination model, combining features from the 2D image to the 3D image for cross fusion with the 3D image to 2D image cross fusion feature combination as a feature , , combining features The sequence length of maintains the normalized sequence length S 0 Invariant, combined features number of channels , Represent the number of channels of the median value of the cross-attention network, and the specific value needs to be set according to the model; it can also be combined as S322 in the embodiment 2 described later;

[0087] S3132. Feature analysis network, using self-attention network to combine features parse, combine features The self-attention calculation formula of is as follows:

[0088] (18)

[0089] in, Represents a combination of features Analytical features obtained after self-attention network analysis; , and is the self-attention network parameter to be trained, d Represents a combination of features the numbe

[0111] The embodiment of the present invention provides a 2D and 3D image registration method, the flow chart of which is as follows figure 1 shown, including the following steps:

[0112] S1. Acquire a 2D image and a 3D image to be registered.

[0113] S2. Input the 2D image and the 3D image into the pre-trained 2D image feature extraction network model and the 3D image feature extraction network model respectively, and obtain 2D image features and 3D image features containing spatial information, and the 2D image feature containing spatial information The image feature has the same spatial dimension as the 3D image feature containing spatial information.

[0114] Figure 5 The network structure of the feature extraction model S22 in the step S2 is shown, including a 2D image feature extraction network model and a 3D image feature extraction network model, wherein:

[0115] S221. The 2D image feature extraction network model directly extracts the 1D features of the 2D image c