Efficient keyword filtering method for FPGA (Field Programmable Gate Array)

[0037] This embodiment provides a high-efficiency keyword filtering method for FPGA implementation, which adopts the hierarchical processing mechanism of the host and the FPGA accelerator card, and can be used for keyword filtering of offline application content data, see figure 1 , the method includes:

[0038] S1: keyword preprocessing;

[0039] Sort by keyword word count. If the word count of a certain keyword exceeds 31, then divide the super-long keyword into multiple keywords (the keyword length cannot be cut too short, and the distribution of the entire keyword (2-31) should be considered as a whole). Keywords with the same number of words constitute a keyword set (kw_i), where i represents the number of words in the keyword, and kw represents keywords with the same length.

[0040] Calculate the hash value of each keyword in the same set, and build a hash-key value table (hash_i_array) and a hash keyword comparison table.

[0041] S2: Load the keyword hash key value ta

[0050] This embodiment provides a specific application of an FPGA-oriented high-efficiency keyword filtering method, which can be used for online real-time search and filtering of network message data, such as figure 2 As shown, the method includes:

[0051] Step1: Keyword preprocessing, and calculate the hash value of each keyword, and get the hash value-keyword comparison table;

[0052] In keyword preprocessing, keywords are classified according to length, and keywords with the same number of words constitute a keyword set, and each length uses a hash function. This method supports the hash of 2 to 31 Chinese characters, and a total of 30 are obtained. The set of hash values, that is, hash2~hash31, the value range of each hash function is 0~511, recorded as hash_array[0:511], the hash function value of the keyword is calculated in turn, and the corresponding bit is set to 1.

[0053] For example, assuming that the existing keyword set is: {"Secret", "Sensitive", "Military",