Website data extraction method based on mobile phone, terminal equipment and storage medium

A data extraction and mobile phone technology, applied in the field of data extraction, can solve problems such as account closure, difficulty in collecting data from social media sites, and limited agency

Inactive Publication Date: 2021-02-19
XIAMEN MEIYA PICO INFORMATION
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] At present, there is no tool for collecting social media website data in the existing technology, but using traditional crawlers to collect social media we

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Website data extraction method based on mobile phone, terminal equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0034] The embodiment of the present invention provides a mobile phone-based website data extraction method, such as figure 1 As shown, it is a flow chart of the mobile phone-based website data extraction method described in the embodiment of the present invention, and the method includes the following steps:

[0035] S1: collect the homepage url of the social media website, construct a task url according to the homepage url, and add the task url to the task table in the database of the PC.

[0036] The method of collecting the homepage address of the social media website is as follows: open the homepage of the social media website on a PC, and check the source code of the homepage; if the source code contains "userID": "xx", determine that the type of the homepage is a personal homepage, and extract the source code userID in the source code; if the source code contains "pageID": "xx", it is determined that the type of the homepage is a public homepage, and the pageID in the sour

Embodiment 2

[0061] The present invention also provides a mobile phone-based website data extraction terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the computer program The steps in the above method embodiment of Embodiment 1 of the present invention are implemented.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a website data extraction method based on a mobile phone, terminal equipment and a storage medium, and the method comprises the following steps: S1, collecting a homepage urlof a social media website, constructing a task url according to the homepage url, and adding the task url into a task table in a database of a PC; s2, deploying a data transmission interface for information interaction between the mobile phone and the PC in the PC; s3, storing an effective account corresponding to the social media website in an account list of a database of the PC, and storing aneffective agent IP in an agent pool; s4, after the mobile phone is connected with the PC, enabling the mobile phone to download the webpage source code of the social media website by calling a data transmission interface; s5, analyzing the downloaded webpage source code through an analysis plug-in to obtain webpage content data; and S6, packaging and storing the acquired webpage content data through different standards according to different types of the acquired webpage content data. The problems that a traditional crawler is difficult to collect social media websites, agents are prone to being sealed, accounts are prone to being sealed and the like are solved.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Owner XIAMEN MEIYA PICO INFORMATION
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap