Write an asynchronous multi-thread download program with VB
In order to efficiently carry a web page for a site, we can use the VB's Internet Transfer control to write your own download programs, the Internet Transfer Control supports Hypertext Transfer Protocol (HTTP) and File Transfer Protocol (FTP), using the Internet Transfer control through OpenURL Or the execute method is connected to any site that uses the two protocols and retrieves files. This program uses multiple Internet Transfer controls to download a site at the same time. And can determine if the file has been downloaded or downloaded by the current file is old on the server, to decide whether to re-download. The links in all downloaded files have been adjusted to see them locally. The OpenURL method transmits data in synchronously. The synchronization refers to the other process cannot be performed until the transmission operation is not completed. Such data transmission must be done before performing other codes. The Execute method transmits data asynchronously. When the Execute method is called, the transfer operation is independent of other processes. Thus, after the Execute method is called, other code can be performed while receiving data in the background. With the OpenURL method, you can directly get the data stream that can be saved to the disk, or reading directly in the TextBox control (if the data is text format). With the Execute method to obtain data, you must monitor the connection status of the control with the STATECHANGED event. When the appropriate state is reached, the GetChunk method is called to obtain data from the buffer of the control. First, establish an initial HTTP retrieval connection.
Public G AS Variant Public K AS Variant Public Spath AS String Dim Links () AS String G = 0 Spath = Local Save Download Path Links (0) = Sales URL INET1.EXECUTE LINKS (0), "Get" use Get method.
Event monitor subroutine (Event monitor subroutine for each Internet Transfer control setting):
Use the StateChanged event to monitor the connection status of the control, when the request has been completed, and all the data has been received, call the GetChunk method to obtain data from the controls buffer.
Private Sub inet1_statechanged (Byval State As Integer) When you use the getchunk method to retrieve the server's response. SELECT CASE State '... did not list other situations. Case icrSponseCompleted '12 'Gets protocols, hosts, and path names in Links (g). Addsuf = Left (Links (g), INSTRREV (Links (g), "/")) 'Gets file names in Links (g). FNAME = Right (Links (g)) - INSTRRRRREV (Links (g), "/")) 'Judging whether it is a hypertext file, it is a hypertext file to analyze the link, if not Save binary files. If INSTR (1, FNAME, "HTM", vbtextCompare) = true kiln initializes the FileSystemObject object for saving files. SET FS = CreateObject ("scripting.filesystemObject") DIM VTDATA AS VARIANT 'data variable. DIM STRDATA AS STRING: STRDATA = "" DIM BDONE AS BOOLEAN: BDONE = false 'acquisition first. VTData = inet1.getchunk (1024, ICString) Doevents do while not bdone strdata = strdata & vtdata doevents' gets the next piece. VTDATA = INET1.GETCHUNK (1024, ICSTRING) If LEN (VTDATA) = 0 THEN BDONE = true end if loop 'Get the link in the document and placed in an array. Dim i As Variant Dim po1 As Variant Dim po2 As Variant Dim oril As String Dim newl As String Dim lmtime, ctime po1 = InStr (1, strData, "href =", vbTextCompare) 5 po2 = 1 Dim newstr As String: newstr = "" Dim whostr as string: whostr = "" I = 0 do while po1> 0 newstr = MID (strdata, po2, po1) whostr = Whostr Newstr PO2 = INSTR (PO1, STRDATA, ">", vbtextCompare) ' Change the original link to new link oril = MID (strdata, po1 1, po2 - po1 - 1) 'If there is quotation mark, remove quotation marks ln = replace (oril, "" ",", ", vbtexTCompare) newl = Right ( Ln, Ln (LN) - Instrrev (ln, "/")) whostr = whostr & newl if ln <> "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "The 'decision file is downloaded.
If FileExists (Spath & newl) = false the links (i) = addsuf & ln i = i 1 else lmtime = inet1.getHeader ("Last-modified") set f = fs.getfile (spath & newl) ctime = f .datecreated 'judging whether the file updates if Datediff ("s", lmtime, ctime) <0 THEN i = i 1 end if end if end if po1 = INSTR (PO2 1, StrData, "HREF =", vbtextCompare) 5 loop newstr = mid (strdata, po2) whostr = whostr newstr set a = fs.createtextfile (spath & fname, true) a.write whostr a.close k = I else Dim vtdata as variant Dim b () AS BYTE DIM BDONE As Boolean: BDONE = false vtdata = inet2.getchunk (1024, icbyteaRray) do while not bdone b () = b () & vtdata vtdata = inet2.getchunk (1024, icbyteaRray) if len (vTData) = 0 Then BDONE = True Endiff Open Spath & Fname for Binary Access Write AS # 1 Put # 1,, B () Close # 1 End If Call DevJob 'calling thread schedule End Select End Sub Private Sub inet2_statechanged (ByVal State As Integer). .. End sub ... thread dispatch subroutine, G and are K common variables, k is the array index of the last link plus one, g initially zero, each additional one, until the last link is processed.
Private sub devjob () IF not g 1
