Write an asynchronous multi-thread download program with VB
Daqing Oilfield Co., Ltd. Exploration and Development Research Institute Network Room Manchu
In order to efficiently carry a web page for a site, we can use the VB's Internet Transfer control to write your own download programs, the Internet Transfer Control supports Hypertext Transfer Protocol (HTTP) and File Transfer Protocol (FTP), using the Internet Transfer control through OpenURL Or the execute method is connected to any site that uses the two protocols and retrieves files. This program uses multiple Internet Transfer controls to download a site at the same time. And can determine if the file has been downloaded or downloaded by the current file is old on the server, to decide whether to re-download. The links in all downloaded files have been adjusted to see them locally.
The OpenURL method transmits data in synchronously. The synchronization refers to the other process cannot be performed until the transmission operation is not completed. Such data transmission must be done before performing other codes.
The Execute method transmits data asynchronously. When the Execute method is called, the transfer operation is independent of other processes. Thus, after the Execute method is called, other code can be performed while receiving data in the background.
With the OpenURL method, you can directly get the data stream that can be saved to the disk, or reading directly in the TextBox control (if the data is text format). With the Execute method to obtain data, you must monitor the connection status of the control with the STATECHANGED event. When the appropriate state is reached, the GetChunk method is called to obtain data from the buffer of the control.
First, establish an initial HTTP retrieval connection.
Public g as variant
Public K As Variant
Public Spath As String
DIM LINKS () AS STRING
g = 0
Spath = Locally saved the path to download files
LINKS (0) = started URL
inet1.execute links (0), "get" 'uses the GET method.
Event monitor subroutine (Event monitor subroutine for each Internet Transfer control setting):
Use the StateChanged event to monitor the connection status of the control, when the request has been completed, and all the data has been received, call the GetChunk method to obtain data from the controls buffer.
Private sub inet1_statechanged (Byval State As Integer)
When 'State = 12, use the getChunk method to retrieve the server's response.
Select Case State
'... did not list other situations.
Case icrSponseCompleted '12
'Get protocols in Links (G), hosts, and pathnames.
Addsuf = Left (Links (g), Instrrev (Links (g), "/"))
'Get the file name in Links (G).
FNAME = Right (LINKS (G), Len (Links (g)) - Instrrev (Links (G), "/"))
'Determine whether it is a hypertext file, is a hypertext file, analyzes the link, if not, the binary file is stored.
IF INSTR (1, FNAME, "HTM", VbtextCompare) = true kil
'Initialize the FileSystemObject object for saving the file.
Set fs = creteObject ("scripting.filesystemobject")
DIM VTDATA AS VARIANT 'data variable.
DIM STRDATA AS STRING: STRDATA = "" DIM BDONE AS BOOLEAN: BDONE = FALSE
'Get the first piece.
VTData = inet1.getchunk (1024, ICString)
Doevents
Do While Not Bdone
Strdata = strdata & vtdata
Doevents
'Get the next piece.
VTData = inet1.getchunk (1024, ICString)
IF LEN (VTDATA) = 0 THEN
BDONE = TRUE
END IF
Loop
'Get the link in the document and placed in an array.
DIM I as Variant
Dim po1 as variant
DIM PO2 As Variant
Dim Oril As String
Dim newl as string
DIM LMTIME, CTIME
PO1 = INSTR (1, strdata, "href =", vbtextcompare) 5
PO2 = 1
Dim Newstr AS String: newstr = ""
Dim whostr as string: whostr = ""
i = 0
Do While Po1> 0
Newstr = MID (strData, PO2, PO1)
Whostr = whostr newstr
PO2 = INSTR (PO1, STRDATA, ">", vbtextcompare)
'Change the original link to a new link
Oril = MID (STRDATA, PO1 1, PO2 - PO1 - 1)
'If there is quotation mark, remove the quotation mark
Ln = Replace (Oril, "" "," ", vbtextcompare)
NEWL = Right (Ln, Ln) - INSTRREV (Ln, "/"))
Whostr = whostr & newl
IF ln <> "".
'Decide whether the file is downloaded.
If FileExists (Spath & Newl) = false Then
LINKS (I) = addsuf & ln
i = i 1
Else
Lmtime = inet1.getHeader ("Last-Modified")
Set f = fs.getfile (Spath & newl)
CTIME = f.datecreated
'Determine if the file is updated
IF Datediff ("S", LMTIME, CTIME) <0 THEN
i = i 1
END IF
END IF
END IF
PO1 = INSTR (PO2 1, StrData, "HREF =", vbtextcompare) 5
Loop
Newstr = MID (strdata, po2)
Whostr = whostr newstr
Set a = fs.createtextfile (Spath & Fname, True)
a.write whostr
a.close
K = I
Else
DIM VTDATA AS VARIANT
DIM B () AS BYTEDIM BDONE As Boolean: BDONE = FALSE
VTData = inet2.getchunk (1024, ICBYTEARRAY)
Do While Not Bdone
b () = b () & vtdata
VTData = inet2.getchunk (1024, ICBYTEARRAY)
IF LEN (VTDATA) = 0 THEN
BDONE = TRUE
END IF
Loop
Open Spath & Fname for Binary Access Write AS # 1
Put # 1,, b ()
Close # 1
END IF
Call devjob 'calling thread schedule
End SELECT
End Sub
Private sub inet2_statechanged (Byval State As Integer)
...
End Sub
...
Thread schedulers, g is a K common variable, k is a group index of the last link plus one, and the G initial value is zero, and each time I add one, until the last link is processed.
Private sub devjob ()
IF not G 1 IF inet1.stillexecuting = false kil g = g 1 INET1.EXECUTE LINKS (G), "Get" END IF IF not G 1 IF inet2.stillexecuting = false kil g = g 1 Inet2.execute links (g), "get" END IF ... Reportline: IF inet1.stillexecuting = false and inet2.stillexecuting = false and ... then Msgbox ("Download is over.") END IF End Sub
