Many multi-thread downloads such as "Network Ants", Falshgets are a must-have tool for netizens. Use these tools to quickly download relatively large files from the server, the working characteristics of these tools are divided into several segments, each Download at the same time, simultaneous downloads. Write such programs, first, you must have a more fully understanding of the HTTP protocol; second, efficient use of multi-threaded programming means to implement on software.
Introduction to the HTTP protocol
The HTTP protocol is a hypertext transfer protocol, working on the network application layer, which is widely used in WWW global information services since 1990. The detailed description of the HTTP protocol can be reviewed online and other documents.
The old standard of the HTTP protocol is HTTP / 1.0, and the most common standard is HTTP / 1.1. HTTP / 1.1 is an upgrade based on HTTP / 1.0, adding some functions, compatible with HTTP / 1.0. HTTP / 1.0 does not support file breakpoints, if the server uses HTTP / 1.0, "Network Ants" can only download single-threaded download; good in the current Web server uses HTTP / 1.1, so, the following will be described based on HTTP / 1.1.
Important commands for HTTP protocol
Based on HTTP-based browser browsing web pages, when downloading files, working principle Similar client / server mode: The browser issues an HTTP request line to the web server; the web server returns a status row or multiple responses after receiving a valid request Title, a blank line and related documentation. Under this work principle, the download program must implement the function to send requests to the server and get the server response state.
1. Send a GET request command to the server
An HTTP request consists of a request line, an optional number of request titles, a blank line, and some additional data in the case of POST. The format of the request line is:
Request method URI HTTP / version number
The GET command is a document request method that is commonly used by the browser, uses in the middle of the program
GET URI HTTP / 1.1
Send request line to the web server (line number 3), the Java code is as follows:. . . . ClientSocket = New Socket (Host, Port); // Open Socket Outstream = New PrintStream (ClientSocket.getputstream ()) to download the file server. . . . OutStream.println ("get" URI "http / 1.1"); OutStream.println ("Host:" host); Outstream.println ("acce: * / *"); Outstream.println ("Referer:") Outstream.println (); . . . Note: The 4th line gives the host name and port number in the URL, and the 5th line shows that the client receives all MIME types. The seventh row sends a blank line, indicating the request of the request.
2. Get server response status
After sending an HTTP request row, the program can read the server's response state. The HTTP response status line includes: HTTP status code and some HTTP response headers.
1) HTTP status code
The HTTP status format is a digital representation of HTTP / version information. The status code example is as follows:
HTTP / 1.0 200 OK // Indicates that the server supports the HTTP / 1.0 protocol, and the HTTP / 1.1 200 ok // indicates that the server supports the HTTP / 1.1 protocol, success.
HTTP / 1.0 404 NOT FOUND / / The server supports the HTTP / 1.0 protocol, and the access file is not found.
In the middle of the program, if you read the "HTTP / 1.1 200 OK" string, you can use the server to save the breakpoint, you can use multithreaded downloads. If you read the string of "http / 1.0 200 ok", indicate that you want to download the file, but the server does not support breakpoints, you can only use single-threaded downloads. . . . . . While ((line = instream.readline ())! = null) // reads the server response status to line. . . . . . . . IF (Line.Substring (0,7). Equals ("HTTP / 1.")) // Judgment whether HTTP / 1.1 {IF (line.charat (7) == '0') {system.out.println ("Server USE HTTP / 1.0"); threadCount = 1;} if (! (line.substring (9, 12)). Equals ("200")) // Decision request is successful {system.out.println (" Error: " line); Return False;}} 2) Read important response titles, get the file length to download the document
If the HTTP status code indicates that the access is successful, the server will return some title line. We are most concerned about this line of Content-Length. For example, if the server returns "Content-Length: 1000", it indicates that the length of the request file is 1000 bytes, so Read this line of information, you can get the length information of the file:. . . . IF (line.substring (0,15) .Equals ("Content-length:")) {fileLength = long.parselong (line.substring (15) .trim ()); System.out.println ("File Length: " FileLength);}. . . . . . Send a break to the server
As mentioned above, if the server supports HTTP / 1.1, send GET requests to the server again:. . . . . Outstream.println ("get" URI "http / 1.1"); OutStream.println ("Host:" host); Outstream.println ("acce: * / *"); outstream.println ("Range: Bytes = " (fileblocklength) * ThisthreadID " - "); outstream.println () ;. . . . . The fourth line is the key. The server starts from the file xxxx byte, which is what we usually say. Split file, multi-thread download
Using multi-threaded programming technology, start multiple threads, depending on the number of threads, calculate the file segmentation position, send several different download breakpoints to the server, and accept data and write to files, you can implement multi-thread downloads. ..... raf = new randomaccessfile (file, "rw"); // Open file in a random access method ..... Synchronized (RAF) // Write each thread to obtain data separately to write file { Raf.seek (moviteength / threadcount) k * buflength); raf.write (readbytes); ......} ...
