There are several components in .NET designed to facilitate downloading from a remote resource such as the HttpClient or the WebClient classes. Unfortunately, these classes do not provide a system to track progress, save to a stream or resume partially completed downloads. I have created the HttpDownloadClient class to satisfy these requirements, including several additional features.
How it Works
The HttpDownloadClient uses a hybrid combination of thread spinning and blocking to allow the AsyncDownload method to wait for a previously called AsyncInitialize method to complete. The non-blocking synchronization is achieved through an atomic exchange using the Interlocked class in the AsyncInitialize method:
/// <summary>
/// Initializes the download.
/// </summary>
public async Task AsyncInitialize()
{
// CompareExchange is used to take control of _InitializingSyncPoint,
// and to determine whether the attempt was successful.
// CompareExchange attempts to put 1 into _InitializingSyncPoint, but
// only if the current value of _InitializingSyncPoint is zero
// (specified by the third parameter). If another thread
// has set _InitializingSyncPoint to 1 or -1, an exception is raised.
if (Interlocked.CompareExchange(ref _InitializingSyncPoint, 1, 0) == 0)
{
try
{
await AsyncInitializeDownload();
}
finally
{
//release control of _InitializingSyncPoint,
//allows waiting download to commence
_InitializingSyncPoint = -1;
}
}
else
throw new NotSupportedException(DOWNLOAD_INITIALIZING_EXCEPTION);
}
In the AsyncDownload method, a loop attempting another atomic exchange, blocks until the AsyncInitialize method has completed:
//this allows the Download method to wait for a previously called
//AsyncInitialize to complete
for (; ; )
{
int pt = Interlocked.CompareExchange(ref _InitializingSyncPoint, 1, 0);
//if original value was 0 (no initialization) or
//-1 (already initialized) break out of loop
if (pt == 0 || pt == -1)
break;
//if a CancellationToken was not provided, delay;
//otherwise, delay until the _CancellationToken is signaled or delay is elapsed
if (_CancellationToken == null)
await Task.Delay(1);
else
{
await Task.Delay(1, _CancellationToken);
_CancellationToken.ThrowIfCancellationRequested();
}
}
The download is ensured to be initialized if applicable:
//initialize download
if (!DownloadInitialized && (RequiresInitialization || AllowResuming))
{
await AsyncInitializeDownload();
//if DownloadFailed or Canceled is true, initialization failed
if (DownloadFailed || DownloadCanceled || DownloadCompleted)
return;
}
Download initialization is completed through the following process:
- Create a request to the remote resource.
- Read the web response to determine target file size and modification date.
- Determine if any data exists in buffer that can be resumed.
Initialization can be called before downloading will commence to gather the content length for concurrently downloads. All requests and delays take advantage of the await keyword for asynchronous operation:
for (; ; )
{
//increment download attempts
initializationAttempts++;
HttpWebRequest request = null;
try
{
//preemptive cancellation check
if (_CancellationToken != null)
_CancellationToken.ThrowIfCancellationRequested();
//create a request to the target file
request = CreateRequest(Target);
//get the response
using (HttpWebResponse response = await request.GetResponseAsync() as HttpWebResponse)
{
//if the Content-Length header is not set in the response
//set TotalBytes to null
if (response.ContentLength < 0)
//unknown data length
TotalBytes = null;
else
//known data length
TotalBytes = response.ContentLength;
//if resuming is allowed, determine if the file can be resumed
if (AllowResuming)
{
//download can be resumed if saving to a stream or if
//the remote file was not modified after the file was modified
if (DownloadMethod == DownloadMethod.ToStream)
BytesDownloaded = _TargetStream.Length; //how much we have already downloaded
else
{
FileInfo fi = new FileInfo(_TargetFile);
if (fi.Exists)
//how much we have already downloaded
BytesDownloaded = fi.Length;
else
BytesDownloaded = 0L;
}
//attempt to resume
if (BytesDownloaded > 0)
{
if (!TotalBytes.HasValue || TotalBytes < BytesDownloaded)
//Cannot retrieve remote size or wrong size so delete/clear what we have so far.
ClearBuffers();
else if (DownloadMethod == DownloadMethod.ToFile)
{
//determine if the time is wrong
DateTime lastModifiedRemote = (response as HttpWebResponse).LastModified.ToUniversalTime();
DateTime lastModifiedLocal = File.GetLastWriteTimeUtc(_TargetFile);
if (lastModifiedRemote > lastModifiedLocal) //compare the local and remote timestamps
{
//Remote file is newer then local.
ClearBuffers();
}
}
if (BytesDownloaded == TotalBytes)
{
//download has completed
DownloadCompleted = true;
//push the whole download length as a single packet received
_HttpDownloadPacketReceived(BytesDownloaded);
//raises HttpDownloadProgressChanged event if percent completed changed
_HttpDownloadProgressChanged();
//raise download complete event
_HttpDownloadComplete();
return;
}
}
}
else
ClearBuffers(); //no resuming, clear the buffers
}
//break out of infinite loop
break;
}
catch (OperationCanceledException)
{
//catch and throw OperationCanceledException to outer try-catch
throw;
}
catch
{
//if maximum initialization attempts are exceeded
if (initializationAttempts >= maxInitializationAttempts)
throw; //throw exception to outer try-catch
}
finally
{
//finally clause aborts request
if (request != null)
request.Abort();
}
//determine if suitable delay defined
if (RepeatDelay > 0)
{
//if a CancellationToken was not provided, delay;
//otherwise, delay until _CancellationToken is signaled or RepeatDelay is elapsed
if (_CancellationToken == null)
await Task.Delay(RepeatDelay);
else
{
await Task.Delay(RepeatDelay, _CancellationToken);
_CancellationToken.ThrowIfCancellationRequested();
}
}
}
If the download cannot be resumed due to invalid timestamps or data lengths, the file or stream buffers are cleared and the download will be started over.
File downloading is completed through the following process:
- Create a request to the remote resource, including the desired data range for resuming.
- Read the web response, determine if partial content is supported for resuming.
- Read the response stream and copy data to destination stream.
- If an error occurs, attempt to resume if supported and permitted.
All requests, reads, writes and delays take advantage of the await keyword for asynchronous operation:
//infinite loop allows repeated download attempts
for (; ; )
{
//increment DownloadAttempts
DownloadAttempts++;
HttpWebRequest request = null;
try
{
//preemptive cancel check
if (_CancellationToken != null)
_CancellationToken.ThrowIfCancellationRequested();
//create a HttpWebRequest from the Uri
request = CreateRequest(Target);
//if download can be resumed
if (AllowResuming && BytesDownloaded > 0)
//set the requested range
request.AddRange(BytesDownloaded);
else
//no resuming, clear the buffers
ClearBuffers();
using (HttpWebResponse response = await request.GetResponseAsync() as HttpWebResponse)
{
//look for partial content
if (AllowResuming && BytesDownloaded > 0 &&
response.StatusCode != HttpStatusCode.PartialContent)
//cannot resume, no partial content
ClearBuffers();
if (BytesDownloaded > 0)
{
//resuming, notify we got a packet (previously)
_HttpDownloadPacketReceived(BytesDownloaded);
//raises HttpDownloadProgressChanged event if percent completed changed
_HttpDownloadProgressChanged();
}
//get the response stream
using (Stream downloadStream = response.GetResponseStream())
{
//defines the stream to write to
Stream str = null;
//indicates the number of bytes read in the last packet
int readCount;
try
{
//if downloading to a stream, use the target stream;
//otherwise, use the file
if (DownloadMethod == DownloadMethod.ToStream)
{
//use target stream
str = _TargetStream;
//set the stream position to the end
str.Seek(0, SeekOrigin.End);
}
else
//open target file
str = File.Open(_TargetFile, FileMode.Append, FileAccess.Write, FileShare.None);
//buffer for data
byte[] buffer = new byte[0x1000];
//read from the download stream
while ((readCount = await downloadStream.ReadAsync(buffer, 0, buffer.Length, _CancellationToken)) > 0)
{
//write the data to the file
await str.WriteAsync(buffer, 0, readCount, _CancellationToken);
//increment the bytes count
BytesDownloaded += readCount;
//raises HttpDownloadPacketReceived event
_HttpDownloadPacketReceived(readCount);
//raises HttpDownloadProgressChanged event if percent completed changed
_HttpDownloadProgressChanged();
//cancel check
if (_CancellationToken != null)
_CancellationToken.ThrowIfCancellationRequested();
}
}
finally
{
//dispose the stream if target file
if (str != null && DownloadMethod == DownloadMethod.ToFile)
str.Dispose();
}
//download completed only if no more bytes are returned
if (readCount == 0)
//set DownloadCompleted flag indicating successful download
DownloadCompleted = true;
}
}
//break the infinite loop
break;
}
catch (OperationCanceledException)
{
//catch and throw OperationCanceledException to outer try-catch
throw;
}
catch (Exception)
{
//if maximum download attempts are exceeded, throw the exception
if (DownloadAttempts >= MaxDownloadAttempts)
//throw exception to outer try-catch
throw;
}
finally
{
//finally clause aborts request
if (request != null)
request.Abort();
}
//determine if suitable delay defined
if (RepeatDelay > 0)
{
//if a CancellationToken was not provided, delay;
//otherwise, delay until _CancellationToken is signaled or RepeatDelay is elapsed
if (_CancellationToken == null)
await Task.Delay(RepeatDelay);
else
{
await Task.Delay(RepeatDelay, _CancellationToken);
_CancellationToken.ThrowIfCancellationRequested();
}
}
}
AsyncInitialize and AsyncInitialize can be called synchronously by using the respected Initialize and Download methods:
/// <summary>
/// Initializes the download.
/// </summary>
public void Initialize()
{
Task t = AsyncInitialize();
//wait for task to complete
t.Wait();
//throw any exception
if (t.IsFaulted)
throw t.Exception;
}
/// <summary>
/// Begins the download.
/// </summary>
public void Download()
{
Task t = AsyncDownload();
//wait for task to complete
t.Wait();
//throw any exception
if (t.IsFaulted)
throw t.Exception;
}
Exceptions
If an error occurs while initializing or downloading, the HttpDownloadClient will attempt the operation again as permitted by the MaxDownloadAttempts property. If the maximum download attempts is exceeded, the HttpDownloadInitializationFailed or HttpDownloadException event will be raised with the last exception thrown.
Cancellation
The HttpDownloadClient can be initialized with a CancellationToken which will be monitored during initialization and downloading. If the CancellationToken is triggered during operation, the HttpDownloadInititializationCanceled or HttpDownloadCanceled event is raised.
Using
I've created an example form which asynchronously downloads a file. A progress bar is updated and the download can be stopped and resumed:
using System;
using System.IO;
using System.Net;
using System.Threading;
using System.Windows.Forms;
public class DownloadTestForm : Form
{
private Button StopBtn;
private ProgressBar DownloadProgress;
private Button DownloadBtn;
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.IContainer components = null;
/// <summary>
/// Clean up any resources being used.
/// </summary>
/// <param name="disposing">true if managed resources should be disposed; otherwise, false.</param>
protected override void Dispose(bool disposing)
{
if (disposing)
{
if (_CancellationTokenSource != null)
_CancellationTokenSource.Cancel();
if (components != null)
components.Dispose();
}
base.Dispose(disposing);
}
#region Windows Form Designer generated code
/// <summary>
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
/// </summary>
private void InitializeComponent()
{
this.StopBtn = new System.Windows.Forms.Button();
this.DownloadProgress = new System.Windows.Forms.ProgressBar();
this.DownloadBtn = new System.Windows.Forms.Button();
this.SuspendLayout();
//
// StopBtn
//
this.StopBtn.Enabled = false;
this.StopBtn.Location = new System.Drawing.Point(316, 12);
this.StopBtn.Name = "StopBtn";
this.StopBtn.Size = new System.Drawing.Size(75, 23);
this.StopBtn.TabIndex = 5;
this.StopBtn.Text = "Stop";
this.StopBtn.UseVisualStyleBackColor = true;
this.StopBtn.Click += new System.EventHandler(this.StopBtn_Click);
//
// DownloadProgress
//
this.DownloadProgress.Location = new System.Drawing.Point(12, 12);
this.DownloadProgress.Name = "DownloadProgress";
this.DownloadProgress.Size = new System.Drawing.Size(217, 23);
this.DownloadProgress.TabIndex = 4;
//
// DownloadBtn
//
this.DownloadBtn.Location = new System.Drawing.Point(235, 12);
this.DownloadBtn.Name = "DownloadBtn";
this.DownloadBtn.Size = new System.Drawing.Size(75, 23);
this.DownloadBtn.TabIndex = 3;
this.DownloadBtn.Text = "Start";
this.DownloadBtn.UseVisualStyleBackColor = true;
this.DownloadBtn.Click += new System.EventHandler(this.DownloadBtn_Click);
//
// DownloadTestForm
//
this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
this.ClientSize = new System.Drawing.Size(406, 48);
this.Controls.Add(this.StopBtn);
this.Controls.Add(this.DownloadProgress);
this.Controls.Add(this.DownloadBtn);
this.Name = "DownloadTestForm";
this.Text = "DownloadTestForm";
this.ResumeLayout(false);
}
#endregion
public DownloadTestForm()
{
InitializeComponent();
}
/// <summary>
/// Used to signal cancellation.
/// </summary>
private CancellationTokenSource _CancellationTokenSource;
/// <summary>
/// Stream to copy remote data to.
/// </summary>
private MemoryStream _TestStream = new MemoryStream();
private async void DownloadBtn_Click(object sender, EventArgs e)
{
//disable download button
DownloadBtn.Enabled = false;
//create cancellation token
using (_CancellationTokenSource = new CancellationTokenSource())
{
//create download client from a target file and destination stream
HttpDownloadClient _Client = new HttpDownloadClient(new Uri("[target file]"), _TestStream, true, _CancellationTokenSource.Token);
//subscribe to the HttpDownloadProgressChanged event to
//monitor progress. since this event is called on a separate thread,
//invoke to UI thread to update progress bar
_Client.HttpDownloadProgressChanged += (obj, arg) =>
{
this.Invoke(() =>
{
//PercentComplete can be null if the remote resource
//content-length is unknown
int? progress = arg.PercentComplete;
if (progress.HasValue)
{
//ensure block style
DownloadProgress.Style = ProgressBarStyle.Blocks;
//set percent complete
DownloadProgress.Value = arg.PercentComplete.Value;
}
else
//if no content-length, switch to a marquee style
DownloadProgress.Style = ProgressBarStyle.Marquee;
});
};
//clears buffer for testing
_Client.HttpDownloadComplete += (obj, arg) =>
{
_TestStream.SetLength(0L);
};
//enable stop button to stop download
StopBtn.Enabled = true;
//download file asynchronously
await _Client.AsyncDownload();
}
//prevents dispose method from attempting cancellation
_CancellationTokenSource = null;
//enable download button to restart or resume download
DownloadBtn.Enabled = true;
}
private void StopBtn_Click(object sender, EventArgs e)
{
//cancel download
_CancellationTokenSource.Cancel();
//disable stop button
StopBtn.Enabled = false;
}
}
The previous example uses an extension method for the invoke call, see Using Lambda Expressions in Control Invoke Calls. Downloading multiple files using a single progress bar can be achieved by initializing each download and subscribing to the HttpDownloadPacketReceived method to increment progress.
