Upload Speed using Trial Account

Last post 10-03-2007 11:47 AM by itsthetaste. 3 replies.
Page 1 of 1 (4 items)
Sort Posts: Previous Next
  • 10-03-2007 12:04 AM

    Upload Speed using Trial Account

    Im testing with trial account and am noticing that the average upload speed I can obtain is 30KB per second.

    I am using the C# SDK, so am using this bit of code.

    // Open the file for reading

    using (FileStream fs = new FileStream(localPath, FileMode.Open, FileAccess.Read))

    {

    using (BinaryReader br = new BinaryReader(fs))

    {

    byte[] fileData;

    do

    {

    // Read a chunk of data from the file.

    fileData = br.ReadBytes(fileChunkSize);

    // Upload some data to the file

    transfer.AppendFile(uploadNode.AccessToken, relativePath, fileData, false);

    // Set the status

    totalBytesUploaded += fileData.Length;

    }
    while (fileData.Length == fileChunkSize);

    // This is the end of file, so set the endofFile flag to 'true'

    transfer.AppendFile(uploadNode.AccessToken, relativePath, null, true);

    }

    }

    I have an ADSL2+ broadband connection which runs at 1.3Mb or ~160KB sec and am running on a 2.66GHz dual core intel cpu which whilst uploading is not getting above 1% usage.

    I have tested my connection and it is able to upload at about 1Mb at the moment. 

     So is there a restriction on the trial account as 30KB/sec is slow.

     

    Cheers

    Richard

     

  • 10-03-2007 5:34 AM In reply to

    • BarryR
    • Top 10 Contributor
    • Joined on 07-20-2007
    • San Diego
    • Posts 614

    Re: Upload Speed using Trial Account

    There are no restrictions on speed for trial accounts.

    Since you are on ADSL2+ I'm going to assume its something in the range of 20Mb down and 1.3Mb up and perhaps in the UK (Most likely BT?)

    The use of the SOAP interface incurs quite a bit of overhead for a few reasons, first its doing base64 encoding which will add roughly 40% to your overall data size. Second each call you make to the AppendFile has its own SOAP envelope which will add some size as well.  So we need to look at the raw data rate not just the total amount of the file being sent.  If you are in windows you can do this by looking at the Task Manager and selecting the Networking Tab and adding a column for Bytes transfered / sec.  This will give you a rough idea outside of the the application of your overall speed (Be sure to shut down any browsers or streaming apps going on in the background.)  Secondary you could use Perfmon to see the traffic.

    So, assuming you have the new value which i would suspect bounces around 50-40Kb/sec you should look at the size of each byte chunk sent.  In this case your fileChunkSize should be as large as you can make it without affecting your memory adversely, i use 64k or so.  Each time you do the AppendFile it has to reconnect which is going to take some time due to connection negotiation at the TCP/IP level.

    I would expect once you have this configured at the most optimal file chunk size you will find your near your maximum when viewing it through something that measures overall bandwidth.

    So, assuming you have kept up to this point your probably wondering how you can avoid that heavy overhead of SOAP yet maintain a simple way to upload.  We have HTTP upload available, this allows you to avoid the encoding and soap envelope.  I will put a sample together tomorrow from C# showing how to do this from code.  To do a test that is outside of your application and relies on the web browser you can use a simple test form we have available at: http://node1.nirvanix.com/upload.htm To use this form you will need to generate an upload token from your IP Address.  A quick note you can try using the WebClient.UploadFile() in the System.web library, but I have found it to have a few significant bugs (Files cannot contain spaces...)

    Let me know the results of your tests, we have been working to increase the capacity significantly to our bandwidth and have done some pings from London that show things should be working well.

    Regards,
        Barry R.

    IM Support (Feel free to add me)

    MSN: barryruffner@msn.com
    Gmail: barryruffner@gmail.com
  • 10-03-2007 11:15 AM In reply to

    Re: Upload Speed using Trial Account

    Thanks for the reply Barry.

    I am in the UK on ADSL2+ but not with BT as they dont offer it yet.  Im with Bethere.

    I was using the task manager with and had added the bytes transfered / sec.  My rate was fluctuating between 36K and 24K /sec so i took the average of 30K.

    I've written a little windows service to take the uploaded videos, encode them and send them over to your selves using SOAP.  That is working now just running a bit slow.  I'll have a look at the WebClient.UploadFIle() method to see if that could speed things up.  But first i'll try increasing the fileChunkSize as in the SDK it is set to 8K.

    I now that the form upload works well as when I use your sample app podvault i get an uplooad speed of about 120K.

    I like using SOAP and the SDK as it means I dont have to write a lot of code. :) so i'll add a config setting to the fileChunkSize property and test different values.

    Thanks again.

    Richard

  • 10-03-2007 11:47 AM In reply to

    Re: Upload Speed using Trial Account

    Ive changed the SDK and added a new config setting entry;

    <setting name="fileChunkSize" serializeAs="String">

    <value>131072</value>

    </setting>

    and edited the PerformUpload method of the FileUpload class so that it pulls is value from there

    int fileChunkSize = NirvanixSDK.Properties.Settings.Default.fileChunkSize;

    When I rebuild and test using the UploadSnippet it now uses almost my full upload speed.  :)  So when I put this on my server i'll have to play around with this setting again, but thanks, all is well again.

Page 1 of 1 (4 items)