Developer Blog

Callbacks and communicating with processing

 I have received a number of direct questions from Nirvanix users regarding callbacks and thought that I would share the information I sent to them with the entire community. Our processing servers allow you to work on requests that may take a significant amount of time.  This means you can transcode videos that are > 200gb or any number of other supported large file operations.  When processing a large file it's necessary to move these calls to an asynchronous process to avoid keeping an active connection open.  Web interfaces aren't the best mechanism for long running connections and it can take quite a while to transcode a 200gb video depending on the CODEC. This setup requires a way to let you know when we are finished with the operation, this is where callbacks are used.

 When something is added to our queue it will be picked up by our processing servers and when they are done with the request they will make a call to the URL presented in the initial request.  The data returned depends on what operation you were performing.  There will always be a response code passed which if 0 means the command succeeded as expected.  If it is non-zero there will also be an error message. This is necessary because the initial call to the method only adds the request to the queue and does not actually perform the operation. You should receive two separate callbacks, the first is a notification that the file has been added to the queue successfully (or un successfully) and the second is notification that the file has been processed successfully (or unsuccessfully).  Cases where the initial notification would not be 0 include paths already existing, users reaching their set limits or other problems.

The callback URL you submit to us is in the following format http://www.mysite.com/mycallback.php which would reside on your site.

Once a file has completed processing you will receive the following information:

NVX.returnCode an integer representing the status of the upload. Did it fail? Succedd? For valid response codes: see standard return codes; return code 0 means no error.
NVX.errorMessage a string describing the error message associated with any error being returned.
NVX.absolutePath a string describing the absolute path of the file that was uploaded.
NVX.sizeBytes a long describing the size in bytes of the file that was uploaded.
 

Transcoding Audio / Video

The transcode calls can take some time to convert the files depending on the input and output formats as well as the quality settings.  For this callback the operation will only return if the call was successful as well as any errors that have occurred during processing.  If you are converting to FLV / MP3 formats you can use this callback to determine when you should make the files visible to the users of your system.

Image Resize / Rotate

The image calls usually occur very quickly and will return if the call was successful as well as any errors that have occurred during processing.  Possible reasons for failure would be that the submitted format is unknown to us.  Often this call is used to swap a preview image with a generated thumbnail.  This can be a good queue for your system to present a resized image instead of a stock icon if you have a way to show previews of image files on your system.

Sideload

For sideloads we are actually going out on the web and making requests of external sites which can have significant overhead if the site we are contacting should be slow or unresponsive.  It is critical that you capture sideload errors since this is calling to an external system which can often behave in unexpected ways.  The sideload call is a great way to save external data into your account without having to proxy the files through your own system.

Upload

Uploads are different in what they return so you can do post processing on a file after it has been uploaded.  Often systems using our web services allow users to upload directly to our servers which allows you to avoid proxying files and letting your customers upload the files directly to us.  The challenge is having a way to update your system or local database with the files those users have uploaded.  The callback can be used as a convenient way to keep your local system informed of when a user uploads their own file.  Another use of this callback is to execute a secondary action when the upload is complete.  You can check the file for the extension to see if its allowed, generate a thumbnail on an image or any number of operations.

 Sample Callback Handler in PHP

 

<?php 

$myFile = "requestlog.txt";
$fh = fopen($myFile, 'a') or die("can't open file");
fwrite($fh, 'Requesting URI: ' . $_SERVER['REQUEST_URI'] . "\r\n");
fwrite($fh, 'Request Time: ' . date("D M j G:i:s T Y") . "\r\n");
fwrite($fh, 'Query String: ' . $_SERVER['QUERY_STRING'] . "\r\n\r\n");

fclose($fh);

?>
The handler will append the results of each callback to a file called requestlog.txt.  Make sure your web server (PHP) has write privileges to that file.

Comments

No Comments