Developer Blog

Silverlight Support

BarryR — Tue, 16 Sep 2008 17:44:00 GMT

We are happy to announce we have added the ability to access your content from Silverlight. The latest release added a file at the root of both servies and the nodes that is a security file allowing Silverlight to connect to and download from both API calls as well as files. Part of the release was to add the Microsoft suggested Mime-types for hosting Silverlight files as well. Some restrictions on hosting the Silverlight application may be having to call the node directly to avoid the redirect.

The security file can be accessed at: http://services.nirvanix.com/clientaccesspolicy.xml and at the root of each node as well.

I look forward to our Silverlight developers showing off the latest interfaces to access both API calls and file downloads in the future. Please post a comment if you have a questions or concerns.

Callbacks and communicating with processing

BarryR — Thu, 14 Aug 2008 07:50:00 GMT

I have received a number of direct questions from Nirvanix users regarding callbacks and thought that I would share the information I sent to them with the entire community. Our processing servers allow you to work on requests that may take a significant amount of time. This means you can transcode videos that are > 200gb or any number of other supported large file operations. When processing a large file it's necessary to move these calls to an asynchronous process to avoid keeping an active connection open. Web interfaces aren't the best mechanism for long running connections and it can take quite a while to transcode a 200gb video depending on the CODEC. This setup requires a way to let you know when we are finished with the operation, this is where callbacks are used.

When something is added to our queue it will be picked up by our processing servers and when they are done with the request they will make a call to the URL presented in the initial request. The data returned depends on what operation you were performing. There will always be a response code passed which if 0 means the command succeeded as expected. If it is non-zero there will also be an error message. This is necessary because the initial call to the method only adds the request to the queue and does not actually perform the operation. You should receive two separate callbacks, the first is a notification that the file has been added to the queue successfully (or un successfully) and the second is notification that the file has been processed successfully (or unsuccessfully). Cases where the initial notification would not be 0 include paths already existing, users reaching their set limits or other problems.

The callback URL you submit to us is in the following format http://www.mysite.com/mycallback.php which would reside on your site.

Once a file has completed processing you will receive the following information:

NVX.returnCode an integer representing the status of the upload. Did it fail? Succedd? For valid response codes: see standard return codes; return code 0 means no error.
NVX.errorMessage a string describing the error message associated with any error being returned.
NVX.absolutePath a string describing the absolute path of the file that was uploaded.
NVX.sizeBytes a long describing the size in bytes of the file that was uploaded.

Transcoding Audio / Video

The transcode calls can take some time to convert the files depending on the input and output formats as well as the quality settings. For this callback the operation will only return if the call was successful as well as any errors that have occurred during processing. If you are converting to FLV / MP3 formats you can use this callback to determine when you should make the files visible to the users of your system.

Image Resize / Rotate

The image calls usually occur very quickly and will return if the call was successful as well as any errors that have occurred during processing. Possible reasons for failure would be that the submitted format is unknown to us. Often this call is used to swap a preview image with a generated thumbnail. This can be a good queue for your system to present a resized image instead of a stock icon if you have a way to show previews of image files on your system.

Sideload

For sideloads we are actually going out on the web and making requests of external sites which can have significant overhead if the site we are contacting should be slow or unresponsive. It is critical that you capture sideload errors since this is calling to an external system which can often behave in unexpected ways. The sideload call is a great way to save external data into your account without having to proxy the files through your own system.

Upload

Uploads are different in what they return so you can do post processing on a file after it has been uploaded. Often systems using our web services allow users to upload directly to our servers which allows you to avoid proxying files and letting your customers upload the files directly to us. The challenge is having a way to update your system or local database with the files those users have uploaded. The callback can be used as a convenient way to keep your local system informed of when a user uploads their own file. Another use of this callback is to execute a secondary action when the upload is complete. You can check the file for the extension to see if its allowed, generate a thumbnail on an image or any number of operations.

Sample Callback Handler in PHP

<?php 

$myFile = "requestlog.txt";
$fh = fopen($myFile, 'a') or die("can't open file");
fwrite($fh, 'Requesting URI: ' . $_SERVER['REQUEST_URI'] . "\r\n");
fwrite($fh, 'Request Time: ' . date("D M j G:i:s T Y") . "\r\n");
fwrite($fh, 'Query String: ' . $_SERVER['QUERY_STRING'] . "\r\n\r\n");

fclose($fh);

?>

The handler will append the results of each callback to a file called requestlog.txt. Make sure your web server (PHP) has write privileges to that file.

ItemID Changes

BarryR — Wed, 02 Jul 2008 15:35:00 GMT

We wanted to let everyone know of a change to the ItemID structure. This is significant if you are storing the ItemID along with your file information. This ID was improved to make sure it will always be Unique across an application. Since the change will affect existing item ID's we suggest that if you are already storing the Item ID you update your database with the new value returned from GetPathInfo.

Please let us know if you have a large number of Item IDs stored or if you will have any trouble with this change. We will work with you through the transition. We will be making the change on the next release date that is targeted for next week.

Base Storage

adrian — Sat, 21 Jun 2008 20:15:00 GMT

All,

We have removed the option to sign up for base storage from our sign up process. This will not affect those of you who are currently signed up for Base storage and your service activity will continue to be locked to the node you specified. We did this because of the benefits our SDN customers have experienced allowing the intelligent uploading, downloading, and movement of files throughout the global Storage Delivery Network. We continue to work hard to provide you with the highest quality Cloud Storage service possible. If you have any questions feel free to contact us at info@nirvanix.com and as always thank you for your continued support.

Staying Secure, Security Best Practices

BarryR — Thu, 29 May 2008 15:34:00 GMT

The security model we have designed allows for a few options when deciding how to stay secure. Today's topic will be to explore security and how you can be sure you know all of the primary methods for keeping data and your credentials secure.

Authentication

A session token is generated using an application key, username and password. With these three credentials you authenticate to a specific application. Its important that you secure the Application Key because like your username it is also part of the total authentication into the system.

Passwords

Each of the password change methods require a full re-authentication by the user. This is to prevent someone from stealing a session token and taking over an account. If you use SSL there should be no way for an external party to take over your account.

Session Tokens

The session token was designed to give you a sliding window in which you can operate on the APIs from a single external location. A session token is in essence the key to our web services after you have proven you are who you say you are. This token is locked to a single IP Address always. This is because a session token is something that is very specific to a user and should never be given to anyone else.

The session token not only will let you get access to the web services but it will also let you download your files. Below is a URL that has been built based on a session token that will access the file file.txt in "application1" and childuser1 which in this example is a child account.

http://services.nirvanix.com/SESSIONTOKEN/application1/childuser1/path/file.txt

Session tokens stay active for 20 minutes unless they are used. Whenever a session token is used it will be updated to be active for another 20 minutes. This means if your session token stays active it could be available indefinately. Because of this its a good idea to call logout which destroys a session token. The reason this is important is to prevent anyone who may be watching your http traffic.

The best use case for a session token is to login, do a single operation using HTTPS and then logout. However, there are a number of reasons this isn't always possible. If you have large amounts of data to pass or are extremely time sensitive the addition of SSL could be too much overhead. If you find yourself in such a situation be aware that if someone is able to get a session token they will still have to try to spoof your IP Address to use the token. This is why its always a best practice to call logout at any time you have completed your operation because that will instantly invalidate the session token assuring you that no-one else may use that token either.

Upload Tokens

The upload token is another form of validation that can be created to allow users to interact with the system. The reason the upload token is separate from session tokens is to allow the ability to pass to the end client in a web site scenario this token to do an upload. If a session token were used the end client could use that token to call any method under their account. With the addition of the GetStorageNodeExtended method you can now also lock an upload token to a specific IP and allow them to only upload to a specific directory.

Download Tokens

A token can be generated with GetOptimalURLs which lets you access a file for a specified period of time. The token can be created in a way that is similar to upload tokens that allows you to only allow a specific IP Address. The download tokens will timeout after a specified period of time. The suggestion is always to keep the timeout as short as possible since this token cannot be destroyed later. If you distribute the link and it is not locked to a specific IP Address anyone can download that file until it expires.

Password Locking

If the login system is called and fails 5 times your login will be locked. This is a temporary lock that will be released after 15 minutes. This is necessary to prevent attempts at guessing passwords. You can unlock your account at the Nirvanix Management Portal (NMP) (http://nmp.nirvanix.com/) using the security question answer.

Re-setting Application Key

The application key should be considered a secure item since it identifies your application. Should someone get your application key and you are concerned that you have been compromised you can regenerate it using the NMP. Any applications using the old key will not be able to login again until the new application key is used.

Additional Security Information

A whitepaper is available that details eavesdropping, impersonation and database attacks. You can find this and other important whitepapers at: Nirvanix Whitepapers

Java SDK 1.3 Released

BarryR — Thu, 01 May 2008 23:06:00 GMT

The new Java SDK has been released. This version includes the following changes:

1. Content-Range was added to support any size files through partial file upload.
2. Added File object array in UploadSummary event after an upload is completed.
3. Added single file uploads through path or java.io.File objects.
4. Fixed thread bug in uploader.
5. Added file overwrite through GetStorageNodeExtended. This can be accessed through an overloaded method in the FileUploader constructor.
6. PresetTranscode has been added.

The Java SDK has been receiving a good amount of attention lately. We just stopped in on the San Diego Java Users Group to discuss integration with Nirvanix using Java. The advantages are clear when you look at the simplicity when porting across platforms. We have compiled the Jar using the 1.5 JDK since the majority of the users are still using 1.5. Later we can compile to multiple versions if requested, the source code is available in the same distribution as well.

We appreciate all suggestions you wish to give us with our SDKs or sample applications. Please leave a note in the forums if you have any questions or comments.

You can find the SDK at: http://developer.nirvanix.com/files/folders/j2ee_java/entry57.aspx

New features in depth

BarryR — Tue, 08 Apr 2008 17:08:00 GMT

Our latest release adds useful tools for your applications. The API now has two new additions in the form of GetPathInfo and Upload callbacks to give you more details about your files and notification when an upload completes.

GetPathInfo

There were many requests for this API call that will give detailed information about a specific path. We have thrown in the kitchen sink with this call; it's fast and has the ability to return every bit of information about a single file or folder. The call returns the following information:

ItemName, IsFile, CreatedDate, SizeBytes, FileType, MetaData, Tags, IsShared, ModifiedDate and ItemID

The ItemID is a new value that is returned in this call. It is an ID that is unique across a single application. This does not tie to the filename but to the file itself.

In REST the xml response looks like this:

<?xml version="1.0" encoding="utf-8"?>
<Response>
   <ResponseCode>0</ResponseCode>
   <GetPathInfo>
      <ItemName>F8.jpg</ItemName>
      <IsFile>true</IsFile>
      <CreatedDate>Wed, 02 May 2007 13:36:41 GMT</CreatedDate>
      <SizeBytes>1000000</SizeBytes>
      <FileType>Image</FileType>
      <Metadata>
      	<MetaData FileName="F8.jpg">
              <Data Key="Height">480</Data>
	      <Data Key="MD5">JCiPH3sH4S3f/Ad93uWk+w==</Data>
              <Data Key="Width">640</Data>
        </MetaData>
      </Metadata>
      <Tags></Tags>
      <IsShared>false</IsShared>
      <ModifiedDate>Wed, 02 May 2007 13:36:41 GMT</ModifiedDate>
      <ItemID>1045632</ItemID>
   </GetPathInfo>
</response>

Upload Callback

The ability to receive a callback after an upload is a critical step when letting a client perform uploads directly to the Nirvanix servers. The primary use for this is when you use a browser to do an HTTP Upload. It gives you an option to receive an update telling your system that a file has been updated. Usually this is used in concert with a database that keeps information on the files and mirrors which appear on the Nirvanix file system.

The parameter is quite simple to add: you will need a new HTTP parameter called callbackURL. The callbackURL parameter takes a fully qualified URI including the server you wish to call. You will need a web server accepting requests to handle this as well. This functionality works the same as all of the rest of our callback operations. The callback is initiated as soon as the upload is complete but the processing to get the MD5 and other metadata may still continue after the callback is returned

Transcoding

BarryR — Fri, 21 Mar 2008 14:14:00 GMT

What is transcoding and how does this help my business? We hear this question quite often. Transcoding is the conversion of one format to another, for example, conversion of windows media video to Flash or .avi to h.264. Since the launch of the beta functionality, we have been gathering feedback and monitoring usage to get the engine to a point where we can quickly add output formats and at the same time simplify the input format.

Codecs tend to be complex which often makes the parameters coming in quite difficult to manage. Adding to that complexity each codec has its own "best" settings which will vary largely between each. The goal of any good API is always to strip away complexity and leave you with a simple interface that does everything you want it to which is what PresetTranscode attempts to do. The goal from the outset was to provide a simple way to convert video from one format to another eliminating the need for the user to know every nuance of the codec and automatically use the “best” settings. At the same time we introduced a new more robust transcoding engine under the hood to achieve this and support new features we will be adding in coming releases.

While our old engine and API are still available through the Transcode call we will be migrating all of the functionality over to the new engine and PresetTranscode over time. We will continue to provide updates and work with customers as we phase the Transcode call out.

You may be asking what the advantage of using the new engine over the old is for you other than the simplified interface. With the new call we have introduced three levels of quality which were not previously available and made the engine easier to configure. What this translates to is a fast and easy way to introduce more presets allowing for a large number of new options and quality levels.

Using the PresetTranscode call is simplifies the task since the majority of the parameters are wrapped up in the preset. To further simplify our example I am using the Nirvanix SDK in the Zend framework to do the sample below. This sample assumes you already uploaded a file called vacation01.mpg to your /video directory under the account you wish to access.

PHP Zend Code Example:

<?php
set_include_path(get_include_path() . PATH_SEPARATOR .'C:/Zend/incubator/library' . PATH_SEPARATOR .'C:/Zend/library');
require_once 'Zend/Service/Nirvanix.php';
// First create the Zend object and get a Session

$appkey = 'APPKEY';
$username = 'USERNAME';
$password = 'PASSWORD';
$auth = array('username' => $username,'password' => $password,'appKey'   => $appkey);
try {
   
   // This authenticates the user and retrieves session

   $nirvanix = new Zend_Service_Nirvanix($auth);
   
   // This retrieves the Video Namespace object

   $video = $nirvanix->getService('Video');
   $params = array('srcFilePath' => '/videos/vacation01.mpg',
                   'destFilePath' => '/videos/vacation01.flv',
                   'preset' => 'H263 FLV Medium',
                   'scaleWidth' => '320',
                   'scaleHeight' => '200');
   
   // We won't wait for the response code to come back since the
   // Zend SDK will thrown an exception if the reponse code is not 0

   $video->PresetTranscode($params);
}
   
?>

This will output a flash video file to /videos/vacation01.flv. This file can then be embedded in a flash player and streamed back to the user. You can read more about transcoding in the SDK document at:

http://developer.nirvanix.com/sitefiles/1000/API.html#_TocPresetTranscode

Welcome to the Zend/PHP community

adrian — Mon, 17 Mar 2008 17:17:00 GMT

In case you haven't heard yet Zend 1.5 is now available and includes integrated Nirvanix support! This means that when you download and install the Zend 1.5 framework all you need to do is call the Nirvanix library, input your Nirvanix credentials and you have access to all of the Nirvanix functionality via the Zend/Nirvanix wrappers. This will significantly reduce the development time of any PHP/Nirvanix application by providing access to all of the Nirvanix WSDLs and integrated error handling. Combine this with support from both Nirvanix and Zend and PHP developers now have an easy and fully supported way to integrate scalable storage and media processing into any PHP app!

You can access a version of the PHP hosted file manager built via Zend here - Nirvanix PHP Zend hosted file manager

You can get more info and downlaod the Zend 1.5 framework here - Zend 1.5 Framework

We will post a version of the more robust PHP file manager built via Zend in the coming weeks. Subscribe to this blog or to the Nirvanix PHP/Zend download section to receive notification

Stay Connected

adrian — Tue, 05 Feb 2008 01:48:00 GMT

We are replacing our Developer News & Events section with the Developer Blog in order to seamlessly communicate with all Nirvanix developers. You can now subscribe to this blog for product updates, developer news, notification of sample app/code snippet posting and anything else we think would interest developers. We hope this help you get to the info you need more quickly. Let us know if we can do anything else to make your development efforts easier!