I am struggling with the following:

1- I have a form from which users can upload files to a common space (no problem with this).

2- I want to be able to automatically set some custom metadata to the uploaded file. E.G. "submittedby", etc once the uploaded has succeeded

I can achieve the above by the use of 2 calls:
    - UploadFile
    - SetMetadata

but I am struggling on how to identify the uploaded file with certainty.

I can do some workaround by retrieving the last created file via ListFolder and then set the metadata, but this is not a bullet proof solution in case of concurrent uploads.

So, more in general, the issue is: after uploading a file,  how can I have a unique reference to that file so that I can perform operations to it?

Any suggestion appreciated...

thanks in advance

ciao Luca

 This has been identified as a feature that is high on our priority list.  We are currently working to get this in.  The future implementation would call a URL via callback on upload.  Unfortunately for now your suggested workaround is what others have used.  We will let you know when this feature becomes available.

The unique reference could be the MD5 which is part of the metadata after it has been calculated.

Regards,
     Barry R.

real pity...

 >The unique reference could be the MD5 which is part of the metadata after it has been calculated.

I am not sure this will work...

the MD5 is actually calculated out of the file content at the moment, so if you upload the same file under 2 different names you will still get the same MD5... 

 

I haven't had a chance to use it myself, but I believe Http Upload supports a forwardingUrl.

If you could be relatively sure your users would have Javascript enabled, then perhaps you could add a few of your own custom parameters (like the path of the file about to be uploaded) just before the form is uploaded, and then handle them when the request is redirected to your custom url.

 

OR (and this would be ugly)

 

I believe GetStorageNodeExtended lets you lock in what directory the file will be uploaded to (even if it doesn't exist yet) when you're generating the upload token. If you made the folder something unique and specific, you would always know where to look for that specific file (the only file in that new, unique directory).

Just some thoughts... 

Adam 

 

Hi Adam,

appreciate your thoughts, I guess some others will have the same issue...

Javascript is a risky solution, I think option two or something similar would be the only safe one.

 I just did a test uploading 2 files at the same time and with the ListFolder trick it was going to pick the latest uploaded file for setting the metadata, so the "trick" is not really working...

the bigger the files the bigger the chances of getting concurrent requests...

requires some more thinking :-) 

Luca:

Maybe I'm a little of base here but after reading through the API's again I couldn't find any solution of appending metadata to a file as you upload it.  However, why not just rename the file on upload?  For example:

Let's say that a user logs in and his/her username is DaMan.  DaMan wants to upload a file called PicOfDaMan.gif.  You want to be able to reference this file after it's uploaded to append metadata to it.  So why not do this..

(I'm going to use SOAP but you can use HTTP Upload too here)...

On this line:  nirvTransfer.UploadFile(node.AccessToken, "UploadsFolder/PicOfDaMan.gif", fileData);

Why not change it to: nirvTransfer.UploadFile(node.AccessToken, "UploadsFolder/PicOfDaMan_DaMan", fileData);

So, when you want to reference the file again you can state the filename and use the _DaMan (that is, the username) and know who uploaded what.  And to be sure you could add a datetime to it too. but that could be overkill for your needs.

Just a thought.  Hope this helps.  Have a great day ya'll!

- Joshua

 Hi Joshua,

 thanks a lot for your contribution, I sincerely appreciate everyone's effort

I am not an expert and I mainly work by reverse-engineering from examples. At the moment though I cannot see how I can change the file name through a web form upload as the Upload.asx only takes 3 parameters and none of those seems to have influence on the  final  file name...

even if I could change the filename though, I could still end up with the concurrent request issue, where basically I cannot uniquely recall the file that was just uploaded and perform operations to it. In my case it's appending metadata, but it could be anything else depending on your needs. Maybe you need to move it to a different directory, maybe you want to share the file, etc etc...

the real issue is that there's no unique identifier that you can use to recall the uploaded file/s

I am thinking out loud, but probably the only unique reference to the file upload operation is the token, as it is known before the upload. If the token was included among the file information (like the MD5) you could always refer to the files uploaded with that token and perform operations to them...

interesting intellectual discussion anyway, curious to see how the final solution from Nirvanix will look like :-)

ciao Luca 

 

Just to move this discussion into the more concrete example space lets do an exercise of putting one of the above suggestions to the test.

<html>
<body>
<form onSubmit="forwardingURL.value =  'http://www.mydomain.com/myuploadcheck.php?filenameuploaded=' + myfileinput.value; return false;">
  <p>Filename Display: <input type="text" id="forwardingURL"></p>
  <p><input type="file" id="myfileinput"></p>
  <p><input type="submit" value="Submit"></p>
</form>
</body>
</html>

 

This example just puts the filename appended to a url into an input box.  It would be similar if you have defined a hidden form input for forwardingURL rather than an a text box.  The second change would be the "return false;" would be removed so it actually submits.  This sample relies on the users browser to pass the filename.  Because sometimes users cannot be trusted with this payload this could easily be handled as well by having on your application a way to know when that user does an upload if they return nonsense or something that does not match to do a full scan of the files uploading using the metadata to mark what files have been processed by your side.  So If I were keeping a sync'ed database I would mark the files with metadata saying the file has made it to my database.  By doing a listfolder whenever anything is amiss I would see what files do not have my Metadata mark.  I'm sure with this method there are several ways to prevent fraud or to do some secondary checks.

A second method is to proxy the upload by having an upload module on your servers that takes the input stream and opens a connection to Nirvanix and does a 1 to 1 move of the file.  This lets you have 100% control over both the user input and will also let you do whatever you like with the file since your in programatic control of the file upload.  I believe there are examples of PHP and C# upload methods here and SOAP was mentioned above.

I would like to know what you finally decide since this definitely helps us shape the future API calls and put emphasis on the current objectives.  Callback URL would be the best of both worlds but until thats available there are still a few ways to handle this.

Thanks for your comments,
    Barry R.
 

Hi Barry and all, 

I think I am going to make a staging area where the user will upload the files to his folder.

Something like /uploaded/username 

From there I will set the metadata for each file and then move the files to a common area after some proper checking.

I will have to build quite some controls and quite some exception handling though which will require more calls to the API.

I don't think that proxy the upload is an option as the whole concept of scalability goes out of the window in my opinion (if I get a lot of uploads at the same time my little server will die... :-)

I think the UploadFile call can be improved by implementing some options for manipulating the files. Something like the renaming feature mentioned above by Joshua, or by setting some metadata. Other cool feature could be MIME type detection etc etc...

hope it helps

ciao Luca

 

 

I'm having the same problem. maybe some way to send a filename for each file input in an hidden field could be nice or when you upload the files are rename with some temp filename that are sent back to the callback url. I found this really weird that if you upload two different files with the same name using http upload you get an error. I was thinking of creating a new folder with a know id at each upload( since the destfolder parameter creates a folder if it doesn't exist) and rename/move files from that folder after but the same name problem still occur.

 You could do two things in the case of the user selecting two files with the same name.  Either create a unique value and change the filename on submit via Javascript similar to the method above or you could use flash or a Java Applet.  We have seen good success with Jumploader for doing uploads: http://www.jumploader.com/. I don't normally endorse a single product since are many Java upload applets that work quite nicely.  This one happens to work and has been tested by me personally.

Regards,
     Barry R.

It's not impossible to change filename in a file input from javascript?

 Opps, you're right, I will have to retract that part of the post.  Please take a look at Flash or Java applet uploaders though. They give quite a nice user experience if you want to do client side uploads via the browser.  This doesn't work quite as well when you talk about mobile phones or other unusual platforms but should offer the majority of cases a good interface to upload.

~Barry R. 

I think I will go with flash since most browser support it without problem. thanks for your help