pure-ftpd can't delete files in /nirvanix directories but can upload/download

Last post 07-06-2010 3:11 PM by DuroSoft. 12 replies.
Page 1 of 1 (13 items)
Sort Posts: Previous Next
  • 07-01-2010 7:17 PM

    pure-ftpd can't delete files in /nirvanix directories but can upload/download

    Ok so I'm using theplanet's cloud storage on a theplanet cloud server running centos 5 and using cloudnas to connect to cloud storage. ThePlanet's modified version of cloudnas apparently uses /nirvanix as the data directory instead of /nirvanix/[account name]. I have set up pure-ftpd such that /nirvanix/testuser is the home directory for an ftp account (I had to chmod 777 the "testuser" directory to get pure-ftpd to be able to read/write in that directory).

    the pure-ftpd setup actually works pretty well. When I connect using the testuser account, I can see files in that directory, upload new files, rename existing files, create directories, and delete directories over FTP with no issue.

     The problem is when I try to delete a file (using FileZilla on windows 7), I get the following response from the server:

    Command:    DELE test image.gif
    Response:    550 Could not delete test image.gif: Invalid argument

     If I refresh the file listing, "test image.gif" gets renamed to ".pureftpd-rename.32673.a898a70b" but does not get deleted. The same happens for all other files when I try to delete them (obviously the random numbers/letters are different, though).

    I'm pretty sure this isn't a permissions issue because I had already chmodded the whole directory to 777, and if it were, why would uploads, downloads, and renames all work?

     I think it has something to do with the way cloudnas caches files. I tried disabling the cache and restarting cloudnas, but I still have the same issue with deleting files over ftp. 

    I need to figure out whether this is a cloudnas-related issue or a pure-ftpd related issue. If its a problem with my pure-ftpd setup, I'll ask in the appropriate forums on another site. Just want to verify that this isn't some limitation of cloudnas.

     

    FYI I am using mysql-based authentication with pure-ftpd, following this tutorial to the letter except where it deals with home directories:

    http://www.howtoforge.com/virtual_hosting_with_pureftpd_and_mysql_centos5.0

    thanks for any help :)

     

  • 07-01-2010 7:30 PM In reply to

    • BarryR
    • Top 10 Contributor
    • Joined on 07-20-2007
    • San Diego
    • Posts 885

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    Thank you for the detailed report. We haven't done extensive testing of pure-ftpd so I'm not sure where the problem is. I will do some tests today to see if i can reproduce your issue.


    Regards,
    Barry R.
    IM Support (Feel free to add me)

    MSN: barryruffner@live.com
    Gmail: barryruffner@gmail.com
  • 07-01-2010 7:31 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

     the system log provides a little more information... here is what happens in the system log when I try to delete a file:

    Jul  1 13:57:31 hostname117865 CloudNAS: [4]Warning: GetPathStatus: Path=/testuser/.pureftpd-rename.588.af0cc336 Response=2  Nirvanix Response PathNotFound, Path [/testuser/.pureftpd-rename.588.af0cc336] not found.
    Jul  1 13:57:32 hostname117865 CloudNAS: [3]: [588] 270ms RenamePath: NewPath=/testuser/.pureftpd-rename.588.af0cc336 Path=/testuser/test image.gif

     

    Interestlingly, here is what happens when I rename a file, which does work, but seems to create some interesting errors in the log:

     

    Jul  1 13:36:41 hostname117865 CloudNAS: [3]Warning: GetPathStatus: Path=/testuser/testform2.pdf Response=2  Nirvanix Response PathNotFound, Path [/testuser/testform2.pdf] not found.
    Jul  1 13:36:41 hostname117865 CloudNAS: [2]Warning: GetPathStatus: Path=/testuser/testform2.pdf Response=2  Nirvanix Response PathNotFound, Path [/testuser/testform2.pdf] not found.
    Jul  1 13:36:41 hostname117865 CloudNAS: [1]: [32462] 53ms RenamePath: NewPath=/testuser/testform2.pdf Path=/testuser/testform.pdf
    Jul  1 13:36:41 hostname117865 pure-ftpd: (testuser@69.74.223.2) [NOTICE] File successfully renamed or moved: [testform.pdf]->[testform2.pdf]
     

  • 07-02-2010 12:23 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    any luck?
  • 07-02-2010 1:59 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

     based on the logs my theory is that pure-ftpd does two things when it deletes a file. First it renames the file to a .pureftpd-rename.[random digits] file. Then it tries to delete it. In the logs what looks like is happening is pure-ftpd goes to rename the file, and then instantly tries to delete it before receiving any sort of confirmation that the file was renamed. I hope there is something I can do about this.

  • 07-02-2010 4:58 PM In reply to

    • BarryR
    • Top 10 Contributor
    • Joined on 07-20-2007
    • San Diego
    • Posts 885

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    I have reproduced the problem and it looks like you are right about the rename / delete before confirmation. I am looking to see if there is any way to change that behavior and deal with the issue. We have seen this issue in the past where certain programs expect a level of performance and when that isn't honored they fail. Often this can be seen when the same application is pointed at network drives.

    I will let you know what I find.

    Regards,
    Barry R.
    IM Support (Feel free to add me)

    MSN: barryruffner@live.com
    Gmail: barryruffner@gmail.com
  • 07-02-2010 5:58 PM In reply to

    • BarryR
    • Top 10 Contributor
    • Joined on 07-20-2007
    • San Diego
    • Posts 885

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    I found something interesting, if you do not authenticate through the mysql database it doesn't exhibit the issue. Below is the config file I am using that does not show the problem.

    ** EDIT: It looks like the tags were eaten by the forum. ** Here is a link to the config: http://services.nirvanix.com/NirvanixDrive/public/email/proftpd.conf?disposition=attachment

    This is based on the standard config and setup to allow an anonymous user to login and delete with the folder and files all set to 777. (please do not use this on a production system its completely unlocked for anonymous). Let me know if this helps, I'm not certain why the mysql config I am using has the error but it could be something to do with the time it takes to authentication the security for the deletion.

    Regards,
    Barry R.
    IM Support (Feel free to add me)

    MSN: barryruffner@live.com
    Gmail: barryruffner@gmail.com
  • 07-02-2010 6:54 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    Very interesting. My setup is actually using pure-ftpd (not proftpd... I also got the two confused a bunch of different times when I was doing my initial setup)... did I say proftpd by accident? I apologize if that is the case.

    Interesting that the same issue occurs across both programs. I take it you are able to rename / move files, but when you delete you get the invalid argument thing as well?

    Maybe I could set up some sort of proxy that intercepts incoming ftp commands and handles the DELE command directly instead of passing it on to pure-ftpd. Most of my development experience is windows specific, so doing so would be rather difficult. Maybe I could just modify the pure-ftpd source code directly

     

    By the way any idea what the proper way would be for me to give pure-ftpd permissions to access /nirvanix? I don't know how to set multiple users as the owner for a file / directory in linux, so I ended up chmodding the whole directory to 777, which I gather is insecure? The account pure-ftpd runs under is set up with very limited access... would having a directory with 777 that it has access to make the system vulnerable? I won't be using anonymous FTP, but I still dont want my users to be able to get into the shell or execute files on the server (like somehow run shell scripts) or anything like that. 

  • 07-02-2010 6:57 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

     the reason I need mysql authentication is because I need very detailed bandwidth and tracking information on all my users.. if that is possible with normal authentication, I could go that route. It looks like pure-ftpd dumps everything in /var/log/pure-ftpd.log anyway, so maybe my php scripts could just read that for bandwidth data. As long as the accounts are created / deleted from some sort of config file, my php scripts could modify that as well instead of doing SQL queries.

    Very peculiar that SQL auth has this sort of effect, particularly considering that it ONLY seems to affect file deletion, not directory deletion, or file moving / uploading / creation / renaming.

  • 07-02-2010 7:17 PM In reply to

    • BarryR
    • Top 10 Contributor
    • Joined on 07-20-2007
    • San Diego
    • Posts 885

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    DuroSoft:
    By the way any idea what the proper way would be for me to give pure-ftpd permissions to access /nirvanix? I don't know how to set multiple users as the owner for a file / directory in linux, so I ended up chmodding the whole directory to 777, which I gather is insecure? The account pure-ftpd runs under is set up with very limited access... would having a directory with 777 that it has access to make the system vulnerable? I won't be using anonymous FTP, but I still dont want my users to be able to get into the shell or execute files on the server (like somehow run shell scripts) or anything like that. 

    You will want to setup groups with groupadd and then assign the group of the destination files / folders. Then you just give group permissions to the files. After that you can assign the user to that group and they will inherit the permissions set on the files.

    From the pure-ftpd source:

    What we do here may look a bit strange. It's to defend against
         * change-after-stat attacks. If we simply do lstat(name), then unlink(name)
         * there's a race. An attacker can rename the file between these two
         * system calls, so that a big file is lstat()ed, but a dummy tiny file is
         * unlinked. That way, an attacker could easily get extra quota.
         * To defend against this attack, we rename the file to an unique dot-file
         * (an atomic operation) . People subject to quotas can't access dot-files.
         * So we can securely stat it and unlink it. Having the pid in the file
         * name should be enough to avoid that two concurrent sessions create the
         * same temporary file. But to be paranoid to the extreme, we add some
         * random number to that.
    

    So it seems this is a way to prevent a race condition that allows users to exceed their quota. I'm not sure how that helps but at least its understood why the action is taking place.

    The good news about my misunderstanding of the pro vs pure is that I went back and tried playing with the mysql implementation and managed to get it working. I'm not sure if this solves your problem but it may provide a secondary avenue for you to try. I was able to get delete working in proftpd after changing my settings back while using mysql for the users.

    ** EDIT: I used this guide and it worked for me perfectly: http://www.howtoforge.com/proftpd_mysql_virtual_hosting **


    Regards,
    Barry R.
    IM Support (Feel free to add me)

    MSN: barryruffner@live.com
    Gmail: barryruffner@gmail.com
  • 07-02-2010 7:27 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

     so what settings were you able to get pro-ftpd to work with? [edit: nevermind, saw the link]

     I won't be needing pure-ftpd-managed quotas, so is there a way I can disable that feature in pure-ftpd? I have already tried enabling/disabling the atomic renaming option in the config, and that doesn't seem to make a difference.

     If it works with proftpd, I could just use that. 

  • 07-02-2010 7:36 PM In reply to

    • BarryR
    • Top 10 Contributor
    • Joined on 07-20-2007
    • San Diego
    • Posts 885

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

    I started fresh with the default config and changed it according to the second page of the link I sent: http://www.howtoforge.com/proftpd_mysql_virtual_hosting_p2

    I think I must have changed the security previously which was giving me an error on delete and it was not the same error as the one pure-ftpd was showing.


    Regards,
    Barry R.
    IM Support (Feel free to add me)

    MSN: barryruffner@live.com
    Gmail: barryruffner@gmail.com
  • 07-06-2010 3:11 PM In reply to

    Re: pure-ftpd can't delete files in /nirvanix directories but can upload/download

     works flawlessly -- thanks so much!

Page 1 of 1 (13 items)