Re: link(2) and atomic file creation on NFS

Re: link(2) and atomic file creation on NFS

[James Youngman]

"Noel L Yap" <yap_noel@jpmorgan.com> writes:

> Would a mkdir do the trick?
> 
> Noel

Ordinarily, yes.  (As CVS does, of course).

However, SCCS itself (i.e. the venerable AT&T-as-was product) uses
lock _files_ of the form z.foo.  I hesitate before using lock
directories instead, because interoperability is the only (!) priority
of CSSC; I cannot afford to asopt a strategy which may fail to
interoperate completely with SCCS itself. 


> I have some existing code which uses link(2) to achieve atomic locking
> on Unix over NFS (since O_EXCL does not work over NFS).  The
> application (a GPLed SCCS clone) now compiles under Cygwin, but one
> remaining problem is that the lockfile code uses link(2) to create a
> hard link to an open file.


[snip]

-- 
James Youngman
Manchester, UK.  +44 161 226 7339
PGP (GPG) key ID for <jay@gnu.org> is 64A95EE5 (F1B83152).

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: link(2) and atomic file creation on NFS

[James Youngman]

Charles Wilson <cwilson@ece.gatech.edu> writes:

> James Youngman wrote:
> 
> 
> > This seems not to work under Cygwin B20.1 (over NT4).
> 
> 
> There have been approximately 12 billion changes in the network layer
> of cygwin over the last two years since B20.1 was current.  Please
> update to cygwin-1.3.x and see if your problems have already been
> fixed.

Er, darn.  Actually, I installed CYGWIN fresh (and for the first time
on that machine) from the Red Hat web site earlier this week.  I just
told you the wrong version number (blush!).

-- 
James Youngman
Manchester, UK.  +44 161 226 7339
PGP (GPG) key ID for <jay@gnu.org> is 64A95EE5 (F1B83152).

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: link(2) and atomic file creation on NFS

[yap_noel]

Would a mkdir do the trick?

Noel

I have some existing code which uses link(2) to achieve atomic locking
on Unix over NFS (since O_EXCL does not work over NFS).  The
application (a GPLed SCCS clone) now compiles under Cygwin, but one
remaining problem is that the lockfile code uses link(2) to create a
hard link to an open file.

This seems not to work under Cygwin B20.1 (over NT4).  The link count
of the file stays at 1 (I think that errno=EPERM, but I could be
wrong).

The link(2) usage is only relevant over NFS, but even if there is an
NFS client for NT [*does* one exist?], I would assume that the Cygwin
link(2) implementation would not correspond with a link(2) operation
via the NT NFS client.   So, the link(2) implementation loses under
NT.  Fine.

Is there a way of safely creating a lock file without race conditions
that will also work under Cygwin?   For the sake of the discussion,
here is the current code :-

(explanation: "mystring" is a C++ class a lot like the standard string
class, and in fact usually *is* the standard string class)


TIA for any helpful hints!


/* WARNING: If you use fstat() on the fd to get the link count, the
 * wrong value is returned for Linux 2.0.34 and glibc-2.0.7, the
 * second time we perform the fstat().  Therefore we use stat(2)
 * rather than fstat(2).
 */
static long get_nlinks(const char *name)
{
  struct stat st;

  if (0 == stat(name, &st))
    {
      return (long)st.st_nlink;
    }
  else
    {
      return -1L;
    }
}

static int atomic_nfs_create(const mystring& path, int flags, int perms)
{
  mystring dirname, basename;
  char buf[32];
  const char *pstr = path.c_str();

  split_filename(path, dirname, basename);

  /* Rely (slightly) on only 11 characters of filename. */
  for (long attempt=0; attempt < 10000; ++attempt)
    {
      /* form the name of a lock file. */
      sprintf(buf, "nfslck%ld", attempt);
      const mystring lockname = dirname + mystring(buf);
      const char *lockstr = lockname.c_str();

      errno = 0;
      int fd = open(lockstr, flags, perms);
      if (fd >= 0)
        {
          if (1 == get_nlinks(lockstr))
            {
              int link_errno = 0;
              errno = 0;
              if (-1 == link(lockstr, pstr))
                link_errno = errno;

              /* ignore other responses */

              if (2 == get_nlinks(lockstr))
                {
                  unlink(lockstr);
                  return fd;    /* success! */
                }
              else              /* link(2) failed. */
                {
                  if (EPERM == link_errno)
                    {
                      /* containing filesystem does not support hard links.
*/
                      close(fd);
                      unlink(lockstr);

                      /* assume that the filesystem supports O_EXCL if it
does
                       * not supprort link(2).
                       */
                      return open(pstr, flags, perms);
                    }
                }
            }
          close(fd);
          unlink(lockstr);
        }
      else                      /* open() failed. */
        {
          switch (errno)
            {
            case EEXIST:
              /* someone else got that lock first; they may in fact not
               * be trying to lock the same s-file (but instead another
               * s-file in the same directory)
               *
               * Try again.  Sleep first if we're not doing well,
               * but try to avoid pathalogical cases...
               */
              if ( (attempt > 4) && (attempt & 1) == (getpid() & 1) )
                {
                  errormsg("Sleeping for one second while "
                           "waiting for lock\n");
                  sleep(1);
                }
              break;

            default:            /* hard failure. */
              /* fall back on the less-safe method, which will
               * probably still fail
               */
              return open(pstr, flags, perms);
            }
        }
    }
  return -1;
}


--
James Youngman
Manchester, UK.  +44 161 226 7339
PGP (GPG) key ID for <jay@gnu.org> is 64A95EE5 (F1B83152).

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/





This communication is for informational purposes only.  It is not intended as
an offer or solicitation for the purchase or sale of any financial instrument
or as an official confirmation of any transaction. All market prices, data
and other information are not warranted as to completeness or accuracy and
are subject to change without notice. Any comments or statements made herein
do not necessarily reflect those of J.P. Morgan Chase & Co., its
subsidiaries and affiliates.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: link(2) and atomic file creation on NFS

[Charles Wilson]

James Youngman wrote:


> This seems not to work under Cygwin B20.1 (over NT4).  


There have been approximately 12 billion changes in the network layer of 
cygwin over the last two years since B20.1 was current.  Please update 
to cygwin-1.3.x and see if your problems have already been fixed.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

link(2) and atomic file creation on NFS

[James Youngman]

I have some existing code which uses link(2) to achieve atomic locking
on Unix over NFS (since O_EXCL does not work over NFS).  The
application (a GPLed SCCS clone) now compiles under Cygwin, but one
remaining problem is that the lockfile code uses link(2) to create a
hard link to an open file.  

This seems not to work under Cygwin B20.1 (over NT4).  The link count
of the file stays at 1 (I think that errno=EPERM, but I could be
wrong).  

The link(2) usage is only relevant over NFS, but even if there is an
NFS client for NT [*does* one exist?], I would assume that the Cygwin
link(2) implementation would not correspond with a link(2) operation
via the NT NFS client.   So, the link(2) implementation loses under
NT.  Fine. 

Is there a way of safely creating a lock file without race conditions
that will also work under Cygwin?   For the sake of the discussion,
here is the current code :-

(explanation: "mystring" is a C++ class a lot like the standard string
class, and in fact usually *is* the standard string class)


TIA for any helpful hints!


/* WARNING: If you use fstat() on the fd to get the link count, the
 * wrong value is returned for Linux 2.0.34 and glibc-2.0.7, the
 * second time we perform the fstat().  Therefore we use stat(2)
 * rather than fstat(2).
 */
static long get_nlinks(const char *name)
{
  struct stat st;

  if (0 == stat(name, &st))
    {
      return (long)st.st_nlink;
    }
  else
    {
      return -1L;
    }
}

static int atomic_nfs_create(const mystring& path, int flags, int perms)
{
  mystring dirname, basename;
  char buf[32];
  const char *pstr = path.c_str();
  
  split_filename(path, dirname, basename);

  /* Rely (slightly) on only 11 characters of filename. */
  for (long attempt=0; attempt < 10000; ++attempt)
    {
      /* form the name of a lock file. */
      sprintf(buf, "nfslck%ld", attempt);
      const mystring lockname = dirname + mystring(buf);
      const char *lockstr = lockname.c_str();

      errno = 0;
      int fd = open(lockstr, flags, perms);
      if (fd >= 0)
        {
          if (1 == get_nlinks(lockstr))
            {
              int link_errno = 0;
              errno = 0;
              if (-1 == link(lockstr, pstr))
                link_errno = errno;

              /* ignore other responses */
              
              if (2 == get_nlinks(lockstr))
                {
                  unlink(lockstr); 
                  return fd;    /* success! */
                }
              else              /* link(2) failed. */
                {
                  if (EPERM == link_errno)
                    {
                      /* containing filesystem does not support hard links. */
                      close(fd);
                      unlink(lockstr);

                      /* assume that the filesystem supports O_EXCL if it does
                       * not supprort link(2).
                       */
                      return open(pstr, flags, perms);
                    }
                }
            }
          close(fd);
          unlink(lockstr); 
        }
      else                      /* open() failed. */
        {
          switch (errno)
            {
            case EEXIST: 
              /* someone else got that lock first; they may in fact not
               * be trying to lock the same s-file (but instead another 
               * s-file in the same directory)
               *
               * Try again.  Sleep first if we're not doing well,
               * but try to avoid pathalogical cases...
               */
              if ( (attempt > 4) && (attempt & 1) == (getpid() & 1) )
                {
                  errormsg("Sleeping for one second while "
                           "waiting for lock\n");
                  sleep(1);
                }
              break;
              
            default:            /* hard failure. */
              /* fall back on the less-safe method, which will
               * probably still fail
               */
              return open(pstr, flags, perms);
            }
        }
    }
  return -1;
}


-- 
James Youngman
Manchester, UK.  +44 161 226 7339
PGP (GPG) key ID for <jay@gnu.org> is 64A95EE5 (F1B83152).

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/