Subject: Re: inodes and log files



> If I understand correctly, the web server opens the file but
> uses the disk address of the file to know where to write the log
> info.  This disk address is in the inode.

sort of.. there are some other pieces between you and the inode itself.  
what programs actually use are 'file descriptors'.   a file descriptor
is simply an integer than that OS uses to look through its file tables
and find a 'vnode'.   the vnode contains information about the type of
file being used, and pointers to the functions which will support i/o
for that particular kind of filesystem.

assuming the file in question is sitting on your hard drive, the vnode
will also contain a copy of the inode.   so.. yes, the thing you're
ultimately dealing with is the inode, but you don't use it directly.



> If I delete that file and then re- create it then I am creating
> a new inode for this "new" file.

yep.. you don't even need the quotes around the word 'new'.   

the inode contains all the information about the physical storage used
by the file, but it *doesn't* contain the human-readable name of the
file.   that information lives in the data structure which defines the
contents of a directory.

at the data level, a directory is just a table of filenames and the
inodes they point to.   it is, in fact, perfectly legal for mutiple
directory entries to point to the same inode.. that's what hard linking
is all about.   the same physical data can be accessible from multiple
points in the filesystem, under as many different names as you want.

when you delete a file, you don't actually do anything to the file
storage, you just remove that entry from the directory's table.   that's
why the low-level routine isn't named 'delete', it's named 'unlink'.  
you also cause the inode's 'link count' to be reduced by one.   the link
count is what the OS uses to decide whether it should actually delete
the blocks associated with the file.   as long as the link count is
greater than zero, the blocks are safe.

the same thing happens when you move a file to a different directory, or
rename it.   the filesystem sets up a new name->inode mapping in the
appropriate directory, then deletes the old one.   you've changed the
way the data is identified in the file tree, but haven't changed the
physical storage at all.



> I have not deleted the old inode ... or did I?

not yet, per se.. you've just made it invisible to the filesystem, which
is the next best thing.

also, you don't actually 'delete' inodes.. the number of inodes in a
filesystem stays constant regardless of how many files are stored on the
disk.   what you do collect the list of physical blocks associated with
that inode and mark them as free for reallocation.   then you clear out
the data in the inode and mark that as free for reallocation as well.

it's a niggly point, but an important one.. the inode's number doesn't
change, even if it's been cleared and reallocated. it's the number which
is used in the directory, so a corrupted directory table with messed up
inode numbers can point to all sorts of wild places in the filesystem.



> If I did not then that means that the data must be written on
> the disk but to a space which is identified only by the "disk
> addresses" which leads me to believe that I could recover the
> data.

theoretically possible, but mechanically tricky.   the OS will preserve
an inode, even if its link count is zero, as long as there's still a
vnode in the file table which points to that inode.   OTOH, as soon as
the program terminates, the file descriptor closes, which eliminates the
vnode, and causes the inode to be cleared for reallocation.

(twig on the branch, and the branch on the tree, and the tree in the
bog, and the bog down in the valley, oh! ..  everybody sing!)

as long as the program keeps the file descriptor open, i suppose you
could use some sort of low-level filesystem tester to walk through the
inodes and see if it can find the one you want, but i don't know what
the name of that utility would be.   you'd probably have just as much
luck shutting down the system and scanning the physical blocks looking
for text which looks like log entries.



> Ummm ....  I think you would say that an inode is not deleted
> until the last application that was using it had closed.  Why
> would the webserver not lock the file?  It should prevent
> someone from changing the file while it was still using it.

that wouldn't work for a multithreaded webserver.   each of the child
processes needs to write to the log, so the kind of lock you're thinking
of would also shut them out.   instead, the processes use an atomic
'find the end of the file and add a line to it' operation.   that's
enough to keep them from tripping over each other's feet, without
cutting off anyone's air.



> Ummm.... restarting the web server must cause the creation of a
> new inode which describes the "human" file name.

nope.. the inode stays the same as long as the storage is allocated.  
what gets created is a new file descriptor which points to the inode
associated with that name in that directory.

if no file by that name exists, the OS will allocate a new inode and all
its associated baggage, but if there's already an inode mapped to the
right name, it will take what it finds.



> But what can you do to this log file that would cause the
> creation of a new inode to describe the file?

Wile E. Coyote syndrome.. since the name in the filepath and the inode
in the file descriptor only have a nodding acquaintance with each other,
you can unlink the old inode and link a new one in under the same name,
and everything will look just fine as far as the filesystem is
concerned.   meanwhile, the webserver is running happily over the abyss,
and will continue to do so until it looks down.

it's easy to duplicate the effect.. go to your log directory, and enter
the folowing sequence of commands:

    $ ls -i
    
      15506 access_log
    
    $ sort access_log >tmp
    
    $ ls -i
    
      15506 access_log    
      15507 tmp
    
    $ mv tmp access_log 
    
    $ ls -i
    
      15507 access_log


the '-i' option shows the inode associated with each file.   as you can
see, the original inode goes away, even though the name of the file at
the end is the same as the name of the file at the beginning.   if your
webserver was writing to inode 15506, it won't suddenly switch over to
inode 15507 just because it has the right name.